I am new to running models locally. I was very happy that I could run a gemma2b model locally which I pulled using ollama. I am using (or loading?) this local model in my python application by using chatOllama as below llm = ChatOllama(model="gemma2") But now I came to a stage where I want to fine tune this locally running model. I can see the model is saved in ~/.ollama/models/blobs as sha256. But I understood from various reddit posts, chatgpt and general web browsing that I cannot fine tune the model with ollama but I should somehow fine tune this outside of ollama and then import it back to ollama to use the inference with fine tuned model. is this understanding correct? If yes, How do I actually take the model out of ollama to train and put it back? I understand GGUF is a model format that I can train. but how to get the GGUF for Gemma2 that is stored locally (in my case) I am planning to use LoRA to train as I understood from some articles that it is efficient and uses less resources. Continue reading...
