[Python] Ollama model keep in memory and prevent unloading between requests (keep_alive?)

No matter what I do from terminal or in code, the agent requests to ollama models take 15–25 seconds each time on my local M2 MacBook pro.

I am pretty sure this is not the hardware issue, because when used from terminal the model is lightning fast.

I am using python's module to load ollama gemma3.4b

So, my conclusion that model keeps unloading from memory each time and loads on new request.

I've looked for keep_alive option, and found that some say it defaults to -1. And it can be changes when running from cli, but I could not find docs on how to use it in python, even though there is, apparently, fixed bug on adding this feature.

https://github.com/ollama/ollama-python/pull/31

Help!

Continue reading...