I am planning to deploy an application that is using LLM models. Currently, I use a Docker container to run a model through Ollama and I use the localhost IP address to access to my LLM. Now I want to deploy it on Azure. I may use DataBricks for my app. What would be the best deployment option for my app ? In my mind, I would deploy my Docker Image in an Azure VM (with SSD storage for my model and GPU for model inference). Where should I store my python app to have access to my ollama local API and being able to run it easily with some user inputs ? Thank you very much. Continue reading...
