I am trying to generate data based on previous texts via LLMs, and therefore making consecutive calls for all texts I have available. I have done this so far with OpenAI's and Ollama's Python libraries, and this worked very well. Since I do not have lots of computational power to run many Ollama models, I wanted to try the Llama-API. For some reason, though, the script always stops working after processing the second document, i.e. it remains stuck in a request for 1-2 minutes and then returns a timeout. I am setting up and calling following function in a loop: client = OpenAI(api_key=os.getenv("LLAMA_API_KEY"), base_url = "https://api.llama-api.com") def get_generated_text(...): response = client.chat.completions.create(model="llama3.3-70b", messages=[ {"role": "system", "content": "..."}, {"role": "user", "content": "..."} ], top_p=0.95) return response.choices[0].message.content Has anyone else had this problem before, or are there certain best practices I have not read in the documentation? All help greatly appreciated, thanks so much! Continue reading...
