[Python] is this right way to make multiple calls to llama 3.1 model?

Discussão em 'Python' iniciado por Stack, Junho 17, 2025.

  1. Stack

    Stack Membro Participativo

    def safe_invoke(llm, messages, retries=5, backoff_base=1.5):
    for attempt in range(retries):
    try:
    return llm.invoke(messages)
    except Exception as e:
    if "429" in str(e):
    sleep_time = backoff_base * (2 ** attempt) + random.uniform(0, 1)
    print(f"[Rate Limit] Sleeping for {sleep_time:.2f} seconds (attempt {attempt + 1})...")
    time.sleep(sleep_time)
    else:
    raise e
    raise Exception("Max retries exceeded due to rate limiting.")

    Continue reading...

Compartilhe esta Página