[Python] OpenAi token limit with Fastapi

What are the best practices to stream an openai response with Fastapi? How do you handle token limits and the related 429 error? And all the other errors I'm probably not thinking about?

Code snippet just for reference

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from openai import OpenAI

app = FastAPI()
client = OpenAI()

@app.get("/chat")
async def chat():
def generate():
with client.chat.completions.stream(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}]
) as stream:
for event in stream:
if event.type == "token":
yield event.text
return StreamingResponse(generate(), media_type="text/plain")

Continue reading...