[Python] Why does thread disorder occur when the FastAPI multithread return generator?...

I want to implement a web service with streaming output using FastAPI. During the pressure test, it is found that for the same request, not the same thread is executed all the time. As a result, some intermediate variables stored in threadlocal are disordered. I have implemented a demo and pressure-tested it. May I ask why doesn't the stream return the same thread for the same request?

import logging
import threading
import time

import uvicorn
from fastapi import FastAPI
from sse_starlette import EventSourceResponse

app = FastAPI()


@app.get("/")
def stream_output():
logging.warning(f"{threading.current_thread().ident}")
return EventSourceResponse(num_generator(10), headers={"thread": str(threading.current_thread().ident)})


def num_generator(n):
for i in range(n):
logging.warning(f"{threading.currentThread().ident}: %s" % i)
time.sleep(2)
yield f"thread: {threading.current_thread().ident} num: {i}"
logging.warning(f"{threading.current_thread().ident}: end")


if __name__ == "__main__":
uvicorn.run(app, host='localhost', port=8000)


Pressure test result:



result of one of the requests:

data: thread: 39404 num: 0

data: thread: 39404 num: 1

data: thread: 57624 num: 2

data: thread: 39404 num: 3

data: thread: 52536 num: 4

data: thread: 39404 num: 5

data: thread: 52536 num: 6

data: thread: 39404 num: 7

data: thread: 39404 num: 8

data: thread: 52536 num: 9


I expect same thread for the same request

data: thread: 56052 num: 0

data: thread: 56052 num: 1

data: thread: 56052 num: 2

data: thread: 56052 num: 3

data: thread: 56052 num: 4

data: thread: 56052 num: 5

data: thread: 56052 num: 6

data: thread: 56052 num: 7

data: thread: 56052 num: 8

data: thread: 56052 num: 9

Continue reading...