[Python] Is a lock needed when multiple tasks push into the same asyncio Queue?

Discussão em 'Python' iniciado por Stack, Abril 15, 2025.

  1. Stack

    Stack Membro Participativo

    Consider this example where I have 3 worker tasks that push results in a queue and a tasks that deals with the pushed data.


    async def worker1(queue: asyncio.Queue):
    while True:
    res = await do_some_work(param=1)

    await queue.put(res)

    async def worker2(queue: asyncio.Queue):
    while True:
    res = await do_some_work(param=2)

    await queue.put(res)

    async def worker3(queue: asyncio.Queue):
    while True:
    res = await do_some_work(param=3)

    await queue.put(res)

    async def handle_results(queue: asyncio.Queue):

    while True:
    res = await queue.get()
    await handle_result(res)
    queue.task_done()

    async def main():
    queue = asyncio.Queue()
    t1 = asyncio.create_task(worker1(queue))
    t2 = asyncio.create_task(worker2(queue))
    t3 = asyncio.create_task(worker3(queue))

    handler = asyncio.create_task(handle_result(queue))

    while True:
    # do some other stuff

    ....

    asyncio.run(main())



    The documentation says that asyncio.Queue is not thread-safe, but this should not apply here because all tasks are running in the same thread. But do I need an asyncio.Lock to protect the queue when I have 3 tasks that push into the same queue? Looking at the implementation in Python 3.12 (which creates a putter future and awaits on it before pushing into the queue) I would say no, but I'm not sure and the documentation does not mention what would happen in this case. So, is the asyncio.Lock in this case necessary?

    Continue reading...

Compartilhe esta Página