[Python] multiprocessing queue full

Discussão em 'Python' iniciado por Stack, Maio 25, 2025.

  1. Stack

    Stack Membro Participativo

    I'm using concurrent.futures to implement multiprocessing. I am getting a queue.Full error, which is odd because I am only assigning 10 jobs.

    import numpy as np
    from concurrent.futures import ProcessPoolExecutor
    A_list = [np.random.rand(2000, 2000) for i in range(10)]

    with ProcessPoolExecutor() as pool:
    pool.map(np.linalg.svd, A_list)


    error:

    Exception in thread Thread-9:
    Traceback (most recent call last):
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 921, in _bootstrap_inner
    self.run()
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 869, in run
    self._target(*self._args, **self._kwargs)
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/process.py", line 251, in _queue_management_worker
    shutdown_worker()
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/process.py", line 209, in shutdown_worker
    call_queue.put_nowait(None)
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/queues.py", line 131, in put_nowait
    return self.put(obj, False)
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/queues.py", line 82, in put
    raise Full
    queue.Full

    Continue reading...

Compartilhe esta Página