[Python] Celery - Running subtasks inside group of chains

Discussão em 'Python' iniciado por Stack, Abril 22, 2025.

  1. Stack

    Stack Membro Participativo

    Hope all is well!

    Issue: I am attempting to run a group of child tasks inside of a chain, then in the parent most level I have a group wrapped around the chains. The child tasks are being executed; however, the results from the parent-most group primitive finishes before the child tasks can spit back their results.

    Example

    --- I have 3 tasks below ---

    @app.task
    def task_one(seconds):
    time.sleep(seconds)
    return [seconds + 0.1, seconds + 0.2]

    @app.task
    def task_two(seconds):
    return group(task_three.signature(args=(x,)) for x in seconds)()

    @app.task
    def task_three(seconds):
    time.sleep(seconds)
    return {
    'sleep': seconds
    }


    ---The tasks are called like so ---

    g = group(
    chain(
    task_one.s(x),
    task_two.s()
    ) for x in range(1, 3)
    )
    promise = g.apply_async()
    print(promise.get())


    The function task_two will create a group of tasks (task_three) to execute. When ran, the outermost group will finish execution and return the results BEFORE the task_three tasks finish.

    Any idea why this is happening?

    Continue reading...

Compartilhe esta Página