[Python] How to set starting condition for airflow task group

Discussão em 'Python' iniciado por Stack, Agosto 4, 2025.

  1. Stack

    Stack Membro Participativo

    I have the following dag structure:

    from datetime import datetime, timedelta
    from airflow import DAG
    from airflow.utils.task_group import TaskGroup
    from airflow.providers.common.sql.operators.sql import SQLExecuteQueryOperator


    with DAG(
    'irb_test',
    description='irb_test',
    tags=["irb_test"],
    schedule_interval=None,
    start_date=datetime(2025, 7, 1),
    default_args={
    'retries': 0,
    'retry_delay': timedelta(minutes=1),
    'conn_id': 'sgk_gp'
    }
    ) as dag:

    with TaskGroup('test_1') as test_1:
    for i in range(15):
    task_1_i = SQLExecuteQueryOperator(
    task_id=f'task_1_{i}',
    sql=f"""
    select pg_sleep({i})
    """
    )
    with TaskGroup('test_2') as test_2:
    for i in range(15):
    task_2_i = SQLExecuteQueryOperator(
    task_id=f'task_2_{i}',
    sql=f"""
    select pg_sleep({i})
    """
    )

    for i in range(15):
    test_1.get_child_by_label(f'task_1_{i}')\
    >> test_2.get_child_by_label(f'task_2_{i}')


    What I would like to do is to start task group test_2 only after 10 of the tasks in the first task group have succeeded. I know that I can rewrite the execution order manually, but is there maybe a more elegant way of doing it?

    Continue reading...

Compartilhe esta Página