[Python] Airflow tasks in sequence: dag file is parsed again?

Discussão em 'Python' iniciado por Stack, Setembro 11, 2025.

  1. Stack

    Stack Membro Participativo

    I'm writing a DAG script for Apache Airflow and I'm running into behaviour that I didn't expect. If you take a look at my example script I was expecting the same timestamp to be printed. Instead, when running this dag, the timestamp gets re-evaluated and prints the date-time of the second task.

    import pendulum

    from airflow.sdk import dag, task

    myTimeStamp = pendulum.now("Europe/Amsterdam").strftime('%Y%m%d_%H%M_%S')

    @dag(
    schedule="0/1 * * * *",
    catchup=False,
    start_date=pendulum.datetime(2025, 9, 2, tz="Europe/Amsterdam"),
    is_paused_upon_creation=False,
    tags=["ssh"],
    )
    def test_local(myDate: str):

    @task.bash()
    def test1(myDate1: str) -> str:
    return f'echo "Message: hi from 1! {myDate1}"; sleep 5;'

    @task.bash()
    def test2(myDate2: str) -> str:
    return f'echo "Message: hi from 2! {myDate2}"'

    test1(myDate) >> test2(myDate)

    test_local(myTimeStamp)


    As if the dag is parsed a second time when running the test2 task. Any tips on how to prevent this?

    Continue reading...

Compartilhe esta Página