[Python] Airflolw Bash operator not passing xcom to another operator

I'm working on a task group that needs to pass a variable from a BashOperator to another BashOperator. Each bash operator is invoking python, and the first python script needs to return a string in a variable that the second bash operator will take and continue with.

first_status = BashOperator(
task_id=1st_taskid_str,
bash_command=f"python myscript.py --dag_id '{dag.dag_id}' \
--task_id '{1st_taskid_str}' --dag_conf configuration_string",
retries=10,
dag=dag,
retry_delay=timedelta(minutes=1),
do_xcom_push=True,
)

#step 2

second_status = BashOperator(
task_id=process_file_task_id
,bash_command=f"python secondscript.py --dag_id '{dag.dag_id}' \
--task_id '{2nd_task_id}' --dag_conf configuration_string --file '{{ ti.xcom_pull(task_ids=\"{1st_taskid_str}\") }}'"
,dag=dag
)

first_status >> second_status


When I view the xcom from the first_status task, i'm not seeing the variable that is logged by the python script invoked.

How can I get this variable passed from the first bash operator to the second bash operator?

Continue reading...