[Python] Is it possible to call another Function from a Function called by a...

I have my example DAG which I am testing on Airflow 1.10.0. When I use the PythonVirtualenvOperator with a python callable as the following example:

def python_virtual_env_callable():
import requests
#some code which uses requests library...

....
with dag:
test_task = PythonVirtualenvOperator(
task_id='test_task',
python_callable=python_virtual_env_callable,
op_kwargs=None,
requirements=['requests'],
python_version='3.6',
system_site_packages=False
)

start_task = DummyOperator(task_id='start')
end_task = DummyOperator(task_id='end')

start_task >> test_task >> end_task


the DAG works fine, and I get no errors. The issue arises when I try to call another function from the python_virtual_env_callable function, as shown in the following example:

def some_secondary_function():
import pandas
# some code with pandas

def python_virtual_env_callable():
import requests
#some code which uses requests library ...
some_secondary_function() #call to another function


....
with dag:
test_task = PythonVirtualenvOperator(
task_id='test_task',
python_callable=python_virtual_env_callable,
op_kwargs=None,
requirements=['requests','pandas'],
python_version='3.6',
system_site_packages=False
)

start_task = DummyOperator(task_id='start')
end_task = DummyOperator(task_id='end')

start_task >> test_task >> end_task


in this case I get: "ERROR - Command '['/tmp/venvndxo9eth/bin/python', '/tmp/venvndxo9eth/script.py', '/tmp/venvndxo9eth/script.in', '/tmp/venvndxo9eth/script.out', '/tmp/venvndxo9eth/string_args.txt']' returned non-zero exit status 1."

My question then is: Is it not possible to call a function from another function called by the PythonVirtualenvOperator? The same DAG with just the PythonOperator works perfectly fine.

Thank you in advance for your help.

Continue reading...