[Python] Creating a stoppable background task in Python

I want to run some function in the background, but be able to shut it down at will. I tried a bunch of solutions, the most obvious of which is asyncio tasks:

# Data source
DataSource = Generator[int, Any, None]
def create_data_source() -> DataSource
...

# Data processing - should be able to run in the background until being shut down
async def data_processor(source: DataSource) -> None:
try:
for item in source:
print("Working")

except Exception as e:
print("Exception", e)
raise

finally:
print("Cancelled")

async def main():
# Run the function
data_source = create_data_source()
processing_task = asyncio.create_task(data_processor(data_source))

# Do other stuff
await asyncio.sleep(3)

# Shut the function down
processing_task.cancel()

asyncio.run(main())


The problem is - once the task starts, the "Working" loop never terminates. I tried making the loop async, I tried sending it a shutdown signal using asyncio.Event, and I even tried rewriting the function using concurrent.futures, but nothing works. What am I missing?

Note:

1. I searched through previous Q&As, but haven't found a solution.
2. This is running on Python 3.11 for dependencies reasons, but if newer versions have the solution then I might be able to update.

Continue reading...