[Python] Unexpected Memory Leak in Python's asyncio When Using contextvars with Nested...

I'm encountering a subtle memory leak in a Python 3.12 application using asyncio, specifically when combining contextvars with deeply nested TaskGroup structures. The issue only appears under high concurrency and long runtime conditions.

Here’s a simplified version of the pattern:

import asyncio
import contextvars

user_context = contextvars.ContextVar("user_context")

async def worker():
user_context.set("user123")
await asyncio.sleep(0.1)

async def nested_group():
async with asyncio.TaskGroup() as tg:
for _ in range(100):
tg.create_task(worker())

async def main():
for _ in range(1000):
await nested_group()

asyncio.run(main())



After running this for a while, memory usage steadily increases and never drops, even after all tasks complete. Profiling shows that contextvars are not being garbage collected as expected. I’ve ruled out circular references and confirmed that user_context is not being accessed outside the scope of each task.

What I’ve tried:

1. Explicitly resetting the context variable after use.
2. Using gc.collect() to force garbage collection.
3. Replacing TaskGroup with manual task management — the leak disappears.

Questions:

1. Is there a known issue with contextvars and TaskGroup interaction in Python 3.12?
2. Could TaskGroup be retaining references to context states even after completion?
3. Are there best practices for isolating context in high-concurrency async environments?

Any insights or suggestions would be greatly appreciated. This seems to be a rare edge case, and I haven’t found any related discussions or bug reports.

Continue reading...