I wish to create something similar to a context switching mechanism, that allows using a shared resource one at a time. In my case it's a single session object connecting to a website, and each context would be the current account connected to the website. As a system constraint - no two accounts can be connected at the same time, and the connection is a costly operation. I have devised the following mechanism which involves letting go of the context (switching account) during expensive background operations: counter = 0 # i.e. ContextVar for current account def increase_counter(): global counter counter += 1 def decrease_counter(): global counter counter -= 1 async def run_operation(): while True: operation = operation_queue.get() increase_counter() task = asyncio.create_task(operation()) task.add_done_callback(lambda fut: decrease_counter()) await wait_for_free() # wait until counter == 0 switch_context() # Log in to a different account async def operation(): # do stuff decrease_counter() sleep(60) # Long background operation await wait_for_context() # Wait for context to come back and be 1. # continue When there is only one chain of operations, this mechanism works well. Inner functions can always release the counter and take it back after the operation. Unfortunately, when there are two operations running at the same time, it stops working: async def operation(): increase_counter() task1 = asyncio.create_task(sub_op()) task1.add_done_callback(decrease) increase_counter() task2 = asyncio.create_task(sub_op()) task2.add_done_callback(decrease) decrease_counter() await asyncio.gather(task1, task2) await wait_for_context() async def sub_op(): decrease_counter() sleep(60) await wait_for_context() The counter in that case is advanced as follows: run_op (+1 = 1) task1_creation (+1 = 2) task2_creation (+1 = 3) gather_release (-1 = 2) task1_sleep (-1 = 1) task2_sleep (-1 = 0) # Context released task1_resume (+1 = 1) task2_resume (+1 = 2) task1_done_callback (-1 = 1) task2_done_callback (-1 = 0) # Context needlessly released gather_resume (+1 = 1) Is it possible to prevent the needless release or is that just a fact of life when using such a mechanism? If so, is there any other mechanism that can solve the issue besides using a counter? Continue reading...
