[Python] Is Cooperative Multitasking with Asyncio Useful in Python if My Whole Stack Does not...

So, recently I read the documentations and some tutorials on python asyncio, and thinking how to use it for production to speed up performance.

But it seems to use it effectively, every I/O must be non-blocking, which means every I/O bound task must have an async interface? As an example, the httpx library seems to offer an async client, which is great. But my typical projects (as a machine learning engineer) also involve a lot of

• model training and inference using tensorflow
• query and insert data from databases, using polars.read_database or polars.write_database and SQL alchemy
• reading and dumping parquet files (locally, via juicefs or to MinIO buckets)

So does it mean I need async interfaces to each of them to await the I/O calls? Do I typically have to find libraries that do that? Or can asyncio itself make those functions async so that they can be awaited? Or, is it up to those library developers to support asyncio for long running calls?

Continue reading...