[Python] How to run `ls` and `glob` asynchronously with `s3fs.S3FileSystem`?

I am trying to use s3fs.S3FileSystem with asynchronous operations.

I can successfully read and write files asynchronously using:

import s3fs

bucket_client = s3fs.S3FileSystem(asynchronous=True)

async with await bucket_client.open_async("path/to/file.xml", "rb") as f:
xml = await f.read()


However, I am having trouble executing ls and glob operations asynchronously.

Issue 1: ls raises RuntimeError


When I try:

await bucket_client.ls("path/to/folder")


I get the following error:

RuntimeError: Loop is not running

Issue 2: ls raises TypeError


I also attempted using an async context manager:

async with bucket_client:
bucket_client.ls("path/to/folder/**/*.xml")


But this raises:

TypeError: 'S3FileSystem' object does not support the asynchronous context manager protocol


Question:

• How can I properly run ls and glob asynchronously with s3fs.S3FileSystem?
• Is there a correct way to integrate this within an async workflow without running into event loop issues?

Continue reading...