[Python] How to transform a S3 directory as os.Path in Python with boto3?

I want to use ImageFolder from Torchvision with an AWS S3 bucket where I have my dataset saved, however, it requires a os.Path like file and with boto3 I can just get a directory or a list which is not allowed by ImageFolder. Do you know how can I get a S3 directory as os.Path in Python with boto3 or a similar strategy?

I'm loading my dataset as follows:

# Initialize S3 client
s3 = boto3.client('s3')
response_test = s3.list_objects_v2(Bucket=BUCKET_NAME_test, Prefix=PREFIX_test)
jpg_files_test = [item['Key'] for item in response_test.get('Contents', []) if item['Key'].lower().endswith('.jpeg')]


And using this code to load the data with Torchvision:

transform_train = transforms.Compose([
transforms.RandomResizedCrop(args.input_size, scale=(0.2, 1.0), interpolation=3), # 3 is bicubic
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(mean=[0.10231429, 0.18161748, 0.26542443], std=[0.06512465, 0.10374635, 0.16232868])])
dataset_train = datasets.ImageFolder(os.path.join(args.data_path, 'train'), transform=transform_train) # 'train'
print(dataset_train)


But as it's expected I get this error:

TypeError: expected str, bytes or os.PathLike object, not list

Continue reading...