I have a self-hosted Minio. Problem: Unable to download .csv and .jpg files. Able to download .txt and .md files. This is strange and I have been bashing my head in. My bucket policies are extremely simple, pretty much open to any operations. No where in my minio setup/settings, I do not have file type restrictions. I am able to download any file type through the web portal. Unable to download the .csv, .jpg, and .tar files programmatically. I tried manually uploading the files via the web portal and then running it programmatically, doesnt work. Only works for .md and .txt files. I tried programmatically uploading and then downloading from the same access key, doesnt work. Only works for .md and .txt files. I am at my wits end. Been at this for hours, cant take it anymore. Exhausted all of chatgpt help. Stackoverflow is my last attempt - trust me, I would rather be elsewhere. Code: class MinioUploader: def __init__(self): self.minioClient = Minio( endpoint="myminio.website.com", region="us-east-1", access_key="myaccesskeywhichiwillnotputhere", secret_key="mysecretketywhichiwillnotputhere", secure=True ) def download_all_files(self, bucket_name, local_path="./storage/local_storage/"): if self.minioClient is None: logs_sys.error("Minio client is not initialized.") return if not self.minioClient.bucket_exists(bucket_name): logs_sys.error(f"Bucket: {bucket_name} does not exist") return local_path = os.path.join(local_path, bucket_name) if not os.path.exists(local_path): os.makedirs(local_path) print(f"local_path: {local_path}") objects = self.minioClient.list_objects(bucket_name, recursive=True) for obj in objects: try: print(obj.object_name) # Decode any percent-encoded characters in the object name decoded_object_name = unquote(obj.object_name) # Create a safe, absolute file path safe_local_path = os.path.join(local_path, *decoded_object_name.split('/')) # Ensure the directory for the file exists safe_local_path = safe_local_path.replace("\\", "/") os.makedirs(os.path.dirname(safe_local_path), exist_ok=True) # Download the object print(f"Downloading {obj.object_name} to {safe_local_path}") self.minioClient.fget_object(bucket_name=bucket_name, object_name=obj.object_name, file_path=safe_local_path) except Exception as e: logs_sys.error(f"Error: {e}") print(f"Error: {e}") Print Out: D:\TRADING_RESTORE_STRATEGIES>python -u "d:\TRADING_RESTORE_STRATEGIES\storage\minio_s3\minio_client.py" local_path: ./storage/local_storage/testdownloadbucket 11.md Downloading 11.md to ./storage/local_storage/testdownloadbucket/11.md 3-7-2024 3-11-49 AM.jpg Downloading 3-7-2024 3-11-49 AM.jpg to ./storage/local_storage/testdownloadbucket/3-7-2024 3-11-49 AM.jpg Error: S3 operation failed; code: AccessDenied, message: Access denied, resource: /testdownloadbucket/3-7-2024%203-11-49%20AM.jpg, request_id: 17BACF23766A4250, host_id: dd9025af9251148b658df7ac2e3e8, bucket_name: testdownloadbucket, object_name: 3- 7-2024 3-11-49 AM.jpg asad.txt Downloading asad.txt to ./storage/local_storage/testdownloadbucket/asad.txt new.csv Downloading new.csv to ./storage/local_storage/testdownloadbucket/new.csv Error: S3 operation failed; code: AccessDenied, message: Access denied, resource: /testdownloadbucket/new.csv, request_id: 17BACF23839DECB7, h ost_id: dd9025af9251148b658df7ac2e3e8, bucket_name: testdownloadbucket, object_name: new.csv something1.csv Downloading something1.csv to ./storage/local_storage/testdownloadbucket/something1.csv Error: S3 operation failed; code: AccessDenied, message: Access denied, resource: /testdownloadbucket/something1.csv, request_id: 17BACF238744 C16C, host_id: dd9025af9251148b658df7ac2e3e8, bucket_name: testdownloadbucket, object_name: something1.csv test.txt Downloading test.txt to ./storage/local_storage/testdownloadbucket/test.txt <minio.api.Minio object at 0x00000248C9B0B130> Continue reading...
