[Python] Deployment failed zip file exceeds 50MB/1GB limit due to model size and dependencies

I am stuck while deploying my model using an S3 bucket. The issue is that my zip file exceeds 50 MB and sometimes even reaches GBs in size.

Could you please provide a solution for this? If possible, a video explanation would be really helpful

Note - I want to deploy with S3 bucket

I am trying to deploy our project to an S3 bucket. The code uses a few libraries including:

• pandas
• PyMuPDF
• tabula-py

To deploy the project, I packaged the code and dependencies into a zip file. However, the resulting zip file often exceeds 1 GB in size.

Due to this, I'm unable to upload the zip to S3, as it exceeds the upload limit or becomes inefficient to manage.

How to deploy the project with minimal package size or an alternative solution that allows me to:

• Efficiently handle large dependencies
• Keep the deployment package within a reasonable size
• Successfully deploy and run the code without hitting S3 or platform-specific size limits

Continue reading...