[Python] If we want use S3 to host Python packages, how can we tell pip where to find the...

we are trying to come up with a solution to have AWS S3 to host and distribute our Python packages.

Basically what we want to do is using python3 setup.py bdist_wheel to create a wheel. Upload it to S3. Then any server or any machine can do pip install $http://path/on/s3. (including a virtualenv in AWS lambda) (We've looked into Pypicloud and thought it's an overkill.)

Creating package and installing from S3 work fine. There is only one issue here: we will release new code and give them different versions. If we host our code on Pypi, you can upgrade some packages to their newest version by calling pip install package --upgrade.

But if you host your packages on S3, how do you let pip know there's a newer version exists? How do you roll back to an older version by simply giving pip the version number? Is there a way to let pip know where to look for different version of wheels on S3?

Continue reading...