[Python] How to ensure hash of zip file created locally is identical to when created during...

How can I use shutil.make_archive() to make zip files that are absolutely identical, including the hash?

I am creating a zip file locally, and then through CI/CD, and I expect them to be identical, but the hashes differ. This causes CI/CD to think that the files are different and it attempts an update, which is undesirable as this would result in an update being made of every occasion, even when no change has been made.

The following precautions have been taken to ensure the zipped packages are identical, however I'm clearly missing something.

• The directory where packages will be installed is first removed, to ensure a clean packages directory.
• Packages are installed by the root user.
• All __pycache__ directories are deleted.
• The last accessed and modified times of each file is changed to ensure they match.

Is there something more (or different) I can do to make sure the hashes of the zip files will always be the same, given the contents is exactly the same?

Please note that it is Terraform generating a hash of the zip file, using filebase64sha256("packages.zip"). I don't believe this makes a difference, but thought it worth mentioning anyway.

#!/bin/bash

REQUIREMENTS_FILE="requirements.txt"
REQUIREMENTS_DIR="packages/"
REQUIREMENTS_ZIP="packages.zip"

rm -rf $REQUIREMENTS_DIR
pip install --upgrade --force-reinstall --no-cache-dir -r $REQUIREMENTS_FILE -t $REQUIREMENTS_DIR
find $REQUIREMENTS_DIR | grep -E "(__pycache__|\.pyc$)" | xargs rm -rf
find $REQUIREMENTS_DIR -exec touch -t 200010101010 {} +

python -c "import shutil; shutil.make_archive('${REQUIREMENTS_ZIP:0:-4}', 'zip', '$REQUIREMENTS_DIR')"

Continue reading...