I am using a CI/CD pipeline in combination with git-lfs. At the beginning of my script, I want to load a pickle file (see code further below) that is stored via LFS. My code runs fine on the local machine, but in the pipeline I get the following error: _pickle.UnpicklingError: invalid load key, 'v' It seems that this error is caused by the LFS file not being fetched and can be resolved with a simple git lfs pull command. The question now is, how can I call git lfs pull in the pipeline? I naively tried adding it in the .yml file where I call my python scripts too, but that didn't work. I'm sorry if is this question is a bit stupid, I'm a bit outside of my normal field here. EDIT with some details on when the error occurs and the CI/CD used: These are the lines that cause the error to occur: open("path_to_file/file", 'rb') data = pickle.load(f) I am using a Gitlab CI/CD and the .yml file defining the workflow is the following: before_script: - rm -rf "*-env"; python3.8 -m venv ${CI_JOB_ID}-env - ${CI_JOB_ID}-env/bin/pip3 install -U pip after_script: - rm -rf ${CI_JOB_ID}-env stages: - calculations CalcStrategicValues: stage: calculations only: - calculate script: - ${CI_JOB_ID}-env/bin/pip3 install -r requirements.txt # - ${CI_JOB_ID}-env/git lfs pull <- my attempt at solving the problem - ${CI_JOB_ID}-env/bin/python3 calculate.py allow_failure: false artifacts: name: results when: always paths: - output/ Continue reading...
