[Python] UnpicklingError from Azure Blob

I'm developing an Azure Function to serve inferences from a machine learning model. The model is saved as a .pkl file on Azure Blob Storage. All attempts read the file into Python fail.

For simplicity, this example us an arbitrary .pkl instead of a model file.

Object Creation Code


import pickle

# Create a pickle
small_object = list(range(0,1000))
name = 'small_file.pkl'

with open(name, 'wb') as f:
pickle.dump(small_object, f)

#### Verify integrity ###

# Loading from disk
with open(name, 'rb') as f:
opened_object_one = pickle.load(f)
print(opened_object_one) # prints expected list

# Loading from byte stream
with open('small_object.pkl', 'rb') as f:
opened_object_two = f.read()
print(pickle.loads(opened_object_two)) # print expected list


Azure Function Code


import logging
import pickle

app = func.FunctionApp()

@app.route(route="http_trigger", auth_level=func.AuthLevel.FUNCTION)
@app.blob_input(
arg_name = 'client',
path = '<container>/small_file.pkl',
connection ='AzureWebJobsStorage'
)
def http_trigger(req: func.HttpRequest, client) -> func.HttpResponse:

# `client` is of type azure.functions.blob.InputStream
package_bytes = client.read() # return bytes
client.close()

small_object = pickle.loads(package_bytes)


I enter my virtual environment with source .venv/scripts/activate (I'm using GitBash). Then I run locally with func host start and trigger the function by following the link to the relevant localhost endpoint. This returns Exception: UnpicklingError: invalid load key, '\xef' in the console.

I've tried using joblib, dill and pickle for both read and write stages. I've tried loading the pickle to the blob with both the Azure CLI and with Firefox. I always get the same results.

I'm on Windows 10 using Python 3.10.5.

All help is appreciated.

Continue reading...