[Python] Azure ML Endpoint Fails with HFValidationError even after using pathlib.Path

I am trying to deploy a fine-tuned Mistral-7B model on an Azure ML Online Endpoint. The deployment repeatedly fails during the init() phase of the scoring script with an huggingface_hub.errors.HFValidationError, even after applying the standard fix for this issue.

My setup consists of a quantized base model, a PEFT adapter, and a sentence-transformer, all stored locally in the model asset.

Initial Attempt: Using String Paths My first scoring script used os.path.join to create string paths to the models.

# This version used os.path.join and failed
import os
from transformers import AutoTokenizer, AutoModelForCausalLM, GPTQConfig
# ... other imports

def init():
model_dir = os.environ.get('AZUREML_MODEL_DIR')
adapter_model_path_str = os.path.join(model_dir, 'mistral-finetuned-website-v1')

# This line failed
tokenizer = AutoTokenizer.from_pretrained(adapter_model_path_str, local_files_only=True)
# ... rest of the loading logic


This produced the following error, suggesting the local path string was being misinterpreted as a Hub repository ID.

huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/var/azureml-app/azureml-models/mistral-package/1/mistral-finetuned-website-v1'.


Second Attempt: Using pathlib.Path Based on the initial error, the standard recommendation is to use pathlib.Path to make the local path unambiguous to the transformers library. I updated my script accordingly.

This is my current scoring script:

import os
import torch
import json
from pathlib import Path
from transformers import AutoTokenizer, AutoModelForCausalLM, GPTQConfig
from peft import PeftModel
from sentence_transformers import SentenceTransformer, util

# FALLBACK_MESSAGE and KNOWN_TOPICS are defined here...

def init():
global model, tokenizer, embedding_model, known_topics_embeddings
try:
model_dir = Path(os.environ.get('AZUREML_MODEL_DIR'))

base_model_path = model_dir / 'mistral-7b-combined-model'
adapter_model_path = model_dir / 'mistral-finetuned-website-v1'
embedding_model_path = model_dir / 'all-MiniLM-L6-v2'

quantization_config = GPTQConfig(bits=4, use_exllama=False)

print(f"Loading tokenizer from: {adapter_model_path}")
# This line is still failing
tokenizer = AutoTokenizer.from_pretrained(adapter_model_path, local_files_only=True)

print(f"Loading base model from: {base_model_path}")
base_model = AutoModelForCausalLM.from_pretrained(
base_model_path,
device_map="cpu",
quantization_config=quantization_config,
local_files_only=True
)

print(f"Loading fine-tuned PEFT adapter from: {adapter_model_path}")
model = PeftModel.from_pretrained(base_model, adapter_model_path)

print("Merging base model and adapter...")
model = model.merge_and_unload()
model.eval()

print("Loading SentenceTransformer model...")
embedding_model = SentenceTransformer(str(embedding_model_path))

print("Computing embeddings for known topics...")
known_topics_embeddings = embedding_model.encode(KNOWN_TOPICS, convert_to_tensor=True)

print("All models and embeddings loaded successfully.")
except Exception as e:
print(f"Error during initialization: {e}")
raise

def run(raw_data):
# (Full run function logic is included here)
pass


The Problem: Despite this change, the deployment failed again with the exact same error and traceback. The pathlib object is still being treated as a string for repository validation.

2025-09-12 04:37:12,668 I [73] azmlinfsrv.print - Loading tokenizer from: /var/azureml-app/azureml-models/mistral-package/1/mistral-finetuned-website-v1
2025-09-12 04:37:12,668 I [73] azmlinfsrv.print - Error during initialization: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/var/azureml-app/azureml-models/mistral-package/1/mistral-finetuned-website-v1'. Use `repo_type` argument if needed.
2025-09-12 04:37:12,668 E [73] azmlinfsrv - User's init function failed
...
Traceback (most recent call last):
File "/azureml-envs/.../site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/var/azureml-app/azureml-models/mistral-package/1/mistral-finetuned-website-v1'.
...


Environment and Dependencies: This is my conda.yml. I am targeting a CPU-only deployment. Note: Per project requirements, the deployment target is a CPU-only compute instance. Therefore, GPU-based solutions are out of scope for this task.

channels:
- conda-forge
- pytorch
- defaults
dependencies:
- python=3.10
- pip=23.3.1
- pytorch=2.0.1
- torchvision=0.15.2
- torchaudio=2.0.2
- cpuonly
- scikit-learn
- scipy
- pip:
- accelerate==0.21.0
- auto-gptq
- azureml-inference-server-http==1.5.0
- huggingface-hub
- peft==0.5.0
- sentence-transformers==2.6.1
- transformers>=4.34.0
# ... other packages


However, the logs from the failed deployment show that the installed environment has GPU packages and a different PyTorch version.

...
torch==2.8.0
nvidia-cublas-cu12==12.8.4.1
nvidia-cuda-cupti-cu12==12.8.90
nvidia-cuda-nvrtc-cu12==12.8.93
nvidia-cuda-runtime-cu12==12.8.90
nvidia-cudnn-cu12==9.10.2.21
transformers==4.56.1
... (full list)


This discrepancy between my requested cpuonly environment and the actual GPU-centric environment seems like a potential root cause for instability.

My Questions:

1. Why is the pathlib.Path solution failing here? Is there a known issue in certain transformers/huggingface-hub versions or Azure ML environments where this standard fix doesn't work?
2. How significant is the environment mismatch? Could the conflict between a requested cpuonly build and the presence of nvidia-* packages cause underlying instability that manifests as this HFValidationError?
3. What is the recommended next step to debug this? Should I focus on forcing a true CPU-only environment build, or is there another way to load the local models that might be more robust?

Continue reading...