[Python] Getting StopIteration when using HuggingFaceEndpoint with LangChain and flan-t5-large

I'm trying to use the langchain_huggingface.HuggingFaceEndpoint integration to call the "google/flan-t5-large" model from Hugging Face in a LangChain pipeline. Here's my code:

from langchain.prompts import PromptTemplate
from langchain_huggingface import HuggingFaceEndpoint
from langchain_core.runnables import RunnableSequence
import os

# Set your API token
os.environ['HUGGINGFACEHUB_API_TOKEN'] = 'hf_***************'

# Initialize the updated Hugging Face model
flan_t5 = HuggingFaceEndpoint(
repo_id="google/flan-t5-large",
temperature=1e-10,
task="text2text-generation"
)

# Define the prompt template
template = "Translate the following to French:\n\n{question}"
prompt = PromptTemplate(template=template, input_variables=["question"])

# Create the RunnableSequence chain
chain = prompt | flan_t5

# Run the chain with input
question = "Hello"
response = chain.invoke({"question": question})

print("Output:", response)


However, I'm getting the following error:

/usr/local/lib/python3.11/dist-packages/huggingface_hub/inference/_providers/__init__.py in get_provider_helper(provider, task, model)
189 raise ValueError("Specifying a model is required when provider is 'auto'")
190 provider_mapping = _fetch_inference_provider_mapping(model)
--> 191 provider = next(iter(provider_mapping)).provider
StopIteration


What I've tried:

• Verified that my Hugging Face token is valid.
• Checked that "google/flan-t5-large" exists and supports text2text-generation.
• Tried other repo IDs (e.g., google/flan-t5-xl) but ran into the same issue.

Question:

• Why am I getting this StopIteration error, and how can I fix it?
• Is there something wrong with how I'm initializing the HuggingFaceEndpoint or how the provider is being inferred?

Continue reading...