[Python] Getting StopIteration or No Output while using HuggingFace InferenceClient with...

I'm trying to run text generation using the Hugging Face InferenceClient in Python, but I keep getting either a StopIteration error or no output at all. Here's my setup:

from huggingface_hub import InferenceClient
from dotenv import load_dotenv
import os

load_dotenv()

client = InferenceClient(token=os.getenv("HUGGINGFACEHUB_API_TOKEN"))

response = client.text_generation(
prompt="What is the capital of India?",
model="tiiuae/falcon-rw-1b", # Also tried with TinyLlama/TinyLlama-1.1B-Chat-v1.0
max_new_tokens=100,
temperature=0.5,
do_sample=True,
)

print(response)


What I Tried:

Verified that my .env is loading correctly (HUGGINGFACEHUB_API_TOKEN is being read properly).

Tried multiple models (gpt2, tiiuae/falcon-rw-1b, TinyLlama/TinyLlama-1.1B-Chat-v1.0).

Added exception handling, but still getting either empty response or internal error like StopIteration

Continue reading...