[Python] Facing issue using a model hosted on HuggingFace Server and talking to it using...

I am trying to create a simple langchain app on text-generation using API to communicate with models on HuggingFace servers.

I created a “.env” file and stored by KEY in the variable: “HUGGINGFACEHUB_API_TOKEN” I also checked it, API token is valid.

Post that, I tried running this code snippet:

from langchain_huggingface import ChatHuggingFace, HuggingFaceEndpoint
from dotenv import load_dotenv

load_dotenv()

llm = HuggingFaceEndpoint(
repo_id="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
task="text-generation"
)

model = ChatHuggingFace(llm=llm)
result = model.invoke("What is the capital of India")
print(result.content)


This is giving an error. I tried multiple things around it, but nothing worked.

Here is the error log: Traceback (most recent call last): File “C:\Users\SS\Desktop\Camp_langchain_models\2.ChatModels\2_chatmodel_hf_api.py”, line 13, in result = model.invoke(“What is the capital of India”) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “C:\Users\SS\Desktop\Camp_langchain_models\venv\Lib\site-packages\langchain_core\language_models\chat_models.py”, line 370, in invoke self.generate_prompt( File “C:\Users\SS\Desktop\Camp_langchain_models\venv\Lib\site-packages\langchain_core\language_models\chat_models.py”, line 947, in generate_prompt return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “C:\Users\SS\Desktop\Camp_langchain_models\venv\Lib\site-packages\langchain_core\language_models\chat_models.py”, line 766, in generate self._generate_with_cache( File “C:\Users\SS\Desktop\Camp_langchain_models\venv\Lib\site-packages\langchain_core\language_models\chat_models.py”, line 1012, in _generate_with_cache result = self._generate( ^^^^^^^^^^^^^^^ File “C:\Users\SS\Desktop\Camp_langchain_models\venv\Lib\site-packages\langchain_huggingface\chat_models\huggingface.py”, line 574, in generate answer = self.llm.client.chat_completion(messages=message_dicts, **params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “C:\Users\SS\Desktop\Camp_langchain_models\venv\Lib\site-packages\huggingface_hub\inference_client.py”, line 886, in chat_completion provider_helper = get_provider_helper( ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\SS\Desktop\Camp_langchain_models\venv\Lib\site-packages\huggingface_hub\inference_providers_init.py", line 165, in get_provider_helper provider = next(iter(provider_mapping)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration

I am new to it. Any guidance around this is much appreciated. Thank you.

Continue reading...