[Python] Deepseek huggingface model loading issue

I am using this code from huggingface:

This code is directly pasted from the HuggingFace website's page on deepseek and is supposed to be plug-and-play code:

from transformers import pipeline

messages = [
{"role": "user", "content": "Who are you?"},
]
pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-R1", trust_remote_code=True)
pipe(messages)


But I'm unable to load the model. When I do, I get this issue:

File "<...>/site-packages/transformers/quantizers/auto.py", line 97, in from_dict
raise ValueError(

ValueError: Unknown quantization type, got fp8 - supported types are:
['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq', 'aqlm', 'quanto', 'eetq',
'hqq', 'compressed-tensors', 'fbgemm_fp8', 'torchao', 'bitnet']


I tried a different code:

import torch
generate_text = pipeline(model="deepseek-ai/DeepSeek-R1",torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
generate_text(messages)


This gives the following error:

raise ValueError( ValueError: Unknown quantization type, got fp8 - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq', 'aqlm', 'quanto', 'eetq', 'higgs', 'hqq', 'compressed-tensors', 'fbgemm_fp8', 'torchao', 'bitnet', 'vptq']

What can I do?

Continue reading...