[Python] Llama-3.2-1B-Instruct generate inconsistent output

I want to use Llama-3.2-1B-Instruct model, and although I have set "temperature": 0.0, "top_p":0.0 and "top_k":0, it still generates inconsistent output. This is how my pipeline looks like:

pipe = pipeline(
"text-generation",
model=model_id,
torch_dtype=torch.bfloat16,
device_map="mps",
model_kwargs={"temperature": 0.0,
"do_sample":True,
"top_p":0.0,
"top_k":0,},
)


Any idea how to solve this issue?

Continue reading...