[Python] setting temperature in Open Llama does not work

I try to generate several alternative continuations of given prompt with Open Llama, setting nonzero temperature:

import re
import torch
from transformers import LlamaTokenizer, LlamaForCausalLM

model_path = 'openlm-research/open_llama_3b_v2'

tokenizer = LlamaTokenizer.from_pretrained(model_path)
model = LlamaForCausalLM.from_pretrained(model_path, torch_dtype=torch.float16, device_map='auto')
text = 'Once upon a time '
text_tokenized_for_llm = tokenizer(text, return_tensors="pt").input_ids
for i in range(25):
result = model.generate(input_ids=text_tokenized_for_llm, max_new_tokens=6, temperature=2)
text = tokenizer.decode(result[0])
print('->' + text + '<-')


However, when I run the program, all continuations are the same:

-><s>Once upon a time 100 years ago,<-
-><s>Once upon a time 100 years ago,<-
-><s>Once upon a time 100 years ago,<-
(...)


What is wrong here?

Continue reading...