[Python] Meta llama 3.2 3b model local download

I am installed the llamma3.2 model from meta directly and got it in this format

-a---- 3/10/2025 3:22 PM 209 checklist.chk
-a---- 3/10/2025 9:47 AM 6425585114 consolidated.00.pth
-a---- 3/10/2025 9:27 AM 220 params.json
-a---- 3/10/2025 9:27 AM 2183982 tokenizer.model


And I am trying to run it, but it is giving me this error

Traceback (most recent call last):
File "C:\Users\user\.llama\checkpoints\Llama3.2-3B-Instruct\LLAMA.PY", line 14, in <module>
model = LlamaForCausalLM.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\modeling_utils.py", line 262, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\modeling_utils.py", line 3808, in from_pretrained
raise EnvironmentError(
OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in director


running this code

# Install required packages if you haven't already
# pip install torch transformers

import torch
from transformers import LlamaForCausalLM, LlamaTokenizer

# Path to your model files
model_path = "C:/Users/user/.llama/checkpoints/Llama3.2-3B-Instruct" # The folder containing the files you listed

# Load the tokenizer
tokenizer = LlamaTokenizer.from_pretrained(model_path)

# Load the model
model = LlamaForCausalLM.from_pretrained(
model_path,
torch_dtype=torch.float16, # Use float16 for efficiency
low_cpu_mem_usage=True, # Helps with memory usage
device_map="auto" # Automatically use GPU if available
)

# Generate text
def generate_text(prompt, max_length=100):
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate
with torch.no_grad():
output = model.generate(
inputs["input_ids"],
max_length=max_length,
temperature=0.7,
top_p=0.9,
do_sample=True
)

# Decode and return the generated text
return tokenizer.decode(output[0], skip_special_tokens=True)

# Test the model
response = generate_text("What is artificial intelligence?")
print(response)


From what I have seen the model format is of meta's base one but I am not sure on how to run it

Continue reading...