[Python] How to Fine-Tune Large Language Models Using Transformers, PEFT, and LoRA

I'm a beginner in AI. I'm currently attempting to fine-tune the Qwen2.5-7B large model using the LoRA algorithm with the PEFT library and Hugging Face’s Transformers library. The code is as follows:

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import TaskType,LoraConfig, get_peft_model
from transformers import pipeline
import inspect
import torch
model = AutoModelForCausalLM.from_pretrained(
"/models/Qwen2.5-7B/",
device_map="auto",
torch_dtype=torch.bfloat16
)
tokenizer = AutoTokenizer.from_pretrained("/models/Qwen2.5-7B/")
peft_config = LoraConfig(task_type=TaskType.CAUSAL_LM, inference_mode=True, r=1024, lora_alpha=32, lora_dropout=0.1)
model = get_peft_model(model, peft_config)
model.print_trainable_parameters()
training_args = TrainingArguments(
learning_rate=1e-5,
per_device_train_batch_size=32,
per_device_eval_batch_size=32,
num_train_epochs=2,
weight_decay=0.01,
eval_strategy="epoch",
save_strategy="epoch",
load_best_model_at_end=True,
)


#How to build train set and eval set in code?
#train_set = ?
#eval_set = ?

trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_set,
eval_dataset=eval_set,
tokenizer=tokenizer,
data_collator=data_collator,
compute_metrics=compute_metrics,
)

trainer.train()

inf = pipeline("text-generation", model = model, tokenizer = tokenizer)
print(inf(["what's your name"]))


Currently, I’m encountering two issues:

1. 
   model.print_trainable_parameters() outputs trainable params:
   
   0 || all params: 7,938,577,920 || trainable%: 0.0000.
   
   I don’t understand why the number of trainable parameters is zero. How should I adjust my code?
   
   
2. 
   How should I construct train_set and eval_set? What data structures should they be, and what content should they include?
   

Continue reading...