[Python] RuntimeError: PassManager::run failed when fine-tuning Phi‑3.5‑mini‑bnb‑4bit with...

I’m trying to fine‑tune the unsloth/Phi-3.5-mini-instruct-bnb-4bit model on a custom text dataset in Google Colab using 4‑bit quantization and TRL’s SFTTrainer. There are no errors in my code, and I have sufficient VRAM/RAM available, but when I call trainer.train() I immediately get:

if os.environ.get("TRITON_DISABLE_LINE_INFO", "0") == "0":
passes.llvmir.add_di_scope(pm)
pm.run(mod)
RuntimeError: PassManager::run failed


My code:

from trl import SFTTrainer
from transformers import TrainingArguments
import torch
from unsloth import FastLanguageModel

# Load 4-bit model
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Phi-3.5-mini-instruct-bnb-4bit",
load_in_4bit=True,
device_map=None,
dtype=None,
max_seq_length=1024
)

# Apply PEFT LoRA
model = FastLanguageModel.get_peft_model(
model,
r=16,
target_modules=[
"q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",
],
lora_alpha=32,
lora_dropout=0.05,
use_gradient_checkpointing="unsloth",
bias="none",
)

# Trainer args
training_args = TrainingArguments(
per_device_train_batch_size=8,
gradient_accumulation_steps=2,
num_train_epochs=3,
learning_rate=2e-4,
fp16=not torch.cuda.is_bf16_supported(),
bf16=torch.cuda.is_bf16_supported(),
logging_steps=25,
optim="paged_adamw_8bit",
weight_decay=0.01,
lr_scheduler_type="cosine",
seed=3407,
output_dir="outputs",
save_strategy="epoch",
save_total_limit=2,
dataloader_pin_memory=False,
gradient_checkpointing=True,
)

trainer = SFTTrainer(
model=model,
train_dataset=ds,
eval_dataset=valid_ds,
tokenizer=tokenizer,
args=training_args,
max_seq_length=128,
dataset_text_field="text",
packing=True,
)

# Train
trainer.train()


What I’ve Tried:

Verifying CUDA, PyTorch, transformers, trl, and unsloth versions are compatible.

Ensuring Colab runtime has enough GPU memory.

Disabling line info via os.environ["TRITON_DISABLE_LINE_INFO"] = "1", but the same error occurs.

Continue reading...