[Python] KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'

raise KeyError(f"Cache only has {len(self)} layers, attempted to access layer with index {layer_idx}")


KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'

When I try to run train_RAMLLaMA in RAMQA, I encounter this problem. When I read other post about this problem, they said change the transfromar version to 4.35.0. But for this model, I also need transfromar.gamma2 for the unsloth package. The transfromar.gamma2 only exist in >= 4.43.0

torch 2.3.0 transformers 4.44.2 unsloth 2024.8

Continue reading...