[Python] Problems with pretraining models using huggingface

When I went to run a demo using hugging face's pre-trained model HumanVLM, there was a KeyError: 'llava' problem, the demo was as follows. And I have downloaded the pre-trained model locally in the folder pretrain_modal.

import requests
from PIL import Image
import torch
from transformers import AutoProcessor, AutoModelForPreTraining

model_id = "./pretrain_modal"
cuda = 0
model = AutoModelForPreTraining.from_pretrained("./pretrain_modal", torch_dtype=torch.float16).to(cuda)

processor = AutoProcessor.from_pretrained(model_id,trust_remote_code=True)
text = "Please describe this picture"
prompt = "USER: <image>\n" + text + "\nASSISTANT:"
image_file = "./test1.jpg"
raw_image = Image.open(image_file)
# raw_image = Image.open(requests.get(image_file, stream=True).raw)
inputs = processor(images=raw_image, text=prompt, return_tensors='pt').to(cuda, torch.float16)

output = model.generate(**inputs, max_new_tokens=400, do_sample=False)
predict = processor.decode(output[0][:], skip_special_tokens=True)
print(predict)


The problem is reported as follows.

Traceback (most recent call last):
File "test.py", line 10, in <module>
model = AutoModelForPreTraining.from_pretrained("./pretrain_modal", torch_dtype=torch.float16).to(cuda)
File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 527, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 1039, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 734, in __getitem__
raise KeyError(key)
KeyError: 'llava'


I've updated the version of transformers.

Continue reading...