[Python] How to resolve transformers dependency issues when running Python from C

Using import T5ForConditionalGeneration, T5Tokenizer will result in the following error:

RuntimeError: Failed to import transformers.models.t5.modeling_t5 because of the following error (look up to see its traceback): DLL load failed while importing _safetensors_rust: The specified module could not be found.
​

But I use python a.py in the Python command line and it is normal in C Python, except for the regular dependencies, it seems to be recognized normally in vs, but when it comes to transformers, the above error occurs.

#include <iostream>
#include <windows.h>
#include "Python.h"

int main() {

if (!SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_DEFAULT_DIRS)) {
std::cerr << "SetDefaultDllDirectories fail" << std::endl;
return 1;
}

PyStatus status;
PyConfig config;
PyConfig_InitPythonConfig(&config);
wchar_t* home_path = Py_DecodeLocale("C:/Users/username/anaconda3/envs/Pytorch-ipx", NULL);
if (home_path == NULL) {
fprintf(stderr, "Py_DecodeLocale fail\n");
return 1;
}

config.home = home_path;
status = Py_InitializeFromConfig(&config);
if (PyStatus_Exception(status)) {
fprintf(stderr, "Python initialization failed\n");
PyConfig_Clear(&config);
PyMem_RawFree(home_path);
return 1;
}

PyConfig_Clear(&config);

PyRun_SimpleString("import torch; import time; import intel_extension_for_pytorch as ipex; from transformers import T5ForConditionalGeneration, T5Tokenizer; print('Hi')");

Py_Finalize();
PyMem_RawFree(home_path);

return 0;
}

Continue reading...