[Python] ImportError: libxxx: cannot open shared object file: No such file or directory

I am trying to build a Python package that interops with C++ code. I created a PyBind11 wrapper to do the interop. I created a setup.py file following the instructions here:

from pybind11.setup_helpers import Pybind11Extension

ext_modules = [
Pybind11Extension(
"python_example",
sorted(glob("src/*.cpp")), # Sort source files for reproducibility
extra_link_args=[f"-Wl,-rpath,$ORIGIN"]
),
]

setup(..., ext_modules=ext_modules)


I set extra_link_args=[f"-Wl,-rpath,$ORIGIN"] as I understand it will embed the origin to the search path of native libraries. (I have also tried without it with no luck). My C++ module depends on a shared library that I put right beside __init__.py. But when I try to import my C++ module I get :

ImportError: libxxx: cannot open shared object file: No such file or directory


My project structure after installation using pip install . is like this:

$ ls venv/lib/python3.10/site-packages/my_package
__init__.py __pycache__ libxxx.so


How can I fix the error?

Continue reading...