[Python] Best project structure for the Python module with C++ extension and enabling...

Here is my current Python project structure, and I am using uv, py-cmake-build, and pybind11.

my_project/
├── my_package/
│ ├── __init__.py
│ └── my_extension.py # Python code that imports the C++ extension
├── cpp_src/
│ └── my_cpp_module.cpp
├── pyproject.toml
├── CMakeLists.txt
├── my_script.py
└── README.md


My requirements for the project structure is:

1. IntelliSense of the editor can find the C++ extension.
2. I can run my_script.py, which imports the C++ extension, with the command uv run my_script.py from the root folder, my_project, of the project.

The problem of my current project structure is that the C++ dynamic library is only installed in site-packages folder in the virtual environment, not in my_package folder under my project root folder, after I use uv pip install . or uv pip install . -e to trigger CMake. When executing uv run my_script.py, the current working folder is prepended to sys.path. The folder under my project root folder without the C++ dynamic library is imported, and the C++ extension is not found by Python. Nevertheless, IntelliSense also raises the error that the C++ extension is not found.

There are two solutions in my mind. I wonder which one is better and more common.

1. Update CMakeLists.txt to also install the dynamic library in-place (in my_package folder) add ignore the dynamic library in .gitignore.
2. Move my_package and cpp_src folders into a new src folder, so that the Python interpreter and IntelliSense will import the package in the virtual environment.

The second one is the same as the example of py-build-cmake

I saw some famous projects using C++ extensions like numpy, scipy, cupy etc. put their package folder under the root folder as my first solution. Do they also install the C++ extension dynamic library in-place? If not, how they solve the problem that the editor cannot find the C++ extension for developers?

Continue reading...