[Python] Using pip install to compile shared object file and include in package installation

I am working on creating a package which contains modules that wrap methods from a shared object file using ctypes. So far I have been successful in compiling the shared object during a pip install of the package; my setup.py looks like:

import subprocess
from typing import List

from setuptools import find_namespace_packages, setup
from setuptools.command.install import install


class CustomInstall(install):
def run(self):
with open("install.log", "w") as f:
subprocess.run(["./compile.sh"], stdout=f)
install.run(self)

requirements: List[str] = [
...
]

setup(
author="",
python_requires=">=3.11",
install_requires=requirements,
name="mypkg",
license="",
packages=find_namespace_packages(include=["mypkg", "mypkg.*"]),
cmdclass={"install": CustomInstall},
include_package_data=True,
zip_safe=False,
)


I also have a MANIFEST.in, that looks like:

global-include libc.* The structure of the directory containing the package source files is:

top
├── c_files
├── compile.sh
├── MANIFEST.in
├── mypkg
│ ├── __init__.py
│ └── mod1.py
└── setup.py


After the install the shared object file is available locally, but is not copied over to the install location; specifically I am using a conda environment for the install and so the expectation is that the file will also be brought over to the location where the install is located. A subsequent pip install command will copy the file over however. My thought is that the precedence for parsing the MANIFEST.in is done before the compilation of the file. I would like to have the file included in the install without the subsequent pip install command. Any and all help is greatly appreciated!

Continue reading...