[Python] most recommended packaging for python CLI apps for clean pip install and pip...

Main Question


What should be file structure of a package and its contents so that upon pip install it will create an app folder where it will store a variable amount of files only used by the app, initially populated by the resources of the python package, so that not only would it allow easy access to said resources on the runtime of the application (i.e. using importlib.resources) but that it will also, upon pip uninstall, remove the app folder?

Basic environment context

• let us assume a Linux OS, where it is typical to create in the user's home a .myappname folder to store configuration files.
• Let us assume a pure python application, only using libraries from the standard python library
• Let us assume python>=3.12
• Let us assume that it is intended to be installed with python -m pip install in a virtual environment, lets say a conda environment located at $HOME/venvironments/myenv.

Current Attempt

Package description


I want to create a python app with a CLI to the user. The CLI has three main actions, print, set and add:

• print allows printing the file contents of the files in $HOME/.myappname/ which upon installation will only contain a defaults.ini file and template_00.txt.
• add allows the user to add to the correct location ($HOME/.myappname/) a file.
• set allows the user to modify the contents of the variables in defaults.ini through the command line interface.

The current source package structure looks like:

- package/
| - myappname/
| | - __init__.py
| | - parser.py
| | - base.py
| | - scripts/
| | | - myappname.py
| | - resources/
| | | - defaults.ini
| | | - template_00.txt
| - setup.py
| - pyproject.toml
| - MANIFEST.in

File contents


the contents of the setup.py are:

import setuptools

setuptools.setup(
name = 'myappname',
version = '0.0.0',
packages = setuptools.find_packages(),
python_requires = '>=3.12',
include_package_data = True,
package_data = {'resources': ['myappname/resources/*']},
scripts = ['myappname/scripts/myappname.py'],
)


The contents of the pyproject.toml:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "myappname"
version = "0.0.0"
requires-python = ">=3.12"


The contents of the MANIFEST.in:

recursive-include myappname/resources *
global-exclude *.pyc
global-exclude .git


The minimal contents of parser.py:

import argparse

def create_print_options(parser):
pass
def create_add_options(parser):
pass
def create_set_options(parser):
pass

parser = argparse.ArgumentParser()

parser = argparse.ArgumentParser(prog='PROG')
subparsers = parser.add_subparsers(help='sub-command help',dest='command')

printparser = subparsers.add_parser('print')
create_print_options(printparser)

addparser = subparsers.add_parser('add')
create_add_options(addparser)

setparser = subparsers.add_parser('set')
create_set_options(setparser)


The minimal contents of base.py:

import importlib.resources

RESOURCESPATH = importlib.resources.file(__package__)/'resources'

def print_main(*args):
# Do stuff
# Use the RESOURCESPATH variable to find the files to print
pass
def add_main(*args):
# Do stuff
# Use the RESOURCESPATH variable to copy the file to that folder
pass
def set_main(*args):
# Do stuff
# Use the RESOURCESPATH variable to find defaults.ini
pass

def main(args):
if args.command == 'print':
return print_main(args)
if args.command == 'add':
return add_main(args)
if args.command == 'set':
return set_main(args)


contents of __init__.py:

from .base import main
from .parser import parser


Minimal contents for scripts/myappname.py:

#!/bin/usr/env python3
from myappname import main, parser

if __name__ == '__name__':
args = parser.parse_arguments()
main(args)

Main problems

• Upon installation this package will not create a $HOME/.myappname folder. Although it is expected since it is not included in any manner in the setup as I am unaware of where I should look for it.
• With the current installation, the resources folder and the defaults.ini and template_00.txt will be successfully copied into $HOME/venvironments/myenv/lib/python3.12/site-packages/myappname/resources and the script will be successfully copied to $HOME/venvironments/myenv/lib/python3.12/bin/myappname.py. However, if any modification is done into the resources folder, such as adding a file, a python -m pip uninstall will retain those files, not allowing a clean uninstall.
• While I think that the general structure shown should be relatively OS agnostic, I am unaware of how to indicate to pip that the resources folder can be safely removed, be it the present attempt ($HOME/venvironments/myenv/lib/python3.12/site-packages/myappname/resources) or the ideal $HOME/.myappname solution.

Note that with the simplified example, the pip uninstall will not show the mentioned behaviour since no file has been added to $HOME/venvironments/myenv/lib/python3.12/site-packages/myappname/resources This can be easily mocked by creating a file in the location after the python -m pip install path/to/myappsrc/ for example by: touch HOME/venvironments/myenv/lib/python3.12/site-packages/myappname/resources/mockfile.txt then attempting to uninstall python -m pip uninstall myappname

Continue reading...