I'm looking to test a Python module using pytest which uses data from an external git submodule. I want to include an (eventually opt-in) option to test my code against many different versions of this submodule automatically, or in other words running all of my tests against each git tag >= 1.0.0. Essentially I would like calling pytest once to expand to the following commands: git checkout 1.0.0 pytest git checkout 1.0.1 pytest git checkout 1.0.2 pytest ... Where the number of tests run in that session is amount of tests * number of submodule versions. After taking a look at the documentation for pytest hook functions, I found pytest_sessionstart(), and am able to dynamically pick a git version tag before running the suite: @pytest.hookimpl() def pytest_sessionstart(session): # Grab and populate the repo repo = git.Repo(path_to_repo) repo.git.fetch() tag_list = sorted([version_string_to_tuple(tag.name) for tag in repo.tags]) # Only select versions >= 1.0.0 tag_list = tag_list[tag_list.index((1, 0, 0)):] # Checkout (singular) version repo.git.checkout(version_tuple_to_string(tag_list[-1])) # latest, because why not # Test collection/execution begins after this function ends But this will only allow me to checkout one version per pytest invocation. Continue reading...
