[Python] How to fix `FileExistsError` when using the `--basetemp` flag with `pytest`?

TL;DR - How do you use the --basetemp flag with pytest-mpi?

I am currently trying to run tests dependent on MPI using pytest-mpi. I've made a minimal example

# Standard Library Imports
import os

# Third-Party Library Imports
from mpi4py import MPI
import pytest

# Local Library Imports

# Relative Imports

@pytest.mark.mpi()
def test_mpi(tmp_path):
## Initialize MPI
if not MPI.Is_initialized():
MPI.init()
MPI_comm: MPI.Intracomm = MPI.COMM_WORLD
comm_rank: int = MPI_comm.Get_rank()

## Write comm info to file
file_name: str = "rank_{}.txt".format(comm_rank)
file_path: str = os.path.join(tmp_path, file_name)

with open(file_path, "w") as file:
file.write("{}".format(comm_rank))


which I am attempting to run with the command

mpirun -n 4 python -m pytest --with-mpi --basetemp=out -v -k test_mpi


However, it gives the error (with some path names sensored with xxx):

=============================================== ERRORS ===============================================
_____________________________________ ERROR at setup of test_mpi _____________________________________

self = PosixPath('/xxx/out')
mode = 448, parents = False, exist_ok = False

def mkdir(self, mode=0o777, parents=False, exist_ok=False):
"""
Create a new directory at this given path.
"""
try:
> os.mkdir(self, mode)
E FileExistsError: [Errno 17] File exists: '/xxx/out'

/home/usr/miniconda3/envs/hpc/lib/python3.12/pathlib.py:1311: FileExistsError


What is causing this error, and how can I fix it?

Things I've tried:

• 
  On the pytest-mpi readthedocs, they mention some 'fixtures' related to temp_path. I'm not really sure what a fixture is and how to use them to be honest, but it looks promising!
  
  
• 
  There's another code on Github that has a block of code that seems to be getting at this problem, but it didn't work conisstently for 3 processes and didn't work at all with more than 2 processes.
  

Both of these did not fix the FileExistsError.

Continue reading...