[Python] Python TimedRotatingFileHandler cannot rename log file due to Windows process still...

I have a Python CLI using Click Library. I am using pyinstaller to configure the final exe to run on windows machines.

I have a core python logger that uses TimedRotatingFileHandler. Each command in click grabs the logger with its name and and it points to the same filename called app.log

The storage location of the log is always in %USER%\AppData\Local\Temp\some_folder\logs\app.log

On Day 1 - I run one independent command, the logger initiated creates the file, logs to it no problem.

On Day 2 - the logger then has the responsibility to look at the last modification date of the log file and determine if it is past the designated window I have set of midnight. If so, it needs to append the date and create a new log file. So the correct process would be:

• Check app.log modification date
• Last date is a day old, rename app.log to app.log.2025-03-03 (this is from the perspective that the command is currently being run on 3/4/2025)
• create new app.log file and starts logging there

My current understanding is that Click CLI or Pyinstaller should be running the CLI with 1 command inside of 1 process. You cannot run CLI command concurrently unless you create some clever programmatic way to do so. From the user perspective, this is not possible.

The problem here is on Day 2 - I get the following:

#Traceback here - last call is to os.rename(source,dest) in logging module handlers.py

PermissionError: [WinError32] The process cannot access the file because it is being used by another process"
'C:\\Users\SOME_USER\AppData\Local\Temp\SOME_FOLDER\logs\app.log' -> 'C:\\Users\SOME_USER\AppData\Local\Temp\SOME_FOLDER\logs\app.log.2025-03-03'


I am having trouble understanding where I need to fix this or how to look into it.

Possible issues:

• Pyinstaller creates a new executable environment for running the application. It stores all its requirements in a temp folder and then removes the temp folder after the execution is complete. So why would this not release the log file being used?
• Can Click cause the process to not let go of the log file?
• Do I need to manually clean up log handlers per command? I would think the process would release right away.

FYI - this permission error does NOT happen on the same day, it only happens when the logger is trying to rename the file

Here is my logger code


logger = logging.getLogger(name)

logger.setLevel(logging.DEBUG)

path = pathlib.Path.home() / filename
file_handler = logging.handlers.TimedRotatingFileHandler(path, when="midnight", interval=1, backupCount=7)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(logging.Formatter(LOG_FORMATTER))
logger.addHandler(file_handler)



I create a log file in Documents folder for example, then I run this command to change the modification date so it appears as the file was not touched since yesterday

import os
import time

file_path = "C:\\path\\to\\your\\file.txt"

# Set the modification time to a specific timestamp (e.g., Jan 1, 2024, at 12:00 PM)
new_mtime = time.mktime((2025, 3, 3, 12, 0, 0, 0, 0, 0))

# Update the modification time (keep access time unchanged)
os.utime(file_path, (os.stat(file_path).st_atime, new_mtime))

print("Modification time updated successfully!")



Once the file is updated to appear as it was written to yesterday. I rerun my code with the logger and it renames the old file successfully and creates a new log. Needless to say, this is not what is happening in my CLI application

Continue reading...