[Python] Track manual changes to autogenerated files while allowing for automatic updates

I want to allow manual changes to automatically generated files (from templates), while still allowing for updates to the templates or to data. I envision to use git to track the human-made and code-generated portions of the generated files and merge them intelligently.

Somewhat along the lines of

• human writes templates
• run code to generate files from templates and fill in data
• human applies changes to the file
• code collects new data and generates a new file from template
• code merges human-made changes into the new file

What I want is something like often done when handling embedded code where the IDE autogenerates code and for example fills register addresses, but still allows manual changes to said code. However, they usually restrict changes only to small segments - which is a restriction that I cannot allow.

I think that it can be done with separate branches and a suitable merge/rebase strategy, but when thinking about it I identify many difficult corner cases. Therefore I am looking for references where something similar has been done already or a more detailed strategy that avoids at least the following issues:

• if the templates live in the same git repository as the generated files, just checking out a "autogenerated" branch does not track the template changes
• after the templates or data was changed, all human-made changes have to be applied (via rebase?) onto the new generated files. I am looking for a good merge strategy that avoids conflicts as much as possible
• the newly merged file has to be pushed back onto the "work" branch, if possible without rewriting history

I can separate the templates from the autogenerated files into different directories, but that does not fix the checkout problem.

Ideally I'd want to write some python code that handles the complete workflow: git checkouts, template generation, merges and user feedback with a single command.

Continue reading...