Say I have some dataset df where I know all the variables used beforehand. Also say I have some transformation called T1 that does something like df.loc[(df["col1"]> 4) & ((df["col3"]== 2)),"col2"] = 3. ThenT1 would use col1,col3 and whatever columns df.index is as input variables, and would have col2 as an output variable I would love to define a bunch of transformations T1 ... Ti and keep track of what columns they use as input and output variables without manually having to register the variables for every transformation. Bonus points if there is a way to resolve the variables before actually executing the command. My initial thought was to make a object read.col1 and write.col2 that when called yielded the column as text, but this also made read_cols=["col1","col2", *df.index] and write.cols=["col2"] and then could be blanked out with read.clear() etc after each transformation using a decorator around the function T1 that does the transformation. Then I would make a proxy object for df that exactly copied a pandas dataframe, but had a special flag that would make all commands pass if the flag was set so I could run through all functions, track variables, turn flag off then call the dataframe transformations for real on a second pass. But this seems quite convoluted and probably is not a good design pattern. Continue reading...
