[Python] How to ensures store_id records missing in the present month receive their store_id...

I created a Databrciks notebook and load the table in databricks using pyspark code, and i'm trying to copy missing store_id information from previous month to current month with historical and present data. Below example table.April data,May data If store_ids 300001, 300003, and 300005 are missing from the May data table, their complete details and all associated historical data should be copy forward from the previous month's data into the current May data table. With logic group by type, category,brand, segment and collection method is IR fixed.

How can i write the code for logic. I expected from result like this.Output data,Please identify the store_ids missing from the current month's report and copy forward their corresponding historical data, applying group-by logic for type, category, brand, and segment, to generate the complete table.Please check above attached images.

Continue reading...