[Python] Apply function to multiple dataframes with different type of data

I'm working with about a million-row dataframe in Pandas.

I have the table 1 (events_payment_data) as below:

• 
  next payment ID: is the payment ID of the next payment for any non_payment event, or the payment ID for the current PAYMENT event
  
  
• 
  last payment ID: is the payment ID of the last payment for any non_payment event, or the payment ID for the current PAYMENT event
  
  
• 
  time to next payment: is the timedelta between any non_payment event to the next payment, or NULL if the current event is PAYMENT event
  

next payment ID last payment ID UserID event eventTime time to next payment
ABXXX1 1234 UPDATE_DETAIL 2024-05-28 05:08:59 00 00:01:32
ABXXX1 ABXXX1 1234 PAYMENT 2024-05-28 05:10:31
ABXXX2 ABXXX1 1234 UPDATE_DETAIL 2024-05-28 05:10:31 00 00:03:43
ABXXX2 ABXXX2 1234 PAYMENT 2024-05-28 05:14:15
ABXXX3 ABXXX2 1234 LOGIN 2024-05-29 05:54:23 00 00:02:55
ABXXX3 ABXXX3 1234 PAYMENT 2024-05-29 05:57:18

From table 1, I calculated table 2 (daily_payments_timediff) as below. This is the aggregated time difference between each PAYMENT event in table 1:

• time to next payment: is the timedelta between two PAYMENT events

UserId last payment ID time to next payment
1234 ABXXX1 00:03:44
1234 ABXXX2 00:43:04
1234 ABXXX3

I calculated the timedelta between any non_payment event to the following payments only if the timedelta <= 2 hours (time to payments dict (new column)).

Currently I want to calculate the no. of time each non_payment event occurred in 2 hours before each payment (event (non_payment) within 2 hours (new column)).

My expected final table would look like the following:

next payment ID last payment ID UserID event eventTime time to next payment time to payments dict (new column) event (non_payment) within 2 hours (new column)
ABXXX1 1234 UPDATE_DETAIL 2024-05-28 05:08:59 00 00:01:32 {'ABXXX1': 00 00:01:32, 'ABXXX2': 00 00:05:16}
ABXXX1 ABXXX1 1234 PAYMENT 2024-05-28 05:10:31 {UPDATE_DETAIL: 1}
ABXXX2 ABXXX1 1234 UPDATE_DETAIL 2024-05-28 05:10:31 00 00:03:43 {'ABXXX2': 00 00:03:43}
ABXXX2 ABXXX2 1234 PAYMENT 2024-05-28 05:14:15 {UPDATE_DETAIL: 2}
ABXXX3 ABXXX2 1234 LOGIN 2024-05-29 05:54:23 00 00:02:55 {'ABXXX3': 00 00:02:55}
ABXXX3 ABXXX3 1234 PAYMENT 2024-05-29 05:57:18 {LOGIN: 1}

My question is how to create column event (non_payment) within 2 hours (new column)?

Continue reading...