In Pandas I do an aggregation rcresult = df.groupby(cohort,as_index=False).agg({'resolvetime': ['count','sum','median','mean','var','std'], 'closetime': ['median','mean','var','std'],'grouptransfers': ['mean','median','var','std']}) My input df has n=60,937 rows. Every row has a valid resolvetime, but note for n=2,364 of the rows, the resolvetime is 0.00. Checking the fields that are mentioned in the agg, resolvetime, closetime, and grouptransfers, they all sum to 60,937 when I sum their value counts. Checking them, there are no NaN in them. There are some values of 0.00 in each. When I do a sum of the counts print("Agg DF Counts in Test summed : ", rcresult[('resolvetime','count')].sum() ) I get a sum of n=42,946. What conceptually can explain this shortfall? I was expecting that I would get n=60,937 again. Continue reading...
