I am trying to understand where my code goes wrong and created a minimalistic example to understand what is happening. As you will see in the end, the reported groups and values make no sense. Perhaps this is intended by the pandas philosophy and perhaps not. I am not sure so I am asking your views. In particular is my understanding of what is going on correct? Is this behaviour intended by pandas? import pandas as pd # define series sf = pd.Series([10, 10, 20, 30, 30, 30], index=np.arange(6)+2) print(sf) # 2 10 # 3 10 # 4 20 # 5 30 # 6 30 # 7 30 # dtype: int64 # group by using the series itself <- makes sense grouped = sf.groupby(sf) for name, group in grouped: print(f"Group: {name}") print(group) # Group: 10 # 2 10 # 3 10 # dtype: int64 # Group: 20 # 4 20 # dtype: int64 # Group: 30 # 5 30 # 6 30 # 7 30 # dtype: int64 # change index in the group by series and examine groups # I understand that: # - first there is index alignment and from the new grouping series, only indices 2 to 5 (inclusive) are kept (with values 20, 30, 30, 30) # -- question: why does it filter the new grouping series and not the original series? # - then grouping happens and produces the groups 20 (new grouping series indices [2]) and 30 (new grouping series indices [3, 4, 5]) # - using these indices we return the values of the original series # -- question: do you think the final group values and series values make sense? grouped = sf.groupby(sf.reset_index(drop=True)) for name, group in grouped: print(f"Group: {name}") print(group) # Group: 20.0 # 2 10 # dtype: int64 # Group: 30.0 # 3 10 # 4 20 # 5 30 # dtype: int64 Continue reading...
