[Python] Why does groupby().apply() produce inconsistent results on identical groups when the...

I noticed that groupby().apply() produces different results for two groups that look identical, except that the overall DataFrame has duplicate index values.

Here is a minimal reproducible example:

import pandas as pd

df = pd.DataFrame({
'group': ['A','A','B','B','B'],
'value': [1,2,1,2,2]
}, index=[0,1,1,2,3]) # note the duplicate index: 1 appears twice

result = df.groupby('group').apply(lambda g: g)
print(result)


Output:

group value
group
A A 1
A 2
B B 1
B 2
B 2


But when I reset the index so it becomes unique:

df2 = df.reset_index(drop=True)
print(df2.groupby('group').apply(lambda g: g))


I get a different structure (especially inside the B group).

Why does the presence of duplicate index values change how groupby().apply() constructs the returned index?What is the correct way to preserve the original rows and avoid unexpected index nesting when applying functions?

Continue reading...