[Python] DeprecationWarning with groupby().apply(): How to properly exclude grouping column in...

I'm on Pandas 2.2.1 with Python 3.9.19 trying to sample 5 rows per group using groupby().apply(), but I keep getting this DeprecationWarning in pandas 2.2+:

DeprecationWarning: DataFrameGroupBy.apply operated on the grouping columns.
This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation.
Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.


This is a reproducible example:

import pandas as pd

# Sample DataFrame
df = pd.DataFrame({
'locale': ['en_US', 'en_US', 'fr_FR', 'fr_FR', 'fr_FR', 'zh_CN', 'zh_CN'],
'query': ['A', 'B', 'C', 'D', 'E', 'F', 'G'],
'score': [1, 2, 3, 4, 5, 6, 7]
})

# Function to sample 5 rows per group
def sample_per_locale(df):
sampled_df = df.groupby("locale", group_keys=False) \
.apply(lambda x: x.sample(min(5, len(x)), random_state=42)) \
.reset_index(drop=True)

return sampled_df # Keeping locale but still getting the warning

# Run function
sampled_df = sample_per_locale(df)
print(sampled_df)


What I’ve Tried: Using include_groups=False (but my pandas' version doesn't support it):

df.groupby("locale", group_keys=False, include_groups=False) # TypeError


Explicitly selecting locale after apply():

sampled_df[['locale'] + [col for col in sampled_df.columns if col != 'locale']]


It still triggers the warning.

Expected Output: I just want groupby().apply() to sample 5 rows per locale without triggering this warning.

How do I properly exclude the grouping column in Pandas 2.2+? What's the correct way to handle this change in future versions? Any help would be greatly appreciated!

Continue reading...