Among other descriptive statistics, I want to get some quantiles out of my pandas DataFrame. I can get the quantiles I want a couple of different ways, but I can't find the right way to do it with aggregate. I'd like to use aggregate because it'd be tidy and maybe computationally efficient to get all my stats in one go. rng = np.random.default_rng(seed=18860504) df = pd.DataFrame({ "dummy": 1, "bell": rng.normal(loc=0, scale=1, size=100), "fish": rng.poisson(lam=10, size=100), "cabin": rng.lognormal(mean=0, sigma=1.0, size=100), }) quants = [x/5 for x in range(6)] quantiles = pd.DataFrame({ "quantile" : [f"q{100*q:02n}" for q in quants], "bell" : df.groupby("dummy")["bell"].quantile(quants), "fish" : df.groupby("dummy")["fish"].quantile(quants), }) print(quantiles) Output: quantile bell fish dummy 1 0.0 q000 -2.313461 4.0 0.2 q020 -0.933831 7.0 0.4 q040 -0.246860 9.0 0.6 q060 0.211076 10.0 0.8 q080 0.685958 13.0 1.0 q100 3.017258 20.0 I'd like to get these quantiles using groupby().agg(), ideally with programmatically named columns like "bell_q90". Here's an example of the aggregate syntax that feels natural to me: df.groupby("dummy").agg( bell_med=("bell", "median"), bell_mean=("bell", "mean"), fish_med=("fish", "median"), fish_mean=("fish", "mean"), # fish_q10=("fish", "quantile(0.1)"), # nothing like it # fish_q10=("fish", "quantile", 0.1), # nothing like it # fish_q10=("fish", "quantile", kwargs({"q":0.1}), # nothing like it ) I can imagine generating the columns by iterating over quants and a list of named columns, using Series.agg and than stitching them together, but this seems like a hack. (For example, it would require me to do my "normal" aggregation first and then add quantiles on afterwards.) my_aggs = dict() for q in quants: for col in ["bell", "fish"]: my_aggs[f"{col}_q{100*q:03n}"] = df.groupby("dummy")[col].quantile(q) print(pd.DataFrame(my_aggs)) # numbers equivalent to those above Is there a better way? Continue reading...
