[Python] How can I efficiently get both a column and a scalar using Polars expressions?

Polars suggests the usage of Expression to avoid eager execution and then execute all expressions together at the very end. I am unsure how this is possible if I want a column and a scalar. For example let's say I start with a single column 'test' and want to calculate its mean and produce a centered column. It is trivial to express this using expressions:

>>> import polars as pl
>>> import numpy as np
>>> df = pl.DataFrame({"test": np.array([0.,1,2,3,4])})
>>> df
shape: (5, 1)
┌──────┐
│ test │
│ --- │
│ f64 │
╞══════╡
│ 0.0 │
│ 1.0 │
│ 2.0 │
│ 3.0 │
│ 4.0 │
└──────┘

>>> mean = pl.col('test').mean().alias('mean')
>>> df.select(mean)
shape: (1, 1)
┌──────┐
│ mean │
│ --- │
│ f64 │
╞══════╡
│ 2.0 │
└──────┘
>>> centered = pl.col('test') - mean
>>> df.select(centered)
shape: (5, 1)
┌──────┐
│ test │
│ --- │
│ f64 │
╞══════╡
│ -2.0 │
│ -1.0 │
│ 0.0 │
│ 1.0 │
│ 2.0 │
└──────┘



Of course you could select them both, but then the mean gets broadcasested over all rows which does not seem storage efficient. Is there a good way to obtain both the column and the scalar?

In this case the best thing to do may be to calculate the mean eagerly and then proceed with the centering. But of course this might not work as well for more general cases.

Continue reading...