[Python] Polars custom function returning multiple features

I'm evaluating polars for timeseries feature extraction - replicating for example the functionality of libraries such as tsfresh.

I have the basics working really well, i.e. using dynamic grouping to create windows and most of the basic tsfel features can be directly reimplemented as polars custom functions (maybe I can optimise this but for now performance is fine), i.e.

def var_larger_than_std(x: pl.Series) -> bool:
"""
Is variance higher than the standard deviation?

Boolean variable denoting if the variance of x is greater than its standard deviation. Is equal to variance of x
being larger than 1

aram x: the time series to calculate the feature of
:type x: pl.Series
:return: the value of this feature
:return type: bool
"""
y = x.var()
return y > np.sqrt(y)

q = (
dataset.lazy() \
.group_by_dynamic("ts_local", every="1d", group_by="id") \
.agg(
pl.col("value").count().alias("value__count"),
var_larger_than_std(pl.col("value")).alias("value__var_larger_than_std"),
)
)

q.collect()


For basic feature maps f(series)->series this works fine. It is quite common though for complex feature maps to produce multiple outputs. For example fft_coefficient produces multiple outputs. In tsfel they're returned as a dict and each key:value is expanded to into a seperate pandas.Series in the final pandas.Dataframe.

How can I replicated this in polars? Say I have the function

def complex_feature(x: pl.Series) -> Dict[str, float]:
return {"col1":0.0, "col2":1.0}

q = (
dataset.lazy() \
.group_by_dynamic("ts_local", every="1d", group_by="id") \
.agg(
complex_feature(pl.col("value")).???,
)
)

q.collect()


┌────────────────────────────────────┬──────┬──────┐
│ ts_local ┆ col1 ┆ col2 │
│ -------- ┆ ---- ┆ ---- │
│ timestamp[ns,tz=Australia/Sydney] ┆ f64 ┆ f64 │
╞════════════════════════════════════╪══════╪══════╡
│ 2023-01-01 ┆ 0.0 ┆ 1.0 │
└────────────────────────────────────┴──────┴──────┘


i.e. the dict is expaned into seperate columns (it doesn't have to be a dict, I can return a tuple of pl.Series or whatever works)

Continue reading...