[Python] Polars bug using windowed aggregate functions on Decimal type columns

Windowed aggregate functions on Decimal-types move decimals to integers


I found a bug in polars (version 1.21.0 in a Python 3.10.8 environment) using windowed aggregate functions. They are not properly handling the decimal, essentially multiplying the result by 100. Here is a minimum reproducible example:

import polars as pl

pl.__version__ # 1.21.0

pl.DataFrame(
{
'People':['John', 'John', 'John', 'John', 'Jane', 'Jane', 'Jane', 'Jane'],
'Balance': [0.00, 10.59, 11.29, 0.00, 12.34, 23.45, 34.56, 45.67]
}
, schema={'People'l.String, 'Balance'l.Decimal(10, 2)}
).with_columns(
Bal_max = pl.col("Balance").max(),
Bal_max_person_wrong = pl.col("Balance").max().over("People"),
Bal_max_person_right = pl.col("Balance").cast(float).max().over("People"),
Bal_min_person_wrong = pl.col("Balance").min().over("People"),
Bal_sum_person_wrong = pl.col("Balance").sum().over("People")
)



The results are below:



What should I do about this? I'm tempted to hack it a bit and divide by 100, but that seems unwise. I will probably treat the data as floats, but I prefer limiting the decimals for values that will always be treated as dollars and cents. Any advice you can give would be appreciated!

Continue reading...