[Python] Downcasting to Float32 displays additional precision numbers in polars which dont...

Whenever we downcast a column from float 64 to float 32, it seems that polars adds extra precision numbers to the display output which didnt even exist. When I compare the actual values, they seem to match, but the display setting throws off any checks done by naked eye. I know there is a display config to fix this and limit precision points when displaying data, but I am just curious as to why this happens in the first place and why some random numbers are added as padding.

I first noticed this when reading data from a csv file with schema overrides where data seemed off when read and printed with polars and compared against csv file opened in a text editor

import polars as pl
df1 = pl.DataFrame(
{
"col1":[2021.9952, 2024.0, 2024.25, 2024.456]
}
)
print("original values")
print(df1)

shape: (4, 1)
┌───────────┐
│ col1 │
│ --- │
│ f64 │
╞═══════════╡
│ 2021.9952 │
│ 2024.0 │
│ 2024.25 │
│ 2024.456 │
└───────────┘

diff1 = df1["col1"][1] - df1["col1"][0]
print("Original: diff row1 - row0")
print(diff1)
2.004799999999932

print("Origirnal value equality")
print(df1["col1"][0] == 2021.9952)
True

print("dDowncasting to float 32")
print(df1.cast({"col1"l.Float32}))

shape: (4, 1)
┌─────────────┐
│ col1 │
│ --- │
│ f32 │
╞═════════════╡
│ 2021.995239 │
│ 2024.0 │
│ 2024.25 │
│ 2024.456055 │
└─────────────┘

print("Downcasted: diff row1 - row0")
diff2 = df1["col1"][1] - df1["col1"][0]
print(diff2)
2.004799999999932
print("Downcasted value equality")
print(df1["col1"][0] == 2021.9952)
True


Expected: I expected the down casted values to look the same as before or maybe with padded zeros instead of random numbers -

┌───────────┐
│ col1 │
│ --- │
│ f32 │
╞═══════════╡
│ 2021.9952 │
│ 2024.0 │
│ 2024.25 │
│ 2024.456 │
└───────────┘

or

┌───────────┐
│ col1 │
│ --- │
│ f32 │
╞═══════════╡
│ 2021.99520│
│ 2024.00000│
│ 2024.25000│
│ 2024.45600│
└───────────┘

Continue reading...