[Python] Should Polars Enum Datatype Result in More Efficient Storage and Memory Footprint of...

I have this dataframe in Python Polars having dimensions (8442x7), basically, 1206 rows for each day of the week. The day of week appears as a simple string.

Thought I would exploit the pl.Enum to encode the ISO_WEEKDAY column, saving space on disk. See the following code.

#!/usr/bin/env python3
# encoding: utf-8

import calendar, polars as pl


df: pl.DataFrame = pl.read_parquet(source='some_data.parquet') # About 268K
# df has a column called ISO_WEEKDAY whose values are Monday, Tuesday...Sunday

weekdaysl.Enum=pl.Enum(categories=iter(calendar.day_name)) # Encode all weekdays in the enum
df=df.with_columns(pl.col(ISO_WEEKDAY).cast(dtype=weekdays))
df.write_parquet('new_file.parquet') # ~ Same 268K


But seems it is not happening, i.e. even after transforming the column to pl.Enum

• the parquet file is of the same size
• the dataframe in memory (if I use the pl.DataFrame.estimated_size() method) actually gets bigger.

So is pl.Enum advisable at all, even though the column values are repeated 1206 times each?

Continue reading...