[Python] Python Polars - creating new columns based on the key-value pair of a dict matched to...

Sorry if the title is confusing.

I'm pretty familiar with Pandas and think I have a solid idea of how I would do this there. Pretty much just brute-force iteration and index-based assignment for the new columns. I recently learned about Polars, though, and want to try it for the parallelization/speed and to stay fresh and up to date on my data skills. This is my first foray, and it's not been going great.

I have a dataframe, and one column of this frame is basically a tag list. Each cell in that column is a list of relevant tags. What I want to do is scan through those lists, row by row, and add a column by the name of a more general tag if the existing tag is in the cell.

For example, say I have a dataframe that looks like this:

Index Person Food Provided
1 Billy Apple, Hot dog
2 Suzy Celery, brownies

Then I also have a dictionary that looks like this: foodTypes_dict = {'Apple':'Fruit', 'Hot dog':'Meat', 'Celery':'Vegetable', 'brownies':'Dessert'}

I would like to create a new column based on the food type that has a simple X or True or something if the "Food Provided" list contains the dict key.

Something like:

Index Person Food Provided Fruit Vegetable Meat Dessert
1 Billy Apple, Hot dog X None X None
2 Suzy Celery, brownies None X None X

I've tried:

for key in foodTypes_dict.keys():
my_df.with_columns((pl.col("Food Provided").str.contains(key)).alias(foodTypes_dict[key]))


This has finally gotten me away from syntax errors, which I was encountering with everything else I tried. It doesn't, however, seem to actually be working at all. Essentially, it doesn't seem to create any new columns whatsoever. I tried adding a my_df.glimpse() call during each iteration of the for loop, but the dataframe dimensions don't change. I do not get any syntax errors or otherwise. I am using Jupyter Notebook which can suppress some of them, but the cell runs and finishes nearly instantly, just not close to the expected output.

Any help would be appreciated. Thanks!

Continue reading...