[Python] group by one column, get mode of another, and fill na with that particular mode...

I am playing with the SpaceShipTitanic dataset on Kaggle.

I have a df. I want to group by a column, find the mode of a second column based on the group, and fill all the np.NaN values in the group in that second column with the most common value.

I feel like I'm skirting around this and not getting to the answer.

I have tried .mask and .transform and .agg. I've used lambda functions and attempted to deal with indices, and I am not accomplishing what I want. I must not be putting the parts together properly

Here is a brief example. Start with something like this.

df =
Passenger_Group HomePlanet
1 1 Earth
2 1 Mars
3 1 Earth
4 1 NaN
5 2 NaN
6 2 Venus
7 3 Ceres


Note that in the full dataset there are a lot more columns and rows, so I have to work around other column names. The goal is to end up with a df that looks like this

df =
Passenger_Group HomePlanet
1 1 Earth
2 1 Mars
3 1 Earth
4 1 Earth
5 2 Venus
6 2 Venus
7 3 Ceres


I feel like this should be a simple command. Everything I try doesn't quite work, though.

What is a succinct pythonic way to do this?

Continue reading...