[Python] Sum of null and empty values in pandas dataframe

I have a dataframe that contains both NULL values and empty strings.

As follows (with code for repro)

data = {'vals1': [None, '100', '', '200'],
'vals2': ['100', '100', '', ''],
'vals3': ['100', None, '100', None],
'vals4': ['', '', '', '']}

df = pd.DataFrame(data)


Output:

vals1 vals2 vals3 vals4
0 None 100 100
1 100 100 None
2 100
3 200 None


You can see there are legit null values (pandas treats None as null), but there are also empty strings, denoted by the blanks, which are also a legit feature of the dataset. I can see the nulls by using df.isnull(), which gives:

vals1 vals2 vals3 vals4
0 True False False False
1 False False True False
2 False False False False
3 False False True False


And I can count the null values by using df.isnull().sum(), which gives:

vals1 1
vals2 0
vals3 2
vals4 0
dtype: int64


However, I also need a way of accounting for the empty values too, such that the output becomes something like:

Nulls Empty
vals1 1 1
vals2 0 1
vals3 2 1
vals4 0 4
dtype: int64


How can I achieve my desired output?

Continue reading...