[Python] Pandas dropna(how= "all", inplace = True) doesnt work

df.dropna(how="all", inplace = True) does not remove all full Nans columns/rows that are in my dataframe.

I created a matrix of every distance between the values in a list.

0 1 2 3 4
0 0.000000,1.003090,7.006241,8.985008,9.012505
1 1.003090,0.000000,6.003151,7.981918,8.009415
2 7.006241,6.003151,0.000000,1.978767,2.006264
3 8.985008,7.981918,1.978767,0.000000,0.027496



I the filtered this list by values that dont fit within a specific interval and turned the values that dont fit into Nans.

0 1 2 3 4
0 Nan ,Nan ,7.006241,8.985008,9.012505
1 Nan ,Nan ,6.003151,7.981918,8.009415
2 7.006241,6.003151,Nan ,Nan ,2.006264
3 8.985008,7.981918,1.978767,Nan ,Nan


Additionally I turned the lower half (diagonally) to Nans because it is essentially the same as the upper half (diagonally).

0 1 2 3 4
0 Nan ,Nan ,7.006241,8.985008,9.012505
1 Nan ,Nan ,6.003151,7.981918,8.009415
2 Nan ,Nan ,Nan ,Nan ,2.006264
3 Nan ,Nan ,Nan ,Nan ,Nan


Now I am trying to remove all columns and rows that are fully turned into Nans with:

df.dropna(how="all", inplace = True)

but it only removes like 6 out of 100 rows that are full of Nans :,(.

This is my full code:

mz_array = [1,2,3,4,5,6,7] # my list with data, actually 603 long.
distance_matrix = []
for column, i in zip(mz_array, range(len(mz_array))):
distance_matrix.append([])
for row in mz_array:
distance_matrix.append(np.abs(row - column))

df_dm = pd.DataFrame(distance_matrix)
df_dm.columns = df_dm.columns.map(str)

for i in range(len(mz_array)):
for j in range(len(mz_array)):
if j <= i:
df_dm.iat[i,j] = None
if df_dm.iat[i,j] < 55 or df_dm.iat[i,j] > 480:
df_dm.iat[i,j] = None

df_dm.dropna(how="all", inplace = True)

Continue reading...