[Python] Discrepancy between MATLAB unique and NumPy np.unique results on floating-point...

Hello I am trying to replicate the matrices produced by MATLAB using Python. It is a 2D matrix (N,2) shape with no NaN values. They are all floats. These values are generated with certain parameters.

For instance, I'm comparing two 2D matrices (shape: (248, 2)) generated from MATLAB and Python. Before applying unique, the matrices are identical (np.allclose returns True, and no NaNs are present).

However, after applying unique:

MATLAB matrix has shape (240, 2)
Python matrix has shape (232, 2)


Another example is:

Comparison before unique() operations
Same shapes, (164, 2)
Same Values (allclose)? True
Comparison after unique() operations
Shapes mismatch. Python version: (160, 2) MATLAB148, 2)


When comparing the results row-by-row, I found that many rows differ, even though they appear visually similar. Comparing the sets of rows reveals that both matrices contain 188 rows that are not in the other — likely due to floating-point precision differences (e.g., -9.9 vs -9.899999999999999).

The matrices are way too big so i can't really attach them here (kind of hard to create other examples with less rows).

def compare_as_sets(p, m):
p_set = set(map(tuple, p))
m_set = set(map(tuple, m))

only_in_p = p_set - m_set
only_in_m = m_set - p_set

print(f"Rows only in Python: {len(only_in_p)}")
print(f"Rows only in MATLAB: {len(only_in_m)}")

if only_in_p:
print("\nRows only in Python:")
for row in sorted(only_in_p):
print(f" {row}")
if only_in_m:
print("\nRows only in MATLAB:")
for row in sorted(only_in_m):
print(f" {row}")

# import the matlab before unique()
m_unique = pd.read_csv("avail_user_data.csv", header=None).to_numpy()
m_non = pd.read_csv("avail_not_unique_user.csv", header=None).to_numpy()

# import python version
p_unique = pd.read_csv("python_unique.csv", header=None).to_numpy()
p_unique = np.round(p_unique, 6)
p_non = pd.read_csv("python_nonunique.csv", header=None).to_numpy()


The some of the outputs of this function gives me this:

Rows only in Python: 188
Rows only in MATLAB: 188

Rows only in Python:
(np.float64(-11.5), np.float64(-9.899999999999999))
(np.float64(-11.5), np.float64(-7.699999999999998))
(np.float64(-11.5), np.float64(-4.3999999999999995))
(np.float64(-11.5), np.float64(-3.299999999999999))
(np.float64(-11.5), np.float64(-2.1999999999999993))
(np.float64(-11.5), np.float64(-1.0999999999999979))
.
.
.
Rows only in MATLAB:
(np.float64(-11.5), np.float64(-9.9))
(np.float64(-11.5), np.float64(-7.7))
(np.float64(-11.5), np.float64(-4.4))
(np.float64(-11.5), np.float64(-3.3))
(np.float64(-11.5), np.float64(-2.2))
(np.float64(-11.5), np.float64(-1.1))


I actually have no idea what is going on. Some cases both before and after are same. But some peculiar cases they give me wrong matrices after unique().

Continue reading...