[Python] Deep equality comparison of python nested dicts containing floats [closed]

Use Case


I want to do a deep equality comparison between two arbitrarily deeply nested and composed dictionaries and/or lists, where many of the scalar values are floats.

ie: dictionaries whole elements maybe other dicts or lists arbitrarily deeply.

Specific Use Case


I have a bunch of data files written in JSON and I want to be able to compare them for equality, but be able to include an equality tolerance when comparing float scalar values.

Problem


While the standard python dictionary comparison, e.g. a == b works in theory for dictionaries of any composition, it falls apart in this use case because standard equality comparison for floats in python fails to take precision into account.

The ideal fix would be to allow/override <float>.__eq__() to call math.isclose(), but you can't override a built-in's methods, so that is out.

Solution and Question


My solution is three parts (below) and my question is whether anyone has any better ideas on how to do this, or suggestions about how to improve?

Solution

1. Define a class to hold float values, and define __eq__ to call math.isclose()
2. Walk through my JSON derived dicts and replace all float scalars with instances of (1)
3. The just use standard python a == b comparison, which will handle arbitrarily composed dicts, but now when comparing floats will use math.isclose()

1 - New Floaty Class


class FloatPlus:

_default_rel_tol = 0.001
rel_tol = _default_rel_tol

def __init__(self, value):
self.value = float(value)

def __eq__(self, other, rel_tol=None):
return math.isclose(self.value, float(other), rel_tol=rel_tol if rel_tol else FloatPlus.rel_tol)

def __float__(self):
return self.value
def __repr__(self):
return str(self.value)

2 - Walk through dicts and convert float to FloatPlus


def convert(node, key=None):
if type(node) is dict:
return {k: convert(v, k) for k,v in node.items()}
elif type(node) is list:
return [convert(x, key) for x in node]
elif type(node) is float:
return FloatPlus(node)
else:
return node

3 - Use standard dict (deep) comparison


a = {
"xyz": 1.1,
"inner1": {
"blah": 100.1,
"blah2": 100.1,
}
}
b = {
"xyz": 1.1,
"inner1": {
"blah": 100.10001,
"blah2": 100.1,
}
}

c_a = convert(a)
c_b = convert(b)

x = FloatPlus(5)
y = FloatPlus(5.0001)
print(f'x==y: {x==y}') # True
print(f'x==y: {x.__eq__(y, rel_tol=0.0000001)}') # False with reduced tolerance

print(f'a == b: {a==b}') # False
print(f'c_a == c_b: {c_a==c_b}'). # now True
FloatPlus.rel_tol = 0.0000000001
print(f'c_a == c_b with reduced tol: {c_a==c_b}') # False again with reduced tolerance

Continue reading...