[Python] User Input Processing(getting rid of typos: when they do not have space(s)

I'm building graph query to Cypher and trying to process user input with fuzzy matching and then filtering with Lavenshtein distance. This works really well for the most cases for me but I struggle with typos when user makes a type with not putting space where it should be. Reason being why it fails because I compare not full string with a full string but use sliding window for that. And when I have single word when it suppose to be 2 words, it finds so many false positives and rarely hits the target. Main issue is that I compare short string like Bangna with a really long addresses, but its not only related to addresses and many different domain specific filters are present and can face same issue.

Example issue that I have :

4371, IDEO MOBI Sukhumvit Eastgate, 1 Sukhumvit Rd, Khwaeng Bang Na, Bang Na, Bangkok 10260

And addresses that includes Bang Kapi, Bang chak being returned

Maybe im just going in the wrong direction at all but I feel stuck with my logic here and struggling to find a simple solution that would not increase query time immensely

My code that build a cypher query:

fulltext_index = f"{node_label}_fulltext"
modified_value = " ".join(word + "~" for word in value.split())
tokens = value.lower().split()
word_count = len(tokens)

# Split input into words and number positions
word_indices = [i for i, tok in enumerate(tokens) if not tok.isdigit()]
number_indices = [(i, tok) for i, tok in enumerate(tokens) if tok.isdigit()]

# Dynamic Levenshtein distance
if any(char.isspace() for char in value):
dist_dynamic = math.ceil(len(value.replace(" ", "")) / 3)
else:
dist_dynamic = len(value.replace(" ", "")) // 3

fulltext_query = (
f'CALL db.index.fulltext.queryNodes("{fulltext_index}", "{modified_value}") YIELD node, score '
f'WITH node, score, '
f'split(apoc.text.replace(toLower(node.value), "[.,/()]", " "), " ") AS node_tokens, '
f'split(toLower("{value}"), " ") AS input_tokens '
f'WITH node, score, node_tokens, input_tokens, '
f'size(node_tokens) AS node_len, '
f'size(input_tokens) AS input_len '
f'WITH node, score, node_tokens, input_tokens, node_len, input_len, '
f'range(0, node_len - {word_count}) AS node_indices '
f'UNWIND node_indices AS n_idx '
f'WITH node, score, '
f'node_tokens[n_idx..n_idx+{word_count}] AS node_phrase_tokens, '
f'split(toLower("{value}"), " ") AS input_phrase_tokens '
)

# Levenshtein only on non-number tokens
fulltext_query += (
f'WITH node, score, node_phrase_tokens, input_phrase_tokens, '
f'apoc.text.levenshteinDistance('
f'apoc.text.join([i IN range(0, size(input_phrase_tokens)-1) WHERE NOT input_phrase_tokens =~ "\\\\d+" | input_phrase_tokens], " "), '
f'apoc.text.join([i IN range(0, size(node_phrase_tokens)-1) WHERE NOT node_phrase_tokens =~ "\\\\d+" | node_phrase_tokens], " ")'
f') AS dist '
)

# Build WHERE clause
where_conditions = [f'dist <= {dist_dynamic}']
for idx, num in number_indices:
# Match the number exactly at the same position
where_conditions.append(f'node_phrase_tokens[{idx}] = "{num}"')

fulltext_query += f'WHERE {" AND ".join(where_conditions)} '

fulltext_query += (
f'RETURN node, min(dist) AS distance, score '
f'ORDER BY distance ASC, score DESC '
f'LIMIT {single_structured_query["limit"]}'
)


So far I have 0 idea of how to tackle this problem. And I just want to increase my precision with these type of typos. I would appreciate any direction or papers I can read or techniques

Continue reading...