[Python] difflib.SequenceMatcher suddenly returns different similarity ratio without code or...

We're using Python’s difflib.SequenceMatcher to compare strings in a production system. Here's the simplified relevant code:

from difflib import SequenceMatcher

similarity = SequenceMatcher(
None,
normalized_transcript,
normalized_expected
).ratio()


Until 4:10 PM UTC today, the above code was returning a similarity ratio above our internal threshold for a specific string comparison.

After that time, without any change in our code, server configuration, or environment, the same comparison started returning a lower similarity score, failing the threshold check.

Some key facts:

• 
  The behavior changed consistently across all environments: both development and production.
  
  
• 
  Servers run a mix of Windows (dev) and Unix (dev + prod), so this is not likely to be an OS-specific issue.
  
  
• 
  There were no code deployments, no dependency changes, and no environment variable alterations.
  
  
• 
  We are aware that SequenceMatcher works entirely locally—there are no third-party requests or models involved.
  
  
• 
  We’ve validated that the inputs to SequenceMatcher are identical to previous values (confirmed via logs).
  

Important detail: the string normalized_transcript comes from OpenAI API completions. That’s the only potentially "variable" external component in the system. However, the strings in question are very short, and we’ve historically seen consistent outputs from OpenAI for this prompt setup.

This behavior is baffling. Is there any known edge case, maybe time-sensitive internal optimization, or anything else that could explain this sudden change in behavior from SequenceMatcher?

Continue reading...