[Python] Extracting event dates from noisy social worker case notes for temporal feature...

Question: Are there better approaches than regex for extracting event dates (including relative) from noisy text? Are there NLP tools that can help disambiguate multiple date mentions in various inconsistent formats?

I'm working with social worker case notes in free-text format, applying NLP and ML to classify events like risks. My goal is to extract the date of an event described in a note and calculate how long ago it occurred relative to the note's date of entry.

The challenge is that these notes are written in varied styles, with no standard formatting. Dates may be expressed explicitly ("10th December", "10/12" -- UK style day then month) or relatively ("three weeks ago", "last month", "on the 10th" -- which one can reasonably assume means the most recent 10th going back in the calendar). Punctuation is inconsistent. A single note may mention multiple dates, not all of which relate to the target event ("visited on 10th to check on them after their fall of 10/09")

This is my naive premise, please let me know if I'm talking nonsense: If they're explicitly relative then I feel the 4 word n-grams that have seemed appropriate previously on our material should do an OK job. But if they're naming an explicit date then there can only be relevance when one considers the date it was recorded on. Because these case notes can refer to events years previously. I'm getting much lower accuracy from my last logistic regression run on different data because with that the class separation was much clearer.

I've tried regex-based pattern matching to identify event descriptions and extract surrounding context, but this approach is brittle due to the noise, mistakes/typos and ambiguity in the text.

This doesn't need to be perfect . I'm looking to engineer a feature that improves classification accuracy as the condition the data has been labelled for is "(roughly) within the last six weeks". I plan to try other ML approaches and may eventually move to LLMs, but I'd like to explore lightweight options first.

I'm still at the thinking phase so forgive the lack of code.

Continue reading...