[Python] Is using an LLM a better solution for parsing inconsistently formatted scripts

I am woring on an app that reads scripts, like movie scripts, and initially I wrote my own line-based parser to detect speaker lines, transitions, and dialogue blocks

But scripts vary widely and even small inconsistencies break the logic, would an LLM (e.g GPT-4) be a better choice for robustly extracting speaker-dialogue pairs from scripts

What would be the pros and cons of using LLMs for parsing compared to writing out static code logic.

Also is there a python library for a task like this

I wrote a custom parser that processes the script line by line. It identifies speaker lines using conditions like line.isupper() and skips known transitions or scene headings. It works decently for clean, conventionally formatted screenplays (e.g, INT/EXT. scene headers, ALL CAPS speaker names, etc).

However, when formatting varies even slightly, like missing scene headers, extra whitespace, lowercase speaker names, or characters speaking in parenthesis, it breaks.

The parser function ends up either skipping entire dialogue blocks or assigning lines to the wrong speaker

I expected the custom logic to be albe to parse most scripts that follow common screenplay conventions, but i quickly realized that real-world scripts often contain a lot of variation and edge cases. That made me to wonder if using a lareg language model like GPT-4 would be a more robust approach for parsing speaker-dialogu pairs, given its flexibility with understanding text

Continue reading...