[Python] Extract Data from Unstructured PDFs, Word, and Excel into a Standardized Excel Format...

I’m developing an application to extract key information and tables from buy contracts provided by various shipping companies. These documents come in different formats and file types: • File Types: PDF, Word (.docx), Excel (.xlsx) • Structure: No consistent layout; each company uses its own format

Goal: • Extract relevant text and tables from these documents • Consolidate the data into an Excel file with a fixed, standardized format • Apply formulas within the Excel file based on the extracted contract information

Challenges: • Variable Document Structures: Lack of a consistent format makes template-based extraction impractical • Data Types: Need to extract both text (e.g., contract terms, dates) and tables (e.g., item lists, pricing) • Formula Application: Need to automatically apply Excel formulas (e.g., calculations, conditional formatting) based on the extracted data

What I’ve Tried: • OCR for PDFs: Using Tesseract OCR, but accuracy varies with complex layouts • Parsing Libraries: • python-docx for Word documents • pandas and openpyxl for Excel files • Rule-Based Extraction: Implemented regular expressions, but it’s not scalable due to format variability • Excel Automation: Used openpyxl to write data to Excel, but unsure how to apply formulas dynamically based on the data

Questions: 1. What approaches or tools are recommended for extracting data from documents with highly variable and unstructured formats? 2. Can machine learning or NLP techniques help interpret and extract relevant information from these documents? 3. How can I accurately extract tables when their structures differ significantly across documents? 4. What are the best practices for consolidating this extracted data into a standardized Excel format? 5. How can I programmatically apply Excel formulas based on the extracted contract information?

Continue reading...