[Python] How can I remove footnote references while converting a PDF to a data frame?

I am trying to extract a PDF from a table using Python. The columns that have values have a footnote reference with them as well. When I extract the table, 752^5 is becoming 7525. I'd like to remove the footnote reference.

I tried to remove them using pdfplumber, but it isn't working:

import pdfplumber
import pandas as pd
from IPython.display import display

pdf_path = "reports_lcc_2024_LTA BPLRT LCC Report 2024 - signed 1.pdf"
page_number = 6 # Page 7 in actual PDF
table_number = 3 # Table to extract (1-based index)

with pdfplumber.open(pdf_path) as pdf:
page = pdf.pages[page_number]
chars = page.chars

# Compute average font size and top
sizes = [float(c['size']) for c in chars]
tops = [float(c['top']) for c in chars]
avg_size = sum(sizes) / len(sizes)
avg_top = sum(tops) / len(tops)

size_thresh = avg_size * 0.8
top_thresh = avg_top * 0.8

# Filter out superscript-style characters
filtered_chars = [
c for c in chars
if not (float(c['size']) < size_thresh and float(c['top']) < top_thresh and c['text'].isdigit())
]

# ✅ Patch pdfplumber page using internal API (safe, no deepcopy)
page._objs["char"] = filtered_chars # trick: replace char list

# Extract table
tables = page.extract_tables()
if tables and len(tables) >= table_number:
table = tables[table_number - 1]
df = pd.DataFrame(table[1:], columns=table[0])
display(df)
else:
print(f"No table #{table_number} found.")


How can I remove the footnote references properly?

Continue reading...