[Python] Trouble scraping dynamic lottery results table – inconsistent parsing

I’ve been trying to scrape lottery results from a website that shows draws. The data is presented in a results table, but I keep running into strange issues where sometimes the numbers are captured correctly, and other times the scraper either misses some rows or duplicates entries.

Code:

import requests
from bs4 import BeautifulSoup

url = "https://star49s.com/results/teatime"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

results = []
for row in soup.select("div.results-row"):
numbers = [num.get_text(strip=True) for num in row.select("span.ball")]
date = row.select_one("span.date").get_text(strip=True) if row.select_one("span.date") else None
if numbers:
results.append({"date": date, "numbers": numbers})

print(results)


The output is inconsistent:

• 
  Sometimes it captures the latest results fine.
  
  
• 
  Other times it skips certain rows.
  
  
• 
  Occasionally, I get duplicated results when re-running the script.
  

I suspect the page might be partially dynamic (maybe using JavaScript rendering) or that I’m not selecting the correct elements. I’m not sure if I should switch to something like Selenium or Playwright, or if I just need to refine my CSS selectors.

What’s the most reliable way to scrape a results page like this so I can always get the latest draw numbers without missing or duplicating rows? Should I rely on a headless browser or is there a way to do this cleanly with requests + BeautifulSoup?

Continue reading...