I’m new to Python (version 3.13.2) and using Selenium with Firefox to scrape data from a racecard webpage, specifically the "Sectional Times" tab at a URL like https://www.attheraces.com/racecard/Cheltenham/11-March-2025/1320. My goal is to collect 88 sectional times, which means 8 per horse for 11 horses, but I’m only getting 77 values. I think the 11 missing ones might be related to horse numbers being filtered out, and I need help capturing all 88. I’ve attached a screenshot named after_times_wait.png to show the grid layout. Can anyone help with these questions? How can I adjust the filter to get all 88 sectional times, especially if some have just one decimal place? Is skipping the first 69 cells to avoid headers correct, or should I find a better way to detect where the horse data starts? Any suggestions to ensure all 11 horses are processed, even if data is incomplete? Thanks for any assistance! I loaded the webpage using Selenium, clicked the "Sectional Times" tab, scrolled to load all content, and scraped 168 total cells. I tried filtering the cells to keep only decimal numbers with 1 to 2 digits before and after the decimal point (like 61.58 or 6.5), excluding values of 10 or less, which I think are horse numbers (e.g., 5). I processed 9 out of 11 horses, each with 8 sectionals, by padding with 0.00 when needed. I also skipped the first 69 cells assuming they were headers based on the raw data pattern. I expected to get 88 sectional times, with 8 valid sectionals for each of the 11 horses (e.g., one horse starting with 61.58), resulting in a complete dataset for all horses. Instead, I got only 77 cells and processed 9 horses. I loaded the webpage using Selenium, clicked the "Sectional Times" tab, scrolled to load all content, and scraped 168 total cells. I tried filtering the cells to keep only decimal numbers with 1 to 2 digits before and after the decimal point (like 61.58 or 6.5), excluding values of 10 or less, which I think are horse numbers (e.g., 5). I processed 9 out of 11 horses, each with 8 sectionals, by padding with 0.00 when needed. I also skipped the first 69 cells assuming they were headers based on the raw data pattern. What were you expecting (Paste This into the 'What were you expecting' Field): I expected to get 88 sectional times, with 8 valid sectionals for each of the 11 horses (e.g., one horse starting with 61.58), resulting in a complete dataset for all horses. Instead, I got only 77 cells and processed 9 horses. atr sectional page I’ve included my code and debug output below for reference. Here’s the debug information I gathered: All time cells found: 168 First 10 raw time cells: ['Pos', 'Silk', 'Horse', '1', '', ...] Filtered time cells found: 77 Accepted samples: ['61.58', '56.55', '30.02', '27.18', '26.97'] Rejected samples: ['5.', '11., '6., '4., '7., '10., '2., '8., '1., '3.'] Horses Found in final data: 9 And here’s my code attempt: from selenium import webdriver from selenium.webdriver.firefox.service import Service from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC import time service = Service("C:\\Scraping\\geckodriver.exe") driver = webdriver.Firefox(service=service) driver.get("https://www.attheraces.com/racecard/Cheltenham/11-March-2025/1320") time.sleep(15) # Click Sectional Times tab tab = WebDriverWait(driver, 40).until( EC.presence_of_element_located((By.XPATH, '//a[contains(@class, "tab") and contains(normalize-space(.), "Sectional")]')) ) driver.execute_script("arguments[0].click();", tab) time.sleep(15) # Scroll for _ in range(5): driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") time.sleep(5) driver.save_screenshot("after_times_wait.png") # Scrape and filter time_cells = driver.find_elements(By.XPATH, "//div[@id='tab-sectionals-times']//div[contains(@class, 'card-cell')]") time_cells_filtered = [] for cell in time_cells[69:]: # Skip estimated 69 header cells spans = cell.find_elements(By.XPATH, ".//span[contains(@class, 'visible')]") for span in spans: text = span.text.strip() cleaned = text.replace(' ', '').replace('x', '') if '.' in text and cleaned.replace('.', '').isdigit(): parts = cleaned.split('.') if len(parts) == 2 and all(part.isdigit() for part in parts) and len(parts[0]) in [1, 2] and len(parts[1]) in [1, 2] and float(cleaned) > 10: time_cells_filtered.append(cell) print("Filtered time cells found:", len(time_cells_filtered)) driver.quit() Continue reading...
