[Python] How to extract intraday stock data from onvista with python and playwright/selenium?

Using 3.9.6 and playwright on MacOS 14.7.5 I am trying to extract the public available intraday stock data from a webpage like

https://www.onvista.de/aktien/Airbus-Group-EADS-Aktie-NL0000235190


To make the Intraday data load you have to click the "Intraday" button. Here is a script I tried but it seems it did not click the Intraday button



import asyncio
from playwright.async_api import async_playwright
from bs4 import BeautifulSoup

async def run():
url = "https://www.onvista.de/aktien/Airbus-Group-EADS-Aktie-NL0000235190"

async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await browser.new_page()
await page.goto(url)

# Wait for the chart area to appear
await page.wait_for_selector("svg")

# Click the 'Intraday' button
try:
await page.click("text=Intraday")
await page.wait_for_timeout(3000) # wait for chart to update
except Exception as e:
print("Couldn't click Intraday:", e)

# Get the full HTML after the chart is rendered
content = await page.content()

# Parse with BeautifulSoup
soup = BeautifulSoup(content, "html.parser")
paths = soup.find_all("path")

print("Extracted SVG path data:")
for path in paths:
d_attr = path.get("d")
if d_attr and len(d_attr) > 100: # skip small icons etc.
print("d:", d_attr[:150], "...\n")

await browser.close()

# Run the async function
asyncio.run(run())


The output is

Couldn't click Intraday: Page.click: Timeout 30000ms exceeded.


Is there a way to fix that script? Or would selenium be the better choice?

Continue reading...