[Python] Access text elements with dynamically loaded classes using Selenium

Discussão em 'Python' iniciado por Stack, Março 12, 2025.

  1. Stack

    Stack Membro Participativo

    I am trying to fetch the co-ordinates for real estate properties but I am unable to fetch the longitude and latitude as it opens without a proper location, I also tried getting them via the google map url but due to some reason the current url doesn't give me the gmaps url but the website (real estate) instead. Can anyone help me? The images for the exact button and google map are: Website Button, Google Maps.

    As you see, when the driver opens the google maps tab, the co-ordinates are listed there, those are the ones I am trying to fetch but can't seem to because the class is dynamically loaded. I saw some tutorials using the div tag with role, which here is main, but that doesn't seem to work either.

    As my last option I am using the current google maps url to fetch the coordinates which changes a bit here and there as the sites change in the loop. but doing it using the html part of google maps is something I want to figure out and need help with. Looking forward to any answers!

    import requests
    from bs4 import BeautifulSoup
    from selenium import webdriver
    from time import sleep
    from selenium.webdriver.chrome.service import Service
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC

    path = r'chromedriver.exe'

    service = Service(path)
    browser = webdriver.Chrome(service=service)

    links= []
    num = 0

    url = f'https://www.quikr.com/homes/property/residential-for-sale-in-bangalore-cid_23?q=eyJjbHVzdGVyQ2l0eSI6WyIyMyJdfQ%3D%3D&page=1'

    headers = {'User-Agent': 'Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; {}; Alawar 2.08; .NET CLR 1.0.3705)'}

    req = requests.get(url, headers=headers)
    soup = BeautifulSoup(req.content, 'html')

    sites = soup.find_all('div',class_='listingviewtiles')

    for site in sites:
    for link in site.find_all('a',class_='img-anchor', href=True):
    links.append(link['href'])

    num+=1
    print(num, 'pages done')

    link = links[1]

    r = requests.get(link, headers=headers)
    soup2 = BeautifulSoup(r.content, 'html')
    price = soup2.find('div', class_ = 'sc-jj9qsf-0 fmMIFj sc-gs8z27-5 ksEDsX').text.strip()

    print(price)

    browser.get(link)
    browser.maximize_window()
    location_button = browser.find_element(By.CSS_SELECTOR, '.sc-ldbotn-0.hdNuPA.sc-yzpzu-3.eLCRYW.hA.bt2.bt1')
    sleep(0.5)
    browser.execute_script("arguments[0].click();", location_button)
    google_maps_url = browser.current_url
    print(f"Google Maps URL: {google_maps_url}")

    import re

    current_url = browser.current_url
    print("Google Maps URL:", current_url)
    # Regex pattern to find latitude and longitude
    match = re.search(r"@([-.\d]+),([-.\d]+)", current_url)

    if match:
    latitude, longitude = match.groups()
    print(f"Latitude: {latitude}, Longitude: {longitude}")
    else:
    print("Coordinates not found in URL!")

    Continue reading...

Compartilhe esta Página