[Python] find_elements only gets the first element when using Selenium

Discussão em 'Python' iniciado por Stack, Setembro 25, 2024.

  1. Stack

    Stack Membro Participativo

    I have a Python program that uses undetected Chromedriver to scrape product data from Walmart. The following is a simplified version of my code, but it only gets the first element. I’ve spent hours trying to figure this out. Can someone show me where the issue is?

    import time
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    import undetected_chromedriver as uc


    url='https://www.walmart.com/search?q=lego&facet=retailer_type%3AWalmart'

    driver = uc.Chrome(enable_cdp_events=True)
    driver.implicitly_wait(15)
    driver.maximize_window()
    driver.get(url)

    sleep(5)
    for _ in range(3):
    try:
    scroll_value = driver.execute_script("return document.body.clientHeight;")
    driver.execute_script(f"return window.scrollBy(0, {str(scroll_value)})")
    sleep(1)
    except:
    pass

    items_list = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "/html/body/div[1]/div[1]/div/div[2]/div/main/div/div[2]/div/div/div[1]/div[2]/div/section/div"))).find_elements(By.CSS_SELECTOR, '[data-testid="list-view"]')

    for raw_item in items_list:
    product_name = WebDriverWait(raw_item, 5).until(EC.presence_of_element_located((By.XPATH, "//div[2]/span/span"))).text.strip()
    print(product_name)


    driver.quit()

    Continue reading...

Compartilhe esta Página