[Python] driver.page_source returns only meta name="ROBOTS" content="NOINDEX, NOFOLLOW" using...

Discussão em 'Python' iniciado por Stack, Maio 26, 2025.

  1. Stack

    Stack Membro Participativo

    I want to scrape one website, to get the page content with this code:

    from selenium import webdriver
    from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
    driver = webdriver.Remote("http://adress:4444/wd/hub", DesiredCapabilities.CHROME)
    link = 'website_url'
    driver.get(link)
    s = driver.page_source
    print((s.encode("utf-8")))
    driver.quit()


    this is what receive:

    <meta name="ROBOTS" content="NOINDEX, NOFOLLOW">


    I also tried a lot of different ways, Luminati, proxy newipnow, phantomjs, but does not work, any suggestions what else i can try to solve this?

    Continue reading...

Compartilhe esta Página