[Python] Adding user agent in chrome options in selenium

Discussão em 'Python' iniciado por Stack, Março 24, 2025.

  1. Stack

    Stack Membro Participativo

    i'm performing data crawling on a webpage using selenium. this is my code:

    from selenium import webdriver
    from selenium.webdriver.chrome.service import Service
    from selenium.webdriver.chrome.options import Options
    from time import sleep
    import tempfile
    from fake_useragent import UserAgent


    ua = UserAgent(browsers=['Chrome'], os=['Linux'], platforms=['desktop'])
    CHROMEDRIVER_PORT = 9515

    chrome_service = Service(executable_path="/path/to/chromedriver_linux64", log_output="/tmp/chromedriver.log", port=CHROMEDRIVER_PORT)
    chrome_options = Options()
    chrome_options.binary_location = "/path/to/google-chrome"
    chrome_options.add_argument("start-maximized")
    chrome_options.add_argument("--headless=new")
    chrome_options.add_argument("--disable-gpu")
    chrome_options.add_argument("--disable-dev-shm-usage")
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--dns-prefetch-disable")
    chrome_options.add_argument(f"user-agent={ua.random}")

    driver = webdriver.Chrome(service=chrome_service, options=chrome_options)
    print("Chrome Browser Invoked")

    driver.get("<url>")
    sleep(2)

    print("Page Title:", driver.title)

    driver.quit()


    i run this code on my local with internet, it completely fine but on my server (no GUI, no internet), it get timeout.

    urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9515): Read timed out. (read timeout=120)

    in general:
    - delete the user-agent -> run but cannot scan the webpage
    - keep the user-agent -> got timeout

    Continue reading...

Compartilhe esta Página