[Python] Selenium 4.25 opens Chrome 136 with existing profile to "New Tab" instead of...

I'm using Python with Selenium 4.25.0 to automate Google Chrome 136. My goal is to have Selenium use my existing, logged-in "Default" Chrome profile to navigate to a specific URL (https://aistudio.google.com/prompts/new_chat) and perform actions.

The Problem:

When I execute my script:

Chrome launches successfully.

It clearly loads my "Default" profile, as I see my personalized "New Tab" page (title: "新分頁", URL: chrome://newtab/) with my usual shortcuts, bookmarks bar, and theme. This confirms the --user-data-dir and --profile-directory arguments are pointing to the correct profile.

However, the subsequent driver.get("https://aistudio.google.com/prompts/new_chat") command does not navigate the browser. The browser remains on the chrome://newtab/ page.

I am very diligent about ensuring all chrome.exe processes are terminated (checked via Task Manager) before running the script.

Environment:

OS: Windows 11 Pro, Version 23H2

Python Version: 3.12.x

Selenium Version: 4.25.0

Chrome Browser Version: 136.0.7103.93 (Official Build) (64-bit)

ChromeDriver Version: 136.0.7103.xx (downloaded from official "Chrome for Testing" site for win64, matching Chrome's major.minor.build)

Simplified Code Snippet:

import time
import os # For path checks
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.common.exceptions import WebDriverException

# --- Configuration ---
# User needs to replace CHROME_DRIVER_PATH with the full path to their chromedriver.exe
CHROME_DRIVER_PATH = r'C:\path\to\your\chromedriver-win64\chromedriver.exe'
# User needs to replace YourUserName with their actual Windows username
CHROME_PROFILE_USER_DATA_DIR = r'C:\Users\YourUserName\AppData\Local\Google\Chrome\User Data'
CHROME_PROFILE_DIRECTORY_NAME = "Default" # Using the standard default profile

TARGET_URL = "https://aistudio.google.com/prompts/new_chat" # Example

def setup_driver():
print(f"Driver Path: {CHROME_DRIVER_PATH}")
print(f"User Data Dir: {CHROME_PROFILE_USER_DATA_DIR}")
print(f"Profile Dir Name: {CHROME_PROFILE_DIRECTORY_NAME}")

if not os.path.exists(CHROME_DRIVER_PATH):
print(f"FATAL: ChromeDriver not found at '{CHROME_DRIVER_PATH}'.")
return None
if not os.path.isdir(CHROME_PROFILE_USER_DATA_DIR):
print(f"FATAL: Chrome User Data dir not found at '{CHROME_PROFILE_USER_DATA_DIR}'.")
return None

chrome_options = Options()
chrome_options.add_argument(f"--user-data-dir={CHROME_PROFILE_USER_DATA_DIR}")
chrome_options.add_argument(f"--profile-directory={CHROME_PROFILE_DIRECTORY_NAME}")

chrome_options.add_experimental_option("excludeSwitches", ["enable-automation", "load-extension"])
chrome_options.add_experimental_option('useAutomationExtension', False)
# chrome_options.add_argument("--disable-blink-features=AutomationControlled") # Tried with and without

chrome_options.add_argument("--start-maximized")

try:
service = Service(executable_path=CHROME_DRIVER_PATH)
driver = webdriver.Chrome(service=service, options=chrome_options)
print("WebDriver initialized.")
return driver
except WebDriverException as e:
print(f"FATAL WebDriverException during setup: {e}")
if "user data directory is already in use" in str(e).lower():
print(">>> Ensure ALL Chrome instances are closed via Task Manager.")
return None
except Exception as e_setup:
print(f"Unexpected FATAL error during setup: {e_setup}")
return None

def main():
print("IMPORTANT: Ensure ALL Google Chrome instances are FULLY CLOSED before running this script.")
input("Press Enter to confirm and continue...")

driver = setup_driver()
if not driver:
print("Driver setup failed. Exiting.")
return

try:
print(f"Browser launched. Waiting a few seconds for it to settle...")
print(f"Initial URL: '{driver.current_url}', Initial Title: '{driver.title}'")
time.sleep(4) # Increased wait after launch for profile to fully 'settle'

print(f"Attempting to navigate to: {TARGET_URL}")
driver.get(TARGET_URL)
print(f"Called driver.get(). Waiting for navigation...")
time.sleep(7) # Increased wait after .get() for navigation attempt

current_url_after_get = driver.current_url
current_title_after_get = driver.title
print(f"After 7s wait - Current URL: '{current_url_after_get}', Title: '{current_title_after_get}'")

if TARGET_URL not in current_url_after_get:
print(f"NAVIGATION FAILED: Browser did not navigate to '{TARGET_URL}'. It's still on '{current_url_after_get}'.")
# Could also try JavaScript navigation here for more info
# print("Attempting JavaScript navigation as a fallback test...")
# driver.execute_script(f"window.location.href='{TARGET_URL}';")
# time.sleep(7)
# print(f"After JS nav attempt - URL: '{driver.current_url}', Title: '{driver.title}'")
else:
print(f"NAVIGATION SUCCESSFUL to: {current_url_after_get}")


except Exception as e:
print(f"An error occurred during main execution: {e}")
finally:
print("Script execution finished or errored.")
input("Browser will remain open for inspection. Press Enter to close...")
if driver:
driver.quit()


if __name__ == "__main__":
# Remind user to update paths if placeholders are detected
if r"C:\path\to\your\chromedriver-win64\chromedriver.exe" == CHROME_DRIVER_PATH or \
r"C:\Users\YourUserName\AppData\Local\Google\Chrome\User Data" == CHROME_PROFILE_USER_DATA_DIR:
print("ERROR: Default placeholder paths are still in the script.")
print("Please update CHROME_DRIVER_PATH and CHROME_PROFILE_USER_DATA_DIR with your actual system paths.")
else:
main()


Console Output (when it gets stuck on New Tab):

Setting up Chrome driver from: C:\Users\stat\Downloads\chromedriver-win64\chromedriver-win64\chromedriver.exe
Attempting to use Chrome User Data directory: C:\Users\stat\AppData\Local\Google\Chrome\User Data
Attempting to use Chrome Profile directory name: Default
WebDriver initialized successfully.
Browser launched. Waiting a few seconds for it to settle...
Initial URL: 'chrome://newtab/', Initial Title: '新分頁'
DevTools remote debugging requires a non-default data directory. Specify this using --user-data-dir.
[... some GCM / fm_registration_token_uploader errors may appear here ...]
Attempting to navigate to: https://aistudio.google.com/prompts/new_chat
Called driver.get(). Waiting for navigation...
After 7s wait - Current URL: 'chrome://newtab/', Title: '新分頁'
NAVIGATION FAILED: Browser did not navigate to 'https://aistudio.google.com/prompts/new_chat'. It's still on 'chrome://newtab/'.


What I've Already Verified/Tried:

ChromeDriver version precisely matches the Chrome browser's major.minor.build version (136.0.7103).

All chrome.exe processes are terminated via Task Manager before script execution.

The paths for CHROME_DRIVER_PATH, CHROME_PROFILE_USER_DATA_DIR, and CHROME_PROFILE_DIRECTORY_NAME are correct for my system.

The browser visibly loads my "Default" profile (shows my theme, new tab page shortcuts).

Tried various time.sleep() delays.

The "DevTools remote debugging requires a non-default data directory" warning appears, as do some GCM errors, but the browser itself opens with the profile.

My Question:

Given that Selenium successfully launches Chrome using my specified "Default" profile (as evidenced by my personalized New Tab page loading), why would driver.get() fail to navigate away from chrome://newtab/? Are there specific Chrome options for Selenium 4.25+ or known issues with Chrome 136 that could cause this behavior when using an existing, rich user profile, even when Chrome is fully closed beforehand? How can I reliably make driver.get() take precedence over the default New Tab page loading in this scenario?

Continue reading...