[Python] Deployment Failing: Chrome Non-Headless Mode Requires Xvfb Setup on Railway

I’m deploying a Python app that uses Botasaurus (a browser scrapping automation library) to parse a website requiring Chrome in non-headless mode (headless=False). The app works locally with Docker (Windows 11 + Docker Desktop), but deployment on Railway fails with a Chrome connection error. Despite configuring Xvfb to emulate a display, ChromeDriver cannot connect to Chrome during runtime.

Error Traceback (Deploy Logs):

Traceback (most recent call last):

File "/opt/venv/lib/python3.12/site-packages/botasaurus_driver/core/retry_on_error.py", line 22, in wrapper

created_result = func(*args, **kwargs)

^^^^^^^^^^^^^^^^^^^^^

File "/opt/venv/lib/python3.12/site-packages/botasaurus_driver/core/browser.py", line 374, in run

self.info = ensure_chrome_is_alive(chrome_url)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/venv/lib/python3.12/site-packages/botasaurus_driver/core/browser.py", line 59, in ensure_chrome_is_alive

raise Exception(f"Failed to connect to Chrome URL: {url}.")

Exception: Failed to connect to Chrome URL: http://127.0.0.1:54411/json/version.


Key Observations:

Botasaurus/ChromeDriver cannot connect to Chrome’s debugger URL (127.0.0.1:54411).


The error suggests Chrome isn’t running or isn’t accessible in the container.

Relevant Code and Configuration:

FROM selenium/standalone-chrome:latest # Base image includes Chrome

USER root

WORKDIR /app

RUN apt-get update && apt-get install -y xvfb # Explicit Xvfb installation

COPY requirements.txt main.py /app/

RUN pip install --no-cache-dir -r requirements.txt

COPY . /app

ENV HEADLESS=False

CMD ["xvfb-run", "-a", "--server-args=-screen 0 1920x1080x24 -ac", "python", "main.py"]


Botasaurus Configuration (Python Code):

@browser(
remove_default_browser_check_argument=True,
add_arguments=[
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-gpu",
"--disable-dev-shm-usage"
]
)
def scrape_visa(driver: Driver, data) -> None:
# ... Botasaurus logic ...


Steps We Tried to Fix the Issue:

• Added Xvfb to Docker

Installed xvfb in the Dockerfile to emulate a display for Chrome.

Command: RUN apt-get update && apt-get install -y xvfb
Expected: Chrome would launch in non-headless mode with a virtual display.
Result: Still fails with Failed to connect to Chrome URL.
​

• Used xvfb-run in CMD

Wrapped the Python script with xvfb-run to force GUI support.

Command: CMD ["xvfb-run", "-a", "--server-args=-screen 0 1920x1080x24 -ac", "python", "main.py"]
Expected: Chrome would start inside the virtual display.
Result: Same connection error; Chrome likely never launched.
​

• Adjusted Chrome Flags

Added Botasaurus arguments to disable sandbox/GPU:

Command: add_arguments=["--no-sandbox","--disable-gpu","--disable-dev-shm-usage"]
Expected: Avoids crashes from missing GPU/sandbox permissions.
Result: No change—Chrome still unreachable.
​

• Switched Base Images

Tried both: selenium/standalone-chrome (pre-configured) & python:3.9-slim (manual Chrome/Xvfb install)
Expected: Pre-configured images would "just work." Result: Neither resolved the connection issue.
​

• Checked Logs for Silent Failures

Scanned Railway logs for hidden errors (e.g., missing libs, permission issues).

Result: Only the Chrome URL error appears; no further clues.

• What We Suspect Now:

Railway’s environment may block Chrome’s debug port (127.0.0.1:54411).

The selenium/standalone-chrome base image might conflict with manual Xvfb.

Chrome in non-headless mode may need additional dependencies (e.g., libgl1, pulseaudio).

Continue reading...