[Python] Python Selenium FireFox (Geckodriver) - Script runs on Windows but fails on Linux...

I have written a Python script for web scraping using Selenium with Firefox (Geckodriver).

The script runs perfectly on Windows, but when I run it on Linux — either as a Python script or packaged as an executable — it fails with the following error:

urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=58083): Read timed out. (read timeout=120)


Full traceback attached (see below).

• 
  I am already using the latest geckodriver.
  
  
• 
  I have increased the timeout settings.
  
  
• 
  I run my own server.
  
  
• 
  I have set Firefox to run in headless mode on both Windows and Linux.
  

Despite all this, the issue only happens on Linux.

Does anyone know why this could happen and how I can fix it?

Any help would be really appreciated!

Error message:

Exception in thread Thread-1 (scrap_amazon):
Traceback (most recent call last):
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
response = conn.getresponse()
^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
httplib_response = super().getresponse()
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse
response.begin()
File "/usr/lib/python3.12/http/client.py", line 331, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 292, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/socket.py", line 707, in readinto
return self._sock.recv_into(b)
^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/var/nova/sourcecode/Python-Scrap-Tool/scraper_amazon/_init_.py", line 239, in scrap_amazon
driver = create_driver()
^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/scraper_amazon/_init_.py", line 210, in create_driver
driver = webdriver.Firefox(service=service, options=options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/selenium/webdriver/firefox/webdriver.py", line 71, in _init_
super()._init_(command_executor=executor, options=options)
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 250, in _init_
self.start_session(capabilities)
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 342, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 427, in execute
response = self.command_executor.execute(driver_command, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/selenium/webdriver/remote/remote_connection.py", line 404, in execute
return self._request(command_info[0], url, body=data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/selenium/webdriver/remote/remote_connection.py", line 428, in _request
response = self._conn.request(method, url, body=body, headers=headers, timeout=self._client_config.timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/_request_methods.py", line 143, in request
return self.request_encode_body(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/_request_methods.py", line 278, in request_encode_body
return self.urlopen(method, url, **extra_kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/poolmanager.py", line 443, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
raise reraise(type(error), error, _stacktrace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
raise value
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/var/nova/sourcecode/Python-Scrap-Tool/venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=58083): Read timed out. (read timeout=120)

Continue reading...