[Python] Selenium cannot find class for car brands and prices and post it on a local html

I received kind of a homework from an internship and it sounds something like this : By Monday, please make me a program in Python that will run in a Docker container and that, at start-up, will serve a web page where the following happens:

• to take the main brands of cars from autovit.ro
• to allow the selection of a brand and, after selection, to display the minimum and maximum prices of the cars of the respective brand on Autovit. You are allowed to get help from chatgpt.

So I am not a very skilled coder but I tried my best with the help of chatgpt. Below is the code in Python for the scraper :

from fastapi import FastAPI, Form
from fastapi.responses import HTMLResponse
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup
import time

app = FastAPI()

# Inițializează driver-ul Chrome cu Selenium
def get_selenium_driver():
options = webdriver.ChromeOptions()
options.headless = True # Rulează în mod headless (fără interfață grafică)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
return driver

# Funcție pentru a obține brandurile de pe autovit.ro
def get_car_brands():
url = "https://www.autovit.ro/"
driver = get_selenium_driver()
driver.get(url)

# Așteaptă să se încarce elementele (poți ajusta timpul după caz)
time.sleep(5)

# Obține sursa paginii
soup = BeautifulSoup(driver.page_source, 'html.parser')

# Încearcă să găsești brandurile folosind selectorul corect (actualizează-l cu unul corect)
brands = []
try:
for brand in soup.find_all('a', class_="e4zbkti0 ooa-17iu0re"): # Actualizează cu selectorul corect
brands.append(brand.text.strip())
except Exception as e:
print(f"Error while extracting brands: {e}")

driver.quit()
return brands

# Funcție pentru a obține prețurile minime și maxime ale unui brand
def get_price_range(brand):
url = f"https://www.autovit.ro/{brand}"
driver = get_selenium_driver()
driver.get(url)

# Așteaptă să se încarce elementele (ajustează timpul după caz)
time.sleep(5)

soup = BeautifulSoup(driver.page_source, 'html.parser')

# Găsește prețurile folosind selectorul corect
prices = []
try:
for price in soup.find_all('span', class_="ooa-ejqwvc"): # Actualizează cu selectorul corect
try:
price_value = int(price.text.replace("€", "").replace(",", "").strip())
prices.append(price_value)
except ValueError:
pass # În cazul în care prețul nu poate fi convertit într-un număr
except Exception as e:
print(f"Error while extracting prices: {e}")

driver.quit()
return min(prices), max(prices) if prices else (0, 0) # Asigură-te că returnezi o valoare validă

@app.get("/", response_class=HTMLResponse)
def read_root():
brands = get_car_brands()
brand_options = "".join([f"<option value='{brand}'>{brand}</option>" for brand in brands])
html_content = f"""
<html>
<body>
<h2>Select a car brand</h2>
<form action="/price_range" method="get">
<select name="brand">
{brand_options}
</select>
<input type="submit" value="Get Price Range">
</form>
</body>
</html>
"""
return html_content

@app.get("/price_range", response_class=HTMLResponse)
def price_range(brand: str = Form(...)):
min_price, max_price = get_price_range(brand)
html_content = f"""
<html>
<body>
<h2>Price Range for {brand}</h2>
<p>Min Price: €{min_price}</p>
<p>Max Price: €{max_price}</p>
</body>
</html>
"""
return html_content



The problem is the container in Docker works, I can open the browser and search for localhost:8000 and the html opens but I cannot find any car brand or price. I asked chatgpt and I also concluded that the classes are wrong but I cannot find anything better. I tried with XPATH as well but with no succes. Do you have any ideas ?

Thanks !

Continue reading...