[Python] Python Selenium script not retrieving product price from a webpage

I'm trying to scrape product prices from the website Ultra Liquors using Python and Selenium, but I'm unable to retrieve the price despite the HTML containing the expected elements. My goal is to compare prices from several shops to find the best deals or any ongoing specials for our venue.

Here's the code I'm using:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By

# Set up Chrome options
options = Options()
options.add_argument('--headless') # Run in headless mode
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

# Initialize Chrome driver
service = Service('path_to_chromedriver') # Replace with your path to chromedriver
driver = webdriver.Chrome(service=service, options=options)

# Open the product page
driver.get('https://george.ultraliquors.co.za/olof-bergh-olof-bergh-brandy-750ml')

try:
# Attempt to retrieve product name
product_name = driver.find_element(By.XPATH, '//h1[@class="product-title"]').text
print(f"Product Name: {product_name}")
except Exception as e:
print(f"Could not locate product name: {e}")

try:
# Attempt to retrieve price
price_element = driver.find_element(By.CLASS_NAME, 'price-value-10677')
price = price_element.text
print(f"Price: {price}")
except Exception as e:
print(f"Could not locate price: {e}")

# Close the driver
driver.quit()


I expect to get the price value 'R169.99', but the script is not finding it and returns an error message. I've tried using different element locators and checking if the element is dynamically loaded.

I'm using Python 3.12, Selenium 4.8, and ChromeDriver. Any help would be greatly appreciated!

Continue reading...