[Python] How to scrape Star Rating from Etsy HTML with Python?

I can not find a way to extract the star rating from each product from the Etsy source code.

This is the code I've used to extract the description, price, number of reviews from a set of text files containing several Etsy webpage source code for several products. The only bit not working is the star rating which I cannot get to extract to my dataframe. Any ideas on how I could point it to be scraped and list them on my dataframe?

import os
import pandas as pd
import re

directory_path = r"C:\Users\..."

descriptions = []
prices = []
review_numbers = []
file_names = []
stars = []

for file_name in os.listdir(directory_path):
if file_name.endswith(".txt"):
file_path = os.path.join(directory_path, file_name)

with open(file_path, "r", encoding="utf-8") as file:
lines = file.readlines()
description = ""
price = ""
review_number = ""
star_rating = ""
for line in lines:
if line.startswith('----------------------------------------'):
if description != "":
descriptions.append(description.strip())
prices.append(price)
review_numbers.append(review_number)
file_names.append(file_name)
stars.append(star_rating)

description = ""
price = ""
review_number = ""
star_rating = ""
elif re.match(r'\$\d+(?:\.\d+)?', line):
price = line.strip()
elif re.search(r'\(\d[\d,]*\)', line):
review_number = re.search(r'\(\d[\d,]*\)', line).group()
else:
description += line.strip() + " "

star_match = re.search(r'class="wt-align-items-center wt-max-height-full wt-display-flex-xs flex-direction-row-xs wt-text-title-small wt-no-wrap">([\d.]+)', line)
if star_match:
star_rating = star_match.group(1)

if 'v2-listing-card_title' in line:
description = line.split('title="', 1)[1].split('"', 1)[0]

if description != "":
descriptions.append(description.strip())
prices.append(price)
review_numbers.append(review_number)
file_names.append(file_name)
stars.append(star_rating)

data = {'Description': descriptions, 'Price': prices, 'Review_Number': review_numbers, 'File_Name': file_names, 'Stars': stars}
df = pd.DataFrame(data)


Here is a typical example of the kind of webpage I'm trying to scrape:

https://www.etsy.com/search?q=hallo...ip_to=FR&price_bucket=1&max=20&ref=search_bar

The HTML snippet containing the stars looks like this:

<b><div class="v2-listing-card_info

<h3 class="wt-text-caption v2-listing-card_title wt-text-truncate

title="Luna moth cape dance win gs costume adult Spanish luna moth cape chiffon be lly halloween burning man" id="ad-listing-title-2 80310434"></h3>

<div class="streamline-spacing-shop-rating">

<div class="shop-name-with-rating wt-flex-wrap w t-display-flex-xs flex-direction-row-xs wt-align -items-center"> flex

<span class="

wt-display-inline-flex-xs wt-flex-wrap wt-align-items-center larger_review_stars

">Flex $0

<div role="img" aria-label="4.9 star rating with 15.4k reviews" class="wt-align-items-ce nter wt-max-height-full wt-display-flex-xs f lex-direction-row-xs wt-text-title-small wt- no-wrap"> flex

"4.9"</b>


The output looks as follows, with the column for stars completely empty:

Description Price Review_Number File_Name Stars
Applied thermostickers witch shoes in liberty ... $6.89 (801) halloween_cape_0-20_clean.txt
Cute Cat with a witch hat Sticker / Cute Kitty... $3.68 (30) halloween_cape_0-20_clean.txt
Navy Velvet Wizard Costume Skirt Tutu Cape Wan... $18.57 (3,990) halloween_cape_0-20_clean.txt
Dog Cape - "QUEEN ELIZABETH" ... $17.26 (25) halloween_cape_0-20_clean.txt
Vampire cat halloween artwork, Halloween cat p... $25.69 (427) halloween_cape_0-20_clean.txt
... ... ... ... ...
Jack Skellington XXL Bow Tie Necklace - Eyes T... $57.52 (104) halloween_tie_50-100_clean.txt
Vintage Halloween Season of the Witch Novel by... $79.57 (1,650) halloween_tie_50-100_clean.txt
Halloween Tie Dye Sweatshirt // Orange, Purple... $50.00 (755) halloween_tie_50-100_clean.txt
Loading... halloween_tie_50-100_clean.txt

Continue reading...