Im trying to scrape data from the indeed website and wanted to first make a transform function that finds all the divs that have this section and return the length: My code keeps returning zero and I can't quite understand why. import requests from bs4 import BeautifulSoup def extract(page): headers = {'User_Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0'} url = f'https://www.indeed.com/jobs?q=data+analyst&start={page}' r = requests.get(url, headers) soup = BeautifulSoup(r.content, 'html.parser') return soup def transform(soup): divs = soup.find_all('div', class_ = 'cardOutline tapItem dd-privacy-allow') return len(divs) #extract everything from page one and stored in variable c = extract(0) #prints length of every instance the class_ argument shows up in the extracted data print(transform(c)) Thank you! Continue reading...
