[Python] Loop isn't scraping multiple pages, only returning data from one page repeatedly

Discussão em 'Python' iniciado por Stack, Fevereiro 25, 2025.

  1. Stack

    Stack Membro Participativo

    import requests
    from bs4 import BeautifulSoup
    import pandas as pd

    headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36'}

    questionlist = []

    url = "https://seekingalpha.com/market-news?page=20"

    r = requests.get(url, headers=headers)

    soup = BeautifulSoup(r.text, 'html.parser')

    questions = soup.find_all('article', {'class': 'mT-jA ga-jA Q-b8 R-cS R-df ks-IX R-cG R-dJ ks-IX R-cG R-dJ ks-I0 ks-I0 mT-NM'})

    for page in range(1, 10):
    for item in questions:
    question = {
    'title': item.find('h3', {'class': 'km-X R-cw Q-cs km-IM V-gT V-g9 V-hj km-IO V-hY V-ib V-ip km-II R-fZ'}).text,
    'link': 'https://seekingalpha.com/market-news' + item.find('a', {'class': 'hq-ox R-fu'})['href'],
    'date': item.find('span', {'class': 'mU-uO mU-gE'}),
    }
    questionlist.append(question)

    print(questionlist)


    Why is my loop not working? I am scraping for multiple pages but the output is coming for a single page multiple times.

    Continue reading...

Compartilhe esta Página