[Python] Web scrapper Python Mongo DB query not working

Discussão em 'Python' iniciado por Stack, Fevereiro 14, 2025.

  1. Stack

    Stack Membro Participativo

    I am fairly new to programming and I am currently learning python.I have a web scrapper project using python and beautiful soup library below where I scrap an entire article from a news web page.I manage to scrap the article but I want to query the entire article for specific words and when I try the mongodb query find,I cannot search for any words.Please could someone let me know how to proceed.Thank you for your help.

    import pymongo
    import requests
    url ="https://www.bbc.com/news/articles/cy081nqx2zjo"
    response = requests.get(url)


    from bs4 import BeautifulSoup
    soup =BeautifulSoup(response.text,'html.parser')


    paragraphs = soup.find_all("p")

    for paragraph in paragraphs:
    print(paragraph.text)


    def check(paragraphs):
    if len(paragraphs) > 100:
    return "extracted fine"
    else:
    return "An exception occurred"
    print(check(paragraphs))

    mydict={}
    doc = {"paragraphs":mydict}
    from pymongo import MongoClient
    client= pymongo.MongoClient("mongodb://localhost:27017")
    db =client["mydb"]
    collection=db["summarizer"]
    collection.insert_one(doc)

    Continue reading...

Compartilhe esta Página