[Python] Web Scraping WeatherUnderground

Discussão em 'Python' iniciado por Stack, Março 6, 2025.

  1. Stack

    Stack Membro Participativo

    I'm doing the project that reads the data every hour from WeatherUnderground and it saves the data into a txt file

    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from datetime import datetime

    def scrape_weather_data(url, time_selector, temperature_selector,
    humidity_selector, precip_selector, wind_speed_selector,
    pressure_selector, amount_selector):
    driver = webdriver.Chrome()

    driver.get(url) # Enter The website

    # Generate text file name based on day and hour
    current_time = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
    filename = f'dane_pogodowe_{current_time}.txt'

    # Saving data into a .txt file
    with open(filename, 'a') as file:
    time_elements = driver.find_elements(By.CSS_SELECTOR,time_selector)
    temperature_elements = driver.find_elements(By.CSS_SELECTOR,temperature_selector)
    humidity_elements = driver.find_elements(By.CSS_SELECTOR,humidity_selector)
    precip_elements = driver.find_elements(By.CSS_SELECTOR,precip_selector)
    wind_elements = driver.find_elements(By.CSS_SELECTOR, wind_speed_selector)
    pressure_elements = driver.find_elements(By.CSS_SELECTOR, pressure_selector)
    amount_elements = driver.find_elements(By.CSS_SELECTOR, amount_selector)

    for i in range(len(time_elements)):
    time = time_elements.text.strip()
    temperature = temperature_elements.text.strip()
    humidity = humidity_elements.text.strip()
    precip = precip_elements.text.strip()
    wind = wind_elements.text.strip()
    pressure = pressure_elements.text.strip()
    amount = amount_elements.text.strip()

    file.write(f"Czas: {time}\n")
    file.write(f"Temperatura: {temperature}\n")
    file.write(f"Wilgotność: {humidity}\n")
    file.write(f"Szansa opadów: {precip}\n")
    file.write(f"Wiatr: {wind}\n")
    file.write(f"Ciśnienie: {pressure}\n")
    file.write(f"Ilość opadów: {amount}\n")
    file.write("\n")
    file.write("------------------------------\n")

    driver.quit() # Close the Web browser

    print("Dane pogodowe zostały zapisane.")

    # URL
    url = 'https://www.wunderground.com/hourly/pl/kucoby'

    # Selectors CSS
    time_selector = '#hourly-forecast-table > tbody > tr:nth-child(1)
    > td.mat-cell.cdk-cell.cdk-column-timeHour.mat-column-
    timeHour.ng-star-inserted > span'
    temperature_selector = '#hourly-forecast-table > tbody > tr:nth-
    child(1) > td.mat-cell.cdk-cell.cdk-column-temperature.mat-
    column-temperature.ng-star-inserted'
    humidity_selector = '#hourly-forecast-table > tbody > tr:nth-
    child(1) > td.mat-cell.cdk-cell.cdk-column-humidity.mat-column-
    humidity.ng-star-inserted'
    precip_selector = '#hourly-forecast-table > tbody > tr:nth-
    child(1) > td.mat-cell.cdk-cell.cdk-column-precipitation.mat-
    column-precipitation.ng-star-inserted'
    wind_speed_selector = '#hourly-forecast-table > tbody > tr:nth-
    child(1) > td.mat-cell.cdk-cell.cdk-column-wind.mat-column-
    wind.ng-star-inserted'
    pressure_selector = '#hourly-forecast-table > tbody > tr:nth-
    child(1) > td.mat-cell.cdk-cell.cdk-column-pressure.mat-column-
    pressure.ng-star-inserted'
    amount_selector = '#hourly-forecast-table > tbody > tr:nth-
    child(1) > td.mat-cell.cdk-cell.cdk-column-
    liquidPrecipitation.mat-column-liquidPrecipitation.ng-star-
    inserted'


    # Calling a function and saving the Data
    scrape_weather_data(url, time_selector, temperature_selector,
    humidity_selector, precip_selector, wind_speed_selector,
    pressure_selector, amount_selector)


    And the data reading is based on CSS_Selectors as you can tell, but the problem is that when it enters the website, the default setting is that it opens in the imperial system, not the metrical that i use. And i can't really convert it mathematically into metric system, due to the fact that the variables are saved as strings. There is a option on this website to convert data from imperial into metrical system, but to be fair i have no idea, how to implement it.

    EDIT:

    I found out the key that is responsible for setting the metric system:

    Continue reading...

Compartilhe esta Página