[Python] Get aggregates from different Dataframe to current Dataframe with conditions

Discussão em 'Python' iniciado por Stack, Março 15, 2025.

  1. Stack

    Stack Membro Participativo

    I have a harvest dataframe and a weather dataframe. I want to get the number of days above a temp threshold for the previous x months before harvest for all blocks. Note the harvest dataframe includes multiple years and the id is not 1-1 between frames, ie 2 blocks in harvest df can share an ID that correspond to a location in the weather frame.

    My current (working) code is below, but it is VERY slow, on the order of minutes. I want to speed it up but unclear how.

    def days_above_thresh(x, weather_df):
    return weather_df.loc[
    (weather_df["id"]==x.id) & \
    (weather_df["day"]>=x['harvest_date']-DateOffset(months=2)) & \
    (weather_df["day"]<=x['harvest_date']) & \
    (weather_df["temperature_max"]>30),
    "temperature_max"].count()

    harvest_df["days_above_30"] = harvest_df.apply(days_above_thresh , args=(weather_df,), axis=1)


    The dataframes would look something like this -

    weather_df
    id day temperature_max
    1 2020-01-01 30
    1 2020-01-02 32
    1 2020-01-03 28
    1 2020-01-04 25
    .
    .
    .
    2 2020-01-01 10
    2 2020-01-02 15
    2 2020-01-03 17
    2 2020-01-04 12
    .
    .
    .

    harvest_df
    id farm_id harvest_date
    1 87 2020-01-02
    1 86 2020-01-03
    2 13 2020-01-30

    Continue reading...

Compartilhe esta Página