[Python] How to split a DataFrame on missing blocks of data?

Discussão em 'Python' iniciado por Stack, Junho 15, 2025.

  1. Stack

    Stack Membro Participativo

    I have a Dataframe with a time series index with several thousand rows. Unfortunately some datapoints are missing it looks like this:

    A
    2023-03-18 05:00:00 3.0
    2023-03-18 06:00:00 4.0
    2023-03-18 07:00:00 24.4
    2023-03-18 12:00:00 5.6
    2023-03-18 13:00:00 3.4
    2023-03-18 15:00:00 4.5
    2023-03-18 20:00:00 8.8
    2023-03-18 21:00:00 3.2


    I want my data to be hourly and thus interpolate the missing data points. This can be done with .reindex() and .interpolate() like this:

    start, end = df.index.min(), df.index.max()
    new_index = pd.DatetimeIndex(pd.date_range(start, end, freq="1h"))
    df = df.reindex(new_index)

    df = df.interpolate(method="time")
    df


    which gives:

    A
    2023-03-18 05:00:00 3.00
    2023-03-18 06:00:00 4.00
    2023-03-18 07:00:00 24.40
    2023-03-18 08:00:00 20.64
    2023-03-18 09:00:00 16.88
    2023-03-18 10:00:00 13.12
    2023-03-18 11:00:00 9.36
    2023-03-18 12:00:00 5.60
    2023-03-18 13:00:00 3.40
    2023-03-18 14:00:00 3.95
    2023-03-18 15:00:00 4.50
    2023-03-18 16:00:00 5.36
    2023-03-18 17:00:00 6.22
    2023-03-18 18:00:00 7.08
    2023-03-18 19:00:00 7.94
    2023-03-18 20:00:00 8.80
    2023-03-18 21:00:00 3.20


    QUESTION
    However, I don't want to interpolate if the gaps are too large e.g. if they are longer than 3 hours. At these large gaps I want to split the DataFrame by rows i.e. so that I have multiple DataFrames in the end (, as a list or whatever). How can I do this?

    The result should look something like this - 3 DataFrames:

    A
    2023-03-18 05:00:00 3.00
    2023-03-18 06:00:00 4.00
    2023-03-18 07:00:00 24.40


    A
    2023-03-18 12:00:00 5.60
    2023-03-18 13:00:00 3.40
    2023-03-18 14:00:00 3.95
    2023-03-18 15:00:00 4.50


    A
    2023-03-18 20:00:00 8.80
    2023-03-18 21:00:00 3.20

    Continue reading...

Compartilhe esta Página