[Python] How to handle large gaps in missing data in time series forecasting (e.g., only 1-2...

I’m working with a time series dataset that tracks monthly expenses for several years. However, I have a major issue with missing values: in some years, I only have data for 1-2 months, while the rest of the months are completely missing.

Here’s a simple example:

Year​

Month​

Expenses​

2020​

1​

50​

2020​

2​

60​

2020​

3-12​

null​

2021​

1​

null​

2021​

2​

80​

2021​

3-12​

null​

As you can see, many months are missing, and the gaps are quite large (almost 10 months are missing per year). My goal is to predict monthly expenses for the future, but these large gaps in the data make it difficult to accurately model the time series.

Questions: How do I handle such large gaps in the time series data? What are the best practices for imputing or managing missing values when nearly an entire year is missing?

What models or techniques (e.g., ARIMA, Prophet, machine learning models) can handle such large missing gaps in time series data for predicting future monthly values? Are there specific techniques designed to deal with long stretches of missing data?

Should I use imputation techniques (like linear interpolation, Kalman filters, or other methods) to fill in the missing values before forecasting, or are there forecasting models that can natively handle large missing gaps?

Any help or guidance would be appreciated, especially for cases where the missing data is so extensive that traditional interpolation techniques may not be appropriate.

Continue reading...