[Python] DFM Statsmodels contemporaneus effects

Discussão em 'Python' iniciado por Stack, Julho 14, 2025.

  1. Stack

    Stack Membro Participativo

    I'm working with a DynamicFactor model in statsmodels that includes two monthly variables (x1, x2) and one quarterly variable (y). The quarterly variable is only observed every three months (e.g., March, June, September, December), and the rest of the time it's set to NaN.

    I estimate the model using data up to November, then update it by adding December's values for the monthly variables (keeping y as NaN for December). However, the nowcast for y in December does not change after this update, even though the new monthly data should, in theory, influence the latent factor and thus the forecast.

    Why doesn't the December monthly data contemporaneously affect the nowcast of the quarterly variable? Is this a limitation of how statsmodels handles mixed-frequency data, or am I missing something in how the model updates predictions?

    I'm attaching the full code below so you can reproduce the issue and verify the behavior.

    Any insights or suggestions would be greatly appreciated!

    import numpy as np
    import pandas as pd
    import matplotlib.pyplot as plt
    from statsmodels.tsa.statespace.dynamic_factor import DynamicFactor

    # ------------------------------------------
    # 1. Simulate data
    # ------------------------------------------

    np.random.seed(42)
    monthly_dates = pd.date_range(start="2020-01-01", periods=48, freq="M") # 4 years of data
    quarterly_dates = monthly_dates[::3]

    factor = np.cumsum(np.random.normal(0, 0.5, len(monthly_dates)))
    x1 = factor + np.random.normal(0, 0.3, len(monthly_dates))
    x2 = 0.8 * factor + np.random.normal(0, 0.3, len(monthly_dates))
    y = pd.Series(0.6 * factor[::3] + np.random.normal(0, 0.2, len(quarterly_dates)), index=quarterly_dates)

    # Full dataset with NaNs in y where there is no quarterly observation
    df = pd.DataFrame(index=monthly_dates)
    df["x1"] = x1
    df["x2"] = x2
    df["y"] = np.nan
    df.loc[y.index, "y"] = y

    # ------------------------------------------
    # 2. Estimate model with data up to November
    # ------------------------------------------

    df_train = df.iloc[:-1].copy() # Up to November
    df_new_month = df.iloc[[-1]].copy() # Only December (with y = nan)

    mod = DynamicFactor(df_train, k_factors=1, factor_order=1, error_order=1)
    res = mod.fit(disp=False)

    # Nowcast for December before incorporating December
    pred_old = res.get_prediction(start='2023', end='2024')
    nowcast_before = pred_old.predicted_mean["y"]

    # ------------------------------------------
    # 3. Update only with December (x1 and x2), without re-estimating
    # ------------------------------------------

    # Create new observation with December only in x1 and x2 (y remains NaN)
    updated_data = pd.concat([df_train, df_new_month])

    # Apply the already estimated model to the new data
    res_updated = res.apply(updated_data)

    # Get new nowcast for December
    pred_new = res_updated.get_prediction(start='2023', end='2024')
    nowcast_after = pred_new.predicted_mean["y"]

    # ------------------------------------------
    # 4. Show results
    # ------------------------------------------
    print("Nowcast before adding December data:", nowcast_before['2023-12-31'])
    print("Nowcast after adding December data:", nowcast_after['2023-12-31'])

    Continue reading...

Compartilhe esta Página