Simple exponential smoothing not working

Hi,

I’m trying to implement a simple exponential smoothing model (https://en.wikipedia.org/wiki/Exponential_smoothing). For those who aren’t familiar, this is a recursive model for sequences like time series that says yhat_t+1 = alpha * (x_t) + (1 - alpha) * (yhat_{t-1}), where alpha is a parameter and x the data.

Here is the code I’m using:

class Holt_Winters:
def __init__(self, freq, seasonal_period):
    self.seasonal_period = seasonal_period
    self.freq = freq


def fit(self, X, y):
    pyro.clear_param_store()
    nuts_kernel = NUTS(self.model)
    posterior = MCMC(nuts_kernel, num_samples=4000, warmup_steps=1000).run(X, y)

    post_summary = summary(posterior, sites=['yhat_initial', 'alpha'])
    return post_summary

def predict(self, X):
    pass

def model(self, X, y):
    alpha = pyro.sample('alpha', Normal(0, 1))
    yhat_prev = pyro.sample('yhat_initial', Normal(0, 1))
    start = X.index.min() + pd.Timedelta(1, self.freq)
    for d_1 in y[start:].index:
        d = d_1 - pd.Timedelta(1, self.freq)
        y_t = torch.tensor(X[d])
        yhat_t_1 = alpha * y_t + (1 - alpha) * yhat_prev
        y_t_1 = torch.tensor(y.loc[d_1])
        sigma = pyro.sample(f'sigma_{d}', Normal(0, 1))
        pyro.sample(f"yhat_{d_1} | {d}", Normal(yhat_t_1, sigma), obs=y_t_1)

        yhat_prev = yhat_t_1

    return alpha, yhat_prev

indent preformatted text by 4 spaces

For some reason, it is not fitting reasonable values on dummy data.

Dummy data:

dummy_df = pd.DataFrame(
    {'x': np.arange(1, 5),
     'y': np.arange(2, 6)
    },
    index=pd.date_range(start='1/1/2018', end='1/4/2018')
)
r = m.fit(dummy_df.x, dummy_df.y)

Can anyone spot an error in my code?

For reference, the most recent run I made, it fit a value of 2 for the initial and -1 for the alpha parameter, when clearly alpha should be 1 (always guess the most recent).

Hi @samghelms, your model looks good to me but there seems have some problems at priors. I think that it is better to use Uniform(0, 1) as a prior for alpha because we know that alpha should belong to (0, 1). In addition, because sigma is scale parameter of your Normal likelihood, it is better to set a prior with positive support for sigma, e.g. HalfNormal(1). Given that your data is small, I think that if you adjust the above suggestion, your inference result would be fine. Please let me know if it doesn’t work out for you.

Btw, if you are interested in time series model with exponential smoothing, you can take a look at time series forecasting tutorial in NumPyro. I think that the tutorial will provide you many helpful information regarding this kind of model. :slight_smile: