Hierarchical Forecasting with varying numbers of subcategories

Tutorials
1
  • What tutorial are you running? - Forecasting III: hierarchical models
  • What version of Pyro are you using? - 1.7.0.

Hi,

I’m relatively new to Pyro and trying to change the tutorial on forecasting with hierarchical models to my needs. I want to forecast sales of products which all belong to subcategories which in turn belong to categories.

The way I understand the tutorial, the data I pass needs to have one dimension for each hierarchy level. In the second model of the tutorial that’s the origin station, the destination station and the time, that’s why zero_data looks like this:

    class Model2(ForecastingModel):
    def model(self, zero_data, covariates):
        num_stations, num_stations, duration, one = zero_data.shape

And then I tell the model which hierarchy level corresponds to which plate statement by passing the dimension, e.g.

origin_plate = pyro.plate("origin", num_stations, dim=-3)

The problem I see here is that in my data

  1. not every category has the same number of subcategories
  2. not every subcategory contains the same number of articles
  3. not every article sales time series is of the same length.

I then found this topic where the solution seems to be to just artificially add zeros to the tensor where there are no real observations and then use poutine.mask to tell the model to ignore them.

So in my case, should zero_data have the shape (number of categories) x (max number of subcategories in one category) x (max number of articles in one subcategory) x (max duration of article time series) x 1 ? And then how do I pass the right parameters to poutine.mask? I’m guessing I need a vector telling the number of subcategories for each category, one vector telling the number of articles for each subcategory and one vector telling the duration of each article time series.

I’m also a bit wary because my dataset is quite big as it is (6,000 articles, most of them daily data for more than 10 years) and artificially enlarging it doesn’t seem like a great idea at first glance.

I’d appreciate if someone could tell me if I’m on the right track or maybe give me a hint for further reading or tutorials. Thanks!