How can an AutoGuide be written as a custom guide?

dilara · April 14, 2022, 1:20pm

I am interested in understanding how AutoGuides work. Therefore, I would like to see how they can be written as a custom guide explicitly. However, I could not find any resource which covers this topic. For example, how can one implement the custom guide that corresponds to AutoDelta in Gaussian Mixture Model — Pyro Tutorials 1.8.4 documentation?

I would appreciate it if someone could point out a resource or share a code snippet. Thanks in advance!

Gioelelm · April 14, 2022, 2:16pm

You could have a look at the source code… you need to ignore a bit of boilerplate and internals but you should be able to see the pattern.

For example, look at this loop and line implementing Autodelta:

github.com

pyro-ppl/pyro/blob/dev/pyro/infer/autoguide/guides.py#L398


      
          
          
    plates = self._create_plates(*args, **kwargs)
              result = {}
              for name, site in self.prototype_trace.iter_stochastic_nodes():
                  with ExitStack() as stack:
                      for frame in site["cond_indep_stack"]:
                          if frame.vectorized:
                              stack.enter_context(plates[frame.name])
                      attr_get = operator.attrgetter(name)
                      result[name] = pyro.sample(
                          name, dist.Delta(attr_get(self), event_dim=site["fn"].event_dim)
                      )
              return result
          
          
@torch.no_grad()
          def median(self, *args, **kwargs):
              """
              Returns the posterior median value of each latent variable.
          
          
    :return: A dict mapping sample site name to median tensor.
              :rtype: dict