# Usage of pyro.param in model

The tutorial materials (e.g., SVI Part I: An Introduction to Stochastic Variational Inference in Pyro — Pyro Tutorials 1.8.1 documentation) show that `pyro.param` is a statement that should be included in the guide as it registers the variational parameters. But I saw in some examples that `pyro.param` is used in the model as well (e.g, scANVI: Deep Generative Modeling for Single Cell Data with Pyro — Pyro Tutorials 1.8.1 documentation). However, it is not clear to me what the use case of this statement is in the model as I could not see an explanation anywhere.

Could someone please explain when `pyro.param` should be used in the model?

Hi @dilara . When you use `pyro.param` in the model you can consider it as the maximum likelihood* estimation method and you get a point estimate for the given parameter. These are `theta` in the SVI Part I tutorial.

As far as I know `pyro.param` and `pyro.sample` statements can be mixed in the model with the difference that at inference you obtain a point estimate for the former (maximum likelihood* treatment) and posterior distribution for the latter (Bayesian treatment). For example, there is a method called Empirical Bayes where the parameters at the highest level of the model hierarchy are set to their most likely value (maximum likelihood treatment).

I personally use `pyro.param` for some parameters when I am developing a complex model (which is easier since they don’t require a prior distribution) and then later change them to `pyro.sample` if I want a full Bayesian treatment for my model.

Pyro also has an effect handler called lift that can be used to “lift” `pyro.param` statements to `pyro.sample` statements given a prior distribution for that parameter.

*Or maximum log evidence to be more precise when there are random variables in the model.

Hi @ordabayev, thank you so much for the explanation! I was not aware of the presence of lift.

I have a question about your comment here. Do you set the prior for `pyro.sample` based on the estimate you get from `pyro.param`?

Sometimes. As an example, there is a `gain` parameter in my model. It’s value can change from experiment to experiment (but always positive). Based on its `pyro.param` values from the fits in the past (and also other independent direct experimental measurements of `gain`) I know that the value of `gain` typically lies in the range between 5 and 50 so I set its prior as `HalfNormal(50.0)` for any future fits to roughly capture that range.

Thank you @ordabayev for the explanation! I also observed that using `pyro.param` instead of a prior worked better in my model.

I have a question about naming the parameters. Should the name of the parameters in the model and guide match, like they do for `pyro.sample`, or should it be avoided? For example, the names of the parameters match in guide and model in Tensor shapes in Pyro — Pyro Tutorials 1.8.1 documentation (see `fun` method).

Edit: I have just seen that you addressed this an hour ago at Pyro curve fitting parameter std estimate - #3 by ordabayev Although, I am still a bit confused why the names match in this particular tutorial

In the tutorial it says Let’s look at a contrived model to see how these work in practice about writing a parallelizable code, not necessarily a proper model/guide pair from a Bayesian inference perspective (just my guess since I didn’t write that tutorial).

Having matching `pyro.param` s in the model and guide won’t produce an error and the code will run but it is about whether it makes sense from the math/inference point of view.

Edit: There are probably cases when it is ok and makes sense to have matching `pyro.param` s in model and the guide, just can’t think of any at the moment.

1 Like