I’m reading the Gaussian mixture model tutorial, and with this model we have a non-identifiability problem because the group labels and their corresponding probability can be interchanged and we’d still have the same output.
One of the solutions to this problem is to enforce an ordering of the parameters. In the mixture example, we can enforce one group mean to be always smaller that the other. So my question is two fold:
- How to enforce an ordering in Pyro, when using
pyro.sampleto generate samples, how to make sure the samples for one parameter is always bigger than another? Of course if we have more than two groups we’d have to enforce an ordering on more variables. The
pyro.distributions.constraintsseems to only work on
- What can we do when the data is multidimensional? If the data is one-dimensional their means have a natural ordering, but if the data is multidimensional there doesn’t seem to exist such a natural ordering.