Measure perplexity of LDA


I trained the model of LDA using pyro (Probabilistic Topic Modeling — Pyro Tutorials 1.8.1 documentation)
Now I need to compute the perplexity of this model to compare this model with other LDA models.
Ideally I would like to compare LDA using Mallet (GitHub - mimno/Mallet: MALLET is a Java-based package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.)
But if someone have an other way to do it please tell me.
I think that I will need to export the trained so that I can pass it to another function that will compute the perplexity.

1 Like