Fox, Emily and Sudderth, Erik and Jordan, Michael and Willsky, Alan
The hierarchical Dirichlet process hidden Markov model (HDP-HMM) is a flexible, nonparametric model which allows state spaces of unknown size to be learned from data. We demonstrate some limitations of the original HDP-HMM formulation, and propose a sticky extension which allows more robust learning of smoothly varying dynamics. Using DP mixtures, this formulation also allows learning of more complex, multimodal emission distributions. We further develop a sampling algorithm that employs a truncated approximation of the DP to jointly resample the full state sequence, greatly improving mixing rates. Via extensive experiments with synthetic data and the NIST speaker diarization database, we demonstrate the advantages of our sticky extension, and the utility of the HDP-HMM in real-world applications.
Discussion
An interesting paper and good results.
Somebody asked about using supervisory data to train the speaker diarization system. I wonder if we can use the idea of state-splitting (Petrov et al, COLING-ACL 2006) to do this, where the supervisory labels are treated as a state sequence? Each state is further split into multiple states and modelled with a HDP-HMM.
I liked this paper also. I didn't ask, but an alternative approach would be to look at semi-Markov processes - where there is an explicit distribution over the tine you stay in each state. That would be a little more powerful - since time-in-a-state doesn't have to be exponentially distributed - but it's not immediately obvious how one would implement an HDP-SMM.
A semi-Markov extension is interest. It makes sense as the current sticky HDP-HMM model assumes a geometric distribution over the length of times the model stays in each state, and this is not necessarily what we believe about our data. From an inference algorithm or model-building perspective I don't see any problems.
It great that it was extended to include the self transition parameter. This makes a very big difference: combined with some other neat ideas they were able to present some impressive performance on speaker diarization. I'm quite delighted with it.
sexi men
model assumes a geometric distribution over the length of times the model stays in each state, and this is not necessarily what we believe about our data. From an inference
model assumes a geometric distribution over the length of times the model stays in each state, and this is not necessarily what we believe about our data. From an inference
look at semi-Markov processes - where there is an explicit distribution over the tine you stay in each state. That would be a little more powerfu