10.4.2 Acoustic Models based on Posterior Classifiers

Another way to think about discriminative training is to choose a classifier at the frame level which is discriminant. Thus while the Gaussian classifier is by far the most commonly used acoustic likelihood classifier, it is possible to instead use classifiers that

are naturally discriminative or posterior estimators, such as neural networks or SVMs (support vector machines).

The posterior classifier (neural net or SVM) is generally integrated with an HMM architecture, is often called a HMM-SVM or HMM-MLP hybrid approach (Bourlard and Morgan, 1994).

The SVM or MLP approaches, like the Gaussian model, estimate the probability with respect to a cepstral feature vector at a single time $ t $. Unlike the Gaussian model, the posterior approaches often uses a larger window of acoustic information, relying on cepstral feature vectors from neighboring time periods as well. Thus the input to a typical acoustic MLP or SVM might be feature vectors for the current frame plus the four previous and four following frames, i.e. a total of 9 cepstral feature vectors instead of the single one that the Gaussian model uses. Because they have such a wide context, SVM or MLP models generally use phones rather than subphones or triphones, and compute a posterior for each phone.

The SVM or MLP classifiers are thus computing the posterior probability of a state $j$ given the observation vectors, i.e. $P(q_j | o_t)$. (also conditioned on the context, but let's ignore that for the moment). But the observation likelihood we need for the HMM, $b_j(o_t)$, is $P(o_t | q_j)$. The Bayes rule can help us see how to compute one from the other. The net is computing:

$$ p(q_{j}|o_{t})=\frac{P(o_{t}|q_{j})p(q_{j})}{p(o_{t})} $$

We can rearrange the terms as follows:

$$ \frac{p(o_{t}|q_{j})}{p(o_{t})}=\frac{P(q_{j}|o_{t})}{p(q_{j})} $$

The two terms on the right-hand side of (10.12) can be directly computed from the posterior classifier; the numerator is the output of the SVM or MLP, and the denominator is the total probability of a given state, summing over all observations (i.e., the sum over all $t$ of $\xi_j(t)$). Thus although we cannot directly compute $P(o_t|q_j)$, we can use (10.12) to compute $\frac{p(o_t|q_j)}{p(o_t)}$, which is known as a scaled likelihood (the likelihood divided by the probability of the observation). In fact, the scaled likelihood is just as good as the regular likelihood, since the probability of the observation $p(o_t)$ is a constant during recognition and doesn't hurt us to have in the equation.

The supervised training algorithms for training a SVM or MLP posterior phone classifiers require that we know the correct phone label $ q_j $ for each observation $ o_t $. We can use the same embedded training algorithm that we saw for Gaussians; we start with some initial version of our classifier and a word transcript for the training sentences. We run a forced alignment of the training data, producing a phone string, and now we retrain the classifier, and iterate.

10.5 MODELING VARIATION

As we noted at the beginning of this chapter, variation is one of the largest obstacles to successful speech recognition. We mentioned variation due to speaker differences from vocal characteristics or dialect, due to genre (such as spontaneous versus read speech), and due to the environment (such as noisy versus quiet environments). Handling this kind of variation is a major subject of modern research.