24.5.2 Interpreting Dialogue Acts

How can we do dialogue act interpretation, deciding whether a given input is a QUESTION, a STATEMENT, a SUGGEST (directive), or an ACKNOWLEDGEMENT? Perhaps we can just rely on surface syntax? We saw in Ch. 12 that yes-no-questions in English have aux-inversion (the auxiliary verb precedes the subject) statements have declarative syntax (no aux-inversion), and commands have no syntactic subject:

YES-NO-QUESTION Will breakfast be served on USAir 1557?

STATEMENT

$$ \textit{I don't}\textit{care about}\textit{lunch} $$

COMMAND

$$ \mathrm{S h o w~m e~f l i g h t s~f r o m~M i l w a u k e e~t o~O r l a n d o}. $$

Alas, as is clear from Abbott and Costello's famous Who's on First routine at the beginning of the chapter, the mapping from surface form to illocutionary act is complex. For example, the following ATIS utterance looks like a YES-NO-QUESTION meaning something like Are you capable of giving me a list of...?:

(24.21) Can you give me a list of the flights from Atlanta to Boston?

In fact, however, this person was not interested in whether the system was capable of giving a list; this utterance was a polite form of a REQUEST, meaning something more like "Please give me a list of....". Thus what looks on the surface like a QUESTION can really be a REQUEST.

Similarly, what looks on the surface like a STATEMENT can really be a QUESTION. The very common CHECK question (Carletta et al., 1997b; Labov and Fanshel, 1977),

is used to ask an interlocutor to confirm something that she has privileged knowledge about. CHECKS have declarative surface form:

Utterances that use a surface statement to ask a question, or a surface question to issue a request, are called indirect speech acts.

In order to resolve these dialogue act ambiguities we can model dialogue act interpretation as a supervised classification task, with dialogue act labels as hidden classes to be detected. We train classifiers on a corpus in which each utterance is hand-labeled for dialogue acts. The features used for dialogue act interpretation derive from the conversational context and from the act's microgrammar (Goodwin, 1996) (its characteristic lexical, grammatical, and prosodic properties):

1. Words and Collocations: Please or would you is a good cue for a REQUEST, are you for YES-NO-QUESTIONS, detected via dialogue-specific N-gram grammars.

2. Prosody: Rising pitch is a good cue for a YES-NO-QUESTION, while declarative utterances (like STATEMENTS) have final lowering: a drop in F0 at the end of the utterance. Loudness or stress can help distinguish the yeah that is an AGREEMENT from the yeah that is a BACKCHANNEL. We can extract acoustic correlates of prosodic features like F0, duration, and energy.

3. Conversational Structure: A yeah following a proposal is probably an AGREEMENT; a yeah after an INFORM is likely a BACKCHANNEL. Drawing on the idea of adjacency pairs (Schegloff, 1968; Sacks et al., 1974), we can model conversational structure as a bigram of dialogue acts

Formally our goal is to find the dialogue act $ d^{*} $ that has the highest posterior probability $ P(d|o) $ given the observation of a sentence,

$$ \begin{aligned}d^{*}&=&\underset{d}{\argmax}P(d|o)\\ &=\underset{d}{\argmax}\frac{P(d)P(o|d)}{P(o)}\\ &=\underset{d}{\argmax}P(d)P(o|d)\\ \end{aligned} $$

Making some simplifying assumptions (that the prosody of the sentence f and the word sequence W are independent, and that the prior of a dialogue act can be modeled by the conditional given the previous dialogue act) we can estimate the observation likelihood for a dialogue act d as in (24.23):

$$ P(o|d)~=~P(f|d)P(W|d) $$

$$ d^{*}=\underset{d}{\operatorname{a r g m a x}}P(d|d_{t-1})P(f|d)P(W|d) $$

where

$$ \begin{array}{c} P(W|d)~=~\prod_{i=2}^{N} P(w_{i}|w_{i-1}...w_{i-N+1},d)\end{array} $$

Training the prosodic predictor to compute $ P(f|d) $ has often been done with a decision tree. Shriberg et al. (1998), for example, built a CART tree to distinguish the four dialogue acts STATEMENT (S), YES-NO QUESTION (QY), DECLARATIVE-QUESTION like CHECK (QD) and WH-QUESTION (QW) based on acoustic features as the slope of F0 at the end of the utterance, the average energy at different places in the utterance, and various normalized duration measures. Fig. 24.19 shows the decision tree which gives the posterior probability $ P(d|f) $ of a dialogue act d type given a set of acoustic features f. Note that the difference between S and QY toward the right of the tree is based on the feature norm_f0_diff (normalized difference between mean F0 of end and penultimate regions), while the difference between QW and QD at the bottom left is based on utt_grad, which measures F0 slope across the whole utterance.

Since decision trees produce a posterior probability $ P(d|f) $, and equation (24.24) requires a likelihood $ P(f|d) $, we need to massage the output of the decision tree by Bayesian inversion (dividing by the prior $ P(d_i) $ to turn it into a likelihood); we saw this same process with the use of SVMs and MLPs instead of Gaussian classifiers in speech recognition in Sec. ??. After all our simplifying assumptions the resulting equation for choosing a dialogue act tag would be:

$$ \begin{align*}d^{*}&=\underset{d}{\operatorname{argmax}}P(d)P(f|d)P(W|d)\\&=\underset{d}{\operatorname{argmax}}P(d|d_{t-1})\frac{P(d|f)}{P(d)}\prod_{i=2}^{N}P(w_{i}|w_{i-1}...w_{i-N+1},d)\end{align*} $$