24.2.5 Dialogue Manager Error Handling: Confirmation/Rejection

In a dialogue system, mishearings are a particularly important class of problems, because speech recognition has such a high error rate. It is therefore important for dialogue systems to make sure that they have achieved the correct interpretation of the user's input. This is generally done by two methods: confirming understandings with the user, and rejecting utterances that the system is likely to have misunderstood.

Various strategies can be employed for confirmation with the user. A system using the explicit confirmation strategy asks the user a direct question to confirm their understanding. Here are two examples of explicit confirmations from travel planning systems. The (boldface) confirmation questions are both yes-no questions, one using a

原书第 956 页

single sentence, the other presenting a declarative sentence followed by a tag question (a short question phrase like "right?" or "isn't it?"):

S: Which city do you want to leave from?

U: Baltimore.

S: Do you want to leave from Baltimore?

U: Yes.

U: I'd like to fly from Denver Colorado to New York City on September twenty first in the morning on United Airlines

S: Let's see then. I have you going from Denver Colorado to New York on September twenty first. Is that correct?

U: Yes

A system using the implicit confirmation strategy, rather than asking a direct question, uses the demonstration or display grounding strategies described above, confirming to the user by repeating back what the system understood the user to have said:

U: I want to travel to Berlin

S: When do you want to travel to Berlin?

U2: Hi I'd like to fly to Seattle Tuesday Morning

A3: Traveling to Seattle on Tuesday, August eleventh in the morning. Your full name?

Explicit and implicit confirmation have complementary strengths. Explicit confirmation makes it easier for users to correct the system's misrecognitions since the user can just answer 'no' to the confirmation question. But explicit confirmation is awkward and increases the length of the conversation (Danieli and Gerbino, 1995; Walker et al., 1998). The explicit confirmation dialogue fragments above sound non-natural and definitely non-human; implicit confirmation is much more conversationally natural.

Confirmation is just one kind of conversational action that a system has to express lack of understanding. Another option is rejection. An ASR system rejects an utterance by giving the user a prompt like I'm sorry, I didn't understand that.

Sometimes utterances are rejected multiple times. This might mean that the user is using language that the system is unable to follow. Thus when an utterance is rejected, systems often follow a strategy of progressive prompting or escalating detail (Yankelovich et al., 1995; Weinschenk and Barker, 2000) as in this example from Cohen et al. (2004):

System: When would you like to leave?

Caller: Well, um, I need to be in New York in time for the first World Series game.

System: . Sorry, I didn't get that. Please say the month and day you'd like to leave.

Caller: I wanna go on October fifteenth.

In this example, instead of just repeating 'When would you like to leave?', the rejection prompt gives the caller more guidance about how to formulate an utterance the system will understand. These you-can-say help messages are important in helping improve systems understanding performance (Bohus and Rudnicky, 2005). If the caller's

原书第 957 页

utterance gets rejected yet again, the prompt can reflect this ('I still didn't get that'), and give the caller even more guidance.

An alternative strategy for error handling is rapid reprompting, in which the system rejects an utterance just by saying “I’m sorry?” or “What was that?”. Only if the caller’s utterance is rejected a second time does the system start applying progressive prompting. Cohen et al. (2004) summarizes experiments showing that users greatly prefer rapid reprompting as a first-level error prompt.

24.3 VOICEXML

VOICEXML VoiceXML is the Voice Extensible Markup Language, an XML-based dialogue design language released by the W3C, and the most commonly used of the various speech markup languages (such as SALT). The goal of VoiceXML (or vxml) is to create simple audio dialogues of the type we have been describing, making use of ASR and TTS, and dealing with very simple mixed-initiative in a frame-based architecture. While VoiceXML is more common in the commercial rather than academic setting, it is a good way for the student to get a hands-on grasp of dialogue system design issues.

Figure 24.11 A minimal VoiceXML script for a form with a single field. User is prompted, and the response is then repeated back.

A VoiceXML document contains a set of dialogues, each of which can be a form or a menu. We will limit ourselves to introducing forms; see http://www.voicexml.org/ for more information on VoiceXML in general. The VoiceXML document in Fig. 24.11 defines a form with a single field named 'transporttype'. The field has an attached prompt, 'Please choose airline, hotel, or rental car', which can be passed to the TTS system. It also has a grammar (language model) which is passed to the speech recognition engine to specify which words the recognizer is allowed to recognize. In the example in Fig. 24.11, the grammar consists of a disjunction of the three words: airline, hotel, and rental car.

A

generally consists of a sequence of s, together with a few other commands. Each field has a name (the name of the field in Fig. 24.11 is transporttype) which is also the name of the variable where the user's response will be stored. The

原书第 958 页

prompt associated with the field is specified via the command. The grammar associated with the field is specified via the command. VoiceXML supports various ways of specifying a grammar, including XML Speech Grammar, ABNF, and commercial standards, like Nuance GSL. We will be using the Nuance GSL format in the following examples.

The VoiceXML interpreter walks through a form in document order, repeatedly selecting each item in the form. If there are multiple fields, the interpreter will visit each one in order. The interpretation order can be changed in various ways, as we will see later. The example in Fig. 24.12 shows a form with three fields, for specifying the origin, destination, and flight date of an airline flight.

The prologue of the example shows two global defaults for error handling. If the user doesn't answer after a prompt (i.e., silence exceeds a timeout threshold), the VoiceXML interpreter will play the prompt. If the user says something, but it doesn't match the grammar for that field, the VoiceXML interpreter will play the prompt. After any failure of this type, it is normal to re-ask the user the question that failed to get a response. Since these routines can be called from any field, and hence the exact prompt will be different every time, VoiceXML provides a command, which will repeat the prompt for whatever field caused the error.

The three fields of this form show another feature of VoiceXML, the tag. The tag for a field is executed by the interpreter as soon as the field has been filled by the user. Here, this feature is used to give the user a confirmation of their input.

The last field, departdate, shows another feature of VoiceXML, the type attribute. VoiceXML 2.0 specifies seven built-in grammar types, boolean, currency, date, digits, number, phone, and time. Since the type of this field is date, a data-specific language model (grammar) will be automatically passed to the speech recognizer, so we don't need to specify the grammar here explicitly.

Fig. 24.13 gives a final example which shows mixed initiative. In a mixed initiative dialogue, users can choose not to answer the question that was asked by the system. For example, they might answer a different question, or use a long sentence to fill in multiple slots at once. This means that the VoiceXML interpreter can no longer just evaluate each field of the form in order; it needs to skip fields whose values are set. This is done by a guard condition, a test that keeps a field from being visited. The default guard condition for a field tests to see if the field's form item variable has a value, and if so the field is not interpreted.

Fig. 24.13 also shows a much more complex use of a grammar. This grammar is a CFG grammar with two rewrite rules, named Flight and City. The Nuance GSL grammar formalism uses parentheses () to mean concatenation and square brackets [] to mean disjunction. Thus a rule like (24.19) means that Wantsentence can be expanded as i want to fly or i want to go, and Airports can be expanded as san francisco or denver.

(24.19) Wantsentence (i want to [fly go]) Airports [(san francisco) denver]

Grammar rules can refer to other grammar rules recursively, and so in the grammar

原书第 959 页

I'm sorry, I didn't hear you.

I'm sorry, I didn't understand that.

Welcome to the air travel consultant.

Which city do you want to leave from?

[(san francisco) denver (new york) barcelona]

OK, from

And which city do you want to go to?

[(san francisco) denver (new york) barcelona]

OK, to

And what date do you want to leave?

OK, on

OK, I have you are departing from

to on

send the info to book a flight...

Figure 24.12 A VoiceXML script for a form with 3 fields, which confirms each field

Figure 24.12 A VoiceXML script for a form with 3 fields, which confirms each field and handles the noinput and nomatch situations.

in Fig. 24.13 we see the grammar for Flight referring to the rule for City.

VoiceXML grammars take the form of CFG grammars with optional semantic attachments. The semantic attachments are generally either a text string (such as "denver, colorado") or a slot and a filler. We can see an example of the former in the semantic attachments for the City rule (the return statements at the end of each line), which pass up the city and state name. The semantic attachments for the Flight rule show the latter case, where the slot ( or or both) is filled with the value passed up in the variable x from the City rule.

Because Fig. 24.13 is a mixed initiative grammar, the grammar has to be applicable to any of the fields. This is done by making the expansion for Flight a disjunction; note that it allows the user to specify only the origin city, only the destination city, or both.

原书第 960 页

I'm sorry, I didn't hear you.

I'm sorry, I didn't understand that.

Figure 24.13 A mixed initiative VoiceXML dialogue. The grammar allows sentences which specify the origin or destination cities or both. User can respond to the initial prompt

Figure 24.13 A mixed initiative VoiceXML dialogue. The grammar allows sentences which specify the origin or destination cities or both. User can respond to the initial prompt by specifying origin city, destination city, or both.

24.2.5 Dialogue Manager Error Handling: Confirmation/Rejection

24.3 VOICEXML

24.4 DIALOGUE SYSTEM DESIGN AND EVALUATION