24.2.3 Generation and TTS components

The generation component of a conversational agent chooses the concepts to express to the user, plans out how to express these concepts in words, and assigns any necessary prosody to the words. The TTS component then takes these words and their prosodic annotations and synthesizes a waveform, as described in Ch. 8.

The generation task can be separated into two tasks: what to say, and how to say it. The content planner module addresses the first task, decides what content to express to the user, whether to ask a question, present an answer, and so on. The content planning component of dialogue systems is generally merged with the dialogue manager, and we will return to it below.

The language generation module addresses the second task, choosing the syntactic structures and words needed to express the meaning. Language generation modules are implemented in one of two ways. In the simplest and most common method, all or most of the words in the sentence to be uttered to the user are prespecified by the dialogue designer. This method is known as template-based generation, and the sentences created by these templates are often called prompts. While most of the words in the template are fixed, templates can include some variables which are filled in by the generator, as in the following:

What time do you want to leave CITY-ORIG?

Will you return to CITY-ORIG from CITY-DEST?

A second method for language generation relies on techniques from the field natural language generation. Here the dialogue manager builds a representation of the meaning of the utterance to be expressed, and passes this meaning representation to a full generator. Such generators generally have three components, a sentence planner, surface realizer, and prosody assigner. A sketch of this architecture is shown in Fig. 24.8. See Reiter and Dale (2000) for further information on natural language generation systems and their use in dialogue.

In the hand-designed prompts that are common in current systems, there are a number of important conversational and discourse constraints that must be implemented.

Like any discourse, a conversation needs to be coherent (Ch. 21). For example, as Cohen et al. (2004) show, the use of discourse markers and pronouns in the hand-built system prompts makes the dialogue in (24.15) more natural than the dialogue in (24.14):

(24.14) Please say the data.

Please say the start time.

Please say the duration.

Please say the subject.

(24.15) First, tell me the date.

Next, I'll need the time it starts.

Thanks. Now, how long is it supposed to last?

Last of all, I just need a brief description...

TAPERED PROMPTS

Another important case of discourse coherence occurs when particular prompts may need to be said to the user repeatedly. In these cases, it is standard in dialogue systems to use tapered prompts, prompts which get incrementally shorter. The following example from Cohen et al. (2004) shows a series of (hand-designed) tapered prompts:

(24.16) System: Now, what's the first company to add to your watch list?

Caller: Cisco

System: What's the next company name? (Or, you can say, "Finished.")

Caller: IBM

System: Tell me the next company name, or say, “Finished.”

Caller: Intel

System: Next one?

Caller: America Online.

System: Next?

Caller: ...

Other constraints on generation are more specific to spoken dialogue, and refer to facts about human memory and attentional processes. For example, when humans are prompted to give a particular response, it taxes their memory less if the suggested response is the last thing they hear. Thus as Cohen et al. (2004) point out, the prompt "To hear the list again, say 'Repeat list'" is easier for users than "Say 'Repeat list' to hear the list again."

Similarly, presentation of long lists of query results (e.g., potential flights, or movies) can tax users. Thus most dialogue systems have content planning rules to deal with this. In the Mercury system for travel planning described in Seneff (2002), for example, a content planning rule specifies that if there are more than three flights to describe to the user, the system will just list the available airlines and describe explicitly only the earliest flight.