8.3.5 Computing duration from prosodic labels

The results of the text analysis processes described so far is a string of phonemes, annotated with words, with pitch accent marked on relevant words, and appropriate boundary tones marked. For the unit selection synthesis approaches that we will describe in Sec. 8.5, this is a sufficient output from the text analysis component.

For diphone synthesis, as well as other approaches like formant synthesis, we also need to specify the duration and the F0 values of each segment.

Phones vary quite a bit in duration. Some of the duration is inherent to the identity of the phone itself. Vowels, for example, are generally much longer than consonants; in the Switchboard corpus of telephone speech, the phone [aa] averages 118 milliseconds, while [d] averages 68 milliseconds. But phone duration is also affected by a wide variety of contextual factors, which can be modeled by rule-based or statistical methods.

The most well-known of the rule-based methods is the method of Klatt (1979), which uses rules to model how the average or ‘context-neutral’ duration of a phone $ \bar{d} $ is lengthened or shortened by context, while staying above a minimum duration $ d_{min} $. Each rule is associated with a duration multiplicative factor; some examples:

Prepasual Lengthening: The vowel or syllabic consonant in the syllable before a pause is lengthened by 1.4.

Non-phrase-final Shortening: Segments which are not phrase-final are shortened by 0.6.

Phrase-final postvocalic liquids and nasals are lengthened by 1.4.

Unstressed Shortening: Unstressed segments are more compressible, so their minimum duration $ d_{min} $ is halved, and are shortened by 0.7 for most phone types.

Lengthening for Accent: A vowel which bears accent is lengthened by 1.4

Shortening in Clusters: A consonant followed by a consonant is shortened by 0.5. Pre-voiceless shortening: Vowels are shortened before a voiceless plosive by 0.7

Given the set of $N$ factor weights $f$, the Klatt formula for the duration of a phone is:

$$ d=d_{\min}+\prod_{i=1}^{N}f_{i}\times(\bar{d}-d_{\min}) $$

More recent machine-learning systems use the Klatt hand-written rules as the basis for defining features, for example using features such as the following:

• identity of the left and right context phone

• lexical stress and accent values of current phone

position in syllable, word, phrase

• following pause

We can then train machine learning classifiers like decision trees or the sum-of-products model (van Santen, 1994, 1997, 1998), to combine the features to predict the final duration of the segment.