8.3.2 Prosodic prominence

In any spoken utterance, some words sound more prominent than others. Prominent words are perceptually more salient to the listener; speakers make a word more salient.

in English by saying it louder, saying it slower (so it has a longer duration), or by varying F0 during the word, making it higher or more variable.

We generally capture the core notion of prominence by associating a linguistic marker with prominent words, a marker called pitch accent. Words which are prominent are said to bear (be associated with) a pitch accent. Pitch accent is thus part of the phonological description of a word in context in a spoken utterance.

Pitch accent is related to stress, which we discussed in Ch. 7. The stressed syllable of a word is where pitch accent is realized. In other words, if a speaker decides to highlight a word by giving it a pitch accent, the accent will appear on the stressed syllable of the word.

The following example shows accented words in capital letters, with the stressed syllable bearing the accent (the louder, longer, syllable) in boldface:

I'm a little SURPRISED to hear it CHARACTERIZED as UPBEAT

Note that the function words tend not to bear pitch accent, while most of the content words are accented. This is a special case of the more general fact that very informative words (content words, and especially those that are new or unexpected) tend to bear accent (Ladd, 1996; Bolinger, 1972).

We’ve talked so far as if we only need to make a binary distinction between accented and unaccented words. In fact we generally need to make more fine-grained distinctions. For example the last accent in a phrase generally is perceived as being more prominent than the other accents. This prominent last accent is called the nuclear accent. Emphatic accents like nuclear accent are generally used for semantic purposes, for example to indicate that a word is the semantic focus of the sentence (see Ch. 21) or that a word is contrastive or otherwise important in some way. Such emphatic words are the kind that are often written IN CAPITAL LETTERS or with STARS around them in SMS or email or Alice in Wonderland; here’s an example from the latter:

'I know SOMETHING interesting is sure to happen,' she said to herself

Another way that accent can be more complex than just binary is that some words can be less prominent than usual. We introduced in Ch. 7 the idea that function words are often phonetically very reduced.

A final complication is that accents can differ according to the tune associated with them; for example accents with particularly high pitch have different functions than those with particularly low pitch; we'll see how this is modeled in the ToBI model in Sec. 8.3.4.

Ignoring tune for the moment, we can summarize by saying that speech synthesis systems can use as many as four levels of prominence: emphatic accent, pitch accent, unaccented, and reduced. In practice, however, many implemented systems make do with a subset of only two or three of these levels.

Let's see how a 2-level system would work. With two-levels, pitch accent prediction is a binary classification task, where we are given a word and we have to decide whether it is accented or not.

Since content words are very often accented, and function words are very rarely accented, the simplest accent prediction system is just to accent all content words and no function words. In most cases better models are necessary.

In principle accent prediction requires sophisticated semantic knowledge, for example to understand if a word is new or old in the discourse, whether it is being used contrastively, and how much new information a word contains. Early models made use of sophisticated linguistic models of all of this information (Hirschberg, 1993). But Hirschberg and others showed better prediction by using simple, robust features that correlate with these sophisticated semantics.

For example, the fact that new or unpredictable information tends to be accented can be modeled by using robust features like $ N $-grams or TF*IDF (Pan and Hirschberg, 2000; Pan and McKeown, 1999). The unigram probability of a word $ P(w_i) $ and its bigram probability $ P(w_i|w_{i-1}) $, both correlate with accent; the more probable a word, the less likely it is to be accented. Similarly, an information-retrieval measure known as TF*IDF (Term-Frequency/Inverse-Document Frequency; see Ch. 23) is a useful accent predictor. TF*IDF captures the semantic importance of a word in a particular document d, by downgrading words that tend to appear in lots of different documents in some large background corpus with N documents. There are various versions of TF*IDF; one version can be expressed formally as follows, assuming $ N_w $ is the frequency of w in the document d, and k is the total number of documents in the corpus that contain w:

$$ \mathrm{TF^{*}IDF}(w)=N w\times\log(\frac{N}{k}) $$

For words which have been seen enough times in a training set, the accent ratio feature can be used, which models a word's individual probability of being accented. The accent ratio of a word is equal to the estimated probability of the word being accented if this probability is significantly different from 0.5, and equal to 0.5 otherwise. More formally,

$$ Accent Ratio(w)=\left\{\begin{aligned}&\frac{k}{N}&\text{if}&B(k,N,0.5)\leq0.05\\ &0.5&\text{otherwise}\end{aligned}\right. $$

where $N$ is the total number of times the word $w$ occurred in the training set, $k$ is the number of times it was accented, and $B(k,n,0.5)$ is the probability (under a binomial distribution) that there are $k$ successes in $n$ trials if the probability of success and failure is equal (Nenkova et al., 2007; Yuan et al., 2005).

Features like part-of-speech, N-grams, TF*IDF, and accent ratio can then be combined in a decision tree to predict accents. While these robust features work relatively well, a number of problems in accent prediction still remain the subject of research.

For example, it is difficult to predict which of the two words should be accented in adjective-noun or noun-noun compounds. Some regularities do exist; for example adjective-noun combinations like new truck are likely to have accent on the right word (new TRUCK), while noun-noun compounds like TREE surgeon are likely to have accent on the left. But the many exceptions to these rules make accent prediction in noun compounds quite complex. For example the noun-noun compound APPLE cake has the accent on the first word while the noun-noun compound apple PIE or city HALL both have the accent on the second word (Liberman and Sproat, 1992; Sproat, 1994, 1998a).

Another complication has to do with rhythm; in general speakers avoid putting accents too close together (a phenomenon known as clash) or too far apart (lapse).

Thus city HALL and PARKING lot combine as CITY hall PARKING lot with the accent on HALL shifting forward to CITY to avoid the clash with the accent on PARKING (Liberman and Prince, 1977),

Some of these rhythmic constraints can be modeled by using machine learning techniques that are more appropriate for sequence modeling. This can be done by running a decision tree classifier left to right through a sentence, and using the output of the previous word as a feature, or by using more sophisticated machine learning models like Conditional Random Fields (CRFs) (Gregory and Altun, 2004).