14.4.1 Independence assumptions miss structural dependencies between rules

Let's look at these problems in more detail. Recall that in a CFG the expansion of a non-terminal is independent of the context, i.e., of the other nearby non-terminals in the parse tree. Similarly, in a PCFG, the probability of a particular rule like $ NP \rightarrow Det $ is also independent of the rest of the tree. By definition, the probability of a group of independent events is the product of their probabilities. These two facts explain why in a PCFG we compute the probability of a tree by just multiplying the probabilities of each non-terminal expansion.

Unfortunately this CFG independence assumption results in poor probability estimates. This is because in English the choice of how a node expands can after all be dependent on the location of the node in the parse tree. For example, in English it turns out that NPs that are syntactic subjects are far more likely to be pronouns, while NPs that are syntactic objects are far more likely to be non-pronominal (e.g., a proper noun or a determiner noun sequence), as shown by these statistics for NPs in the Switchboard corpus (Francis et al., 1999): $ ^{1} $

Unfortunately there is no way to represent this contextual difference in the probabilities in a PCFG. Consider two expansions of the non-terminal NP as a pronoun or as a determiner+noun. How shall we set the probabilities of these two rules? If we set their probabilities to their overall probability in the Switchboard corpus, the two rules have about equal probability.

$$ NP\,\to\,DT~NN\ .28 $$

$$ N P\;\to\;P R P\quad.25 $$

Because PCFGs don’t allow a rule probability to be conditioned on surrounding context, this equal probability is all we get; there is no way to capture the fact that in subject position, the probability for $NP \to PRP$ should go up to .91, while in object position, the probability for $NP \to DT$ NN should go up to .66.

These dependencies could be captured if the probability of expanding an NP as a pronoun (e.g., $NP \to PRP$) versus a lexical NP (e.g., $NP \to DT NN$) were conditioned on whether the NP was a subject or an object. Sec. 14.5 will introduce the technique of parent annotation for adding this kind of conditioning.