23.2.1 Question Processing

The goal of the question processing phase is to extract two things from the question: a keyword query suitable as input to an IR system and an answer type, a specification of the kind of entity that would constitute a reasonable answer to the question.

Query Formulation

The process of query formulation is very similar to the processing done on other IR queries. Our goal is to create from the question a list of keywords that forms an IR query.

Exactly what query to form depends on the question answering application. If question answering is applied to the Web, we might simply create a keyword from every word in the question, letting the web search engine automatically remove any stopwords. Often we leave out the question word (where, when, etc). Alternatively, keywords can be formed from only the terms found in the noun phrases in the question, applying stopword lists to ignore function words and high-frequency, low-content verbs.

When question answering is applied to smaller sets of documents, for example to answer questions about corporate information pages, we still use an IR engine to search our documents for us. But for this smaller set of documents we generally need to apply query expansion. On the Web the answer to a question might appear in many different forms, and so if we search with words from the question we'll probably find an answer written in the same form. In smaller sets of corporate pages, by contrast, an answer might appear only once, and the exact wording might look nothing like the question. Thus query expansion methods can add query terms hoping to match the particular form of the answer as it appears.

Thus we might add to the query all morphological variants of the content words in the question, as well as applying the thesaurus-based or other query expansion algorithms discussed in the previous section to get a larger set of keywords for the query. Many systems use WordNet as a thesaurus, while others rely on special-purpose thesauruses that are specifically hand-built for question-answering.

Another query formulation approach that is sometimes used when questioning the Web is to apply a set of query reformulation rules to the query. The rules rephrase the question to make it look like a substring of possible declarative answers. For example the question “when was the laser invented?” would be reformulated as the laser was

invented; the question “where is the Valley of the Kings?” might be reformulated as “the Valley of the Kings is located in”. We can apply multiple such rules to the query, and pass all the resulting reformulated queries to the web search engine. Here are some sample hand-written reformulation rules from Lin (2007):

(23.20) Where is A $ \rightarrow $ A is located in

(23.19) wh-word did A verb B $ \rightarrow $ $ \ldots $A verb+ed B

Question Classification

The second task in question processing is to classify the question by its expected answer type. For example a question like “Who founded Virgin Airlines” expects an answer of type PERSON. A question like “What Canadian city has the largest population?” expects an answer of type CITY. This task is called question classification or answer type recognition. If we know the answer type for a question, we can avoid looking at every sentence or noun phrase in the entire suite of documents for the answer, instead focusing on, e.g., just people or cities. Knowing an answer type is also important for presenting the answer. A DEFINITION question like “What is a prism” might use a simple answer template like “A prism is...” while an answer to a BIOGRAPHY question like “Who is Zhou Enlai?” might use a biography-specific template, perhaps beginning with the persons nationality and proceeding to their dates of birth and other biographical information.

As some of the above examples suggest, we might draw the set of possible answer types for a question classifier from a set of named entities like the PERSON, LOCATION, and ORGANIZATION described in Ch. 22. Usually, however, a somewhat richer set of answer types is used. These richer tagsets are often hierarchical, and so we usually call them an answer type taxonomy or a question ontology. Such taxonomies can be built semi-automatically and dynamically, for example from WordNet (Harabagiu et al., 2000; Pasca, 2003), or they can be designed by hand.

Fig. 23.9 shows one such hand-built ontology, the hierarchical Li and Roth (2005) tagset. In this tagset, each question can be labeled with a coarse-grained tag like HUMAN, or a fine-grained tag like HUMAN:DESCRIPTION, HUMAN:GROUP, HUMAN:IND, and so on. Similar tags are used in other systems; the type HUMAN:DESCRIPTION is often called a BIOGRAPHY question, because the answer requires giving a brief biography of the person, rather than just a name.

Question classifiers can be built by hand-writing rules, via supervised machine learning, or via some combination. The Webclopedia QA Typology, for example, contains 276 hand-written rules associated with the approximately 180 answer types in the typology (Hovy et al., 2002). A regular expression rule for detecting an answer type like BIOGRAPHY (which assumes the question has been named-entity tagged) might be:

who {is | was | are | were} PERSON

Most modern question classifiers, however, are based on supervised machine learning techniques. These classifiers are trained on databases of questions that have been hand-labeled with an answer type such as the corpus of Li and Roth (2002). Typical features used for classification include the words in the questions, the part-of-speech of each word, and the named entities in the questions.

| Tag | Example |

| --- | --- |

| ABBREVIATION | |

| abb\nexp | What's the abbreviation for limited partnership?\nWhat does the “c” stand for in the equation E=mc $ ^{2} $? |

| DESCRIPTION | |

| definition\ndescription\nmanner\nreason | What are tannins ?\nWhat are the words to the Canadian National anthem?\nHow can you get rust stains out of clothing?\nWhat caused the Titanic to sink ? |

| ENTITY | |

| animal\nbody\ncolor\ncreative\ncurrency\ndisease/medicine\nevent\nfood\ninstrument\nlang\nletter\nother\nplant\nproduct\nreligion\nsport\nsubstance\nsymbol\ntechnique\nterm\nvehicle\nword | What are the names of Odin’s ravens?\nWhat part of your body contains the corpus callosum ?\nWhat colors make up a rainbow ?\nIn what book can I find the story of Aladdin?\nWhat currency is used in China?\nWhat does Salk vaccine prevent ?\nWhat war involved the battle of Chapultepec?\nWhat kind of nuts are used in marzipan?\nWhat instrument does Max Roach play?\nWhat’s the official language of Algeria?\nWhat letter appears on the cold-water tap in Spain?\nWhat is the name of King Arthur’s sword?\nWhat are some fragrant white climbing roses?\nWhat is the fastest computer ?\nWhat religion has the most members ?\nWhat was the name of the ball game played by the Mayans?\nWhat fuel do airplanes use?\nWhat is the chemical symbol for nitrogen ?\nWhat is the best way to remove wallpaper?\nHow do you say “Grandma” in Irish ?\nWhat was the name of Captain Bligh’s ship ?\nWhat’s the singular of dice? |

| HUMAN | |

| description\ngroup\nind\ntitle | Who was Confucius?\nWhat are the major companies that are part of Dow Jones ?\nWho was the first Russian astronaut to do a spacewalk?\nWhat was Queen Victoria’s title regarding India? |

| LOCATION | |

| city\ncountry\nmountain\nother\nstate | What’s the oldest capital city in the Americas ?\nWhat country borders the most others?\nWhat is the highest peak in Africa?\nWhat river runs through Liverpool?\nWhat states do not have state income tax? |

| NUMERIC | |

| code\ncount\ndate\ndistance\nmoney\norder\nother\nperiod\npercent\nspeed\ntemp\nsize\nweight | What is the telephone number for the University of Colorado?\nAbout how many soldiers died in World War II?\nWhat is the date of Boxing Day?\nHow long was Mao’s 1930s Long March?\nHow much did a McDonald’s hamburger cost in 1963?\nWhere does Shanghai rank among world cities in population?\nWhat is the population of Mexico?\nWhat was the average life expectancy during the Stone Age?\nWhat is the speed of the Mississippi River?\nHow fast must a spacecraft travel to escape Earth’s gravity?\nWhat is the size of Argentina?\nHow many pounds are there in a stone? |

Often a single word in the question gives extra information about the answer type, and its identity is used as a feature. This word is sometimes called the question headword or the answer type word, and may be defined as the headword of the first NP after the question's wh-word; headwords are indicated in boldface in the following examples:

(23.22) Which city in China has the largest number of foreign financial companies.

(23.23) What is the state flower of California?

Finally, it often helps to use semantic information about the words in the questions. The WordNet synset id of the word can be used as a feature, as can the ids of the hypernym and hyponyms of each word in the question.

In general question classification accuracies are relatively high on easy question types like PERSON, LOCATION, and TIME questions; detecting REASON and DESCRIPTION questions can be much harder.