I noticed that in the example sentence given for the last assignment, "The green dog ate a large cookie on the table", neither ATE nor COOKIE are contained in the lexicon.

Should our taggers be expected to recognize them as VB and NN respectively, anyway? (As of now, my tagger treats them both as NNPs)

Thank you.

-- Anonymous, March 27, 1999

Answers

The easy solution is to add 'ate' and 'cookie' to the lexicon (using one of the techniques we discussed for the tagger assignment).

A more general solution is to make your tagger able to recognize that an unknown word between an adjective and a preposition is a noun, and that an unknown word between a noun and an article is probably a verb (it could also be a preposition, but the class of prepositions is closed and it's probably fair to assume that you've got them all).

Alternatively, your parser could recognize these patterns involving unknown words -- but then it's essentially duplicating the work that your heuristic tagger already has the architecture to

-- Anonymous, March 30, 1999