Problem with StanfordParser

[Piyush Paliwal]

Hi,

While running the StanfordParser in a pipeline for processing xmi files, I got the following message:

FactoredParser: exceeded MAX_ITEMS work limit [200000 items]; aborting.

So far what I found is that the setting -MAX_ITEMS = large number, may solve this issue. However, I do not seem to find as such parameter setting which I could change in StanfordParser annotator.

Is that PARAM_MAX_TOKENS? I changed this parameter from default to higher number, still the problem persists.

Thanks!

Piyush

[Richard Eckart de Castilho]

Hello Piyush,

the PARAM_MAX_TOKENS sets a limit for tokens in a sentence. If there are more than that many tokens, the parser will skip the sentence. This is meant to avoid out-of-memory situations with very large sentences.

The message you get from the factored parser rather unrelated. If the factored parser finds the sentence too complex, then it falls back to the PCFG parser.

See also: https://mailman.stanford.edu/pipermail/parser-user/2011-June/001091.html

If you think that we should add a parameter the the StanfordParser to change the MAX_ITEMS used by the parser, please open an issue on the Google Code issue tracker of DKPro Core ASL [1].

Cheers,

-- Richard

[1] https://code.google.com/p/dkpro-core-asl/issues/list