PageIterator - Out of memory error

[Nitish Gupta]

For faster processing, I am trying to use the PageIterator class. I set the bufferLimit to be 500,000. After reading those many pages, I get a Java Out of Memory: heap space error. I have tried allocating 32GB to the JVM, still get the same issue. 

Any ideas for processing the Pages faster?

Thanks in advance,

Nitish Gupta

[Torsten Zesch]

well, if a buffer limit of 500,000 gives you memory troubles, either buy more memory or reduce the buffer size :)

If the processing with a smaller buffer is not fast enough for your purposes, I don't think there is a good solution without refactoring most of the JWPL core.

-Torsten

--
You received this message because you are subscribed to the Google Groups "jwpl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jwpl+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nitish Gupta]

Sorry for the very stupid question. Because it was processing first 'BufferSize' document correctly then was giving error on the next buffer, I thought something to be broken. Reducing the buffer slightly was able to run. 

To unsubscribe from this group and stop receiving emails from it, send an email to jwpl+uns...@googlegroups.com.