j.u.concurrent.RejectedExecutionException with ConverterServlet

[Emerout]

Hi,

I'm using jodconverter-3.0-beta-2 and the associated sample-webapp.

Running the ConverterServlet with a simple test case is Ok.

I'm now trying to stress the servlet with its converter, sending many
post requests concurrently (about 2 to 5 requests simultaneously).

The Servlet quickly fail serving requests with this error :

GRAVE: "Servlet.service()" pour la servlet ConverterServlet a généré
une exception
java.util.concurrent.RejectedExecutionException
at java.util.concurrent.ThreadPoolExecutor
$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1760)
at java.util.concurrent.ThreadPoolExecutor.reject
(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute
(ThreadPoolExecutor.java:658)
at java.util.concurrent.AbstractExecutorService.submit
(AbstractExecutorService.java:78)
at net.sf.jodconverter.office.ManagedProcessOfficeManager.execute
(ManagedProcessOfficeManager.java:77)
at net.sf.jodconverter.OfficeDocumentConverter.convert
(OfficeDocumentConverter.java:71)
at net.sf.jodconverter.sample.web.ConverterServlet.doPost
(ConverterServlet.java:75)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)

[Mirko Nasato]

Hi Emeric,

2009/5/11 Emerout <emeric....@gmail.com>:

>
> I'm using jodconverter-3.0-beta-2 and the associated sample-webapp.
>
> Running the ConverterServlet with a simple test case is Ok.
>
> I'm now trying to stress the servlet with its converter, sending many
> post requests concurrently (about 2 to 5 requests simultaneously).
>
> The Servlet quickly fail serving requests with this error :
>
> GRAVE: "Servlet.service()" pour la servlet ConverterServlet a généré
> une exception
> java.util.concurrent.RejectedExecutionException
>

That's what you get if OOo is busy converting previous requests and it
doesn't become free within the configured timeout - by default 30
seconds. Following good stability patterns[1] if OOo can't cope with
the load JODConverter will start rejecting requests, rather than
causing the whole application to hang.

What's the goal of your stress test? I.e. how many conversions per
time unit and for what document types/sizes? There are ways to improve
throughput if required.

Cheers

Mirko

[1] see "Release It!", Michael T. Nygard, 2007

[Emeric Chardiny]

Thanks for your response Mirko,

Mirko Nasato a écrit :

Hi Emeric,

2009/5/11 Emerout <emeric....@gmail.com>:
  

I'm using jodconverter-3.0-beta-2 and the associated sample-webapp.

Running the ConverterServlet with a simple test case is Ok.

I'm now trying to stress the servlet with its converter, sending many
post requests concurrently (about 2 to 5 requests simultaneously).

The Servlet quickly fail serving requests with this error :

GRAVE: "Servlet.service()" pour la servlet ConverterServlet a généré
une exception
java.util.concurrent.RejectedExecutionException

    

That's what you get if OOo is busy converting previous requests and it
doesn't become free within the configured timeout - by default 30
seconds. Following good stability patterns[1] if OOo can't cope with
the load JODConverter will start rejecting requests, rather than
causing the whole application to hang.

  

Ok, so this behaviour is intentional : the managed OfficeConverter queue the requests until timeout is reached.

What's the goal of your stress test? I.e. how many conversions per
time unit and for what document types/sizes? There are ways to improve
throughput if required.

  

The idea of this stress was to evaluate Ooo / JodConverter capabilities for converting lot of documents.

In fact, I'm designing a platform responsible of converting various office documents (text, spreadsheets, images, ppt, ...)  to PDF/A.
This platform will potentially process sessions of many (thousands) documents. It will have to manage queue and concurrency.

In this case, servlet will not be the choice.
An asynch. mode for serving this type of jobs will be necessary.
I know that in asynch mode, it is not the same design for serving requests,
but queueing and concurrency will have to be managed in a similar way.

Starting many Ooo processes will also be necessary to increase service rate.

Does the targeted JodConverter 3.0 will deal with this queue and concurrency requirements ?

Regards,
Emeric.

[Mirko Nasato]

Hi Emeric,

2009/5/11 Emeric Chardiny <emeric....@gmail.com>:

>
> In fact, I'm designing a platform responsible of converting various office
> documents (text, spreadsheets, images, ppt, ...)  to PDF/A.
> This platform will potentially process sessions of many (thousands)
> documents. It will have to manage queue and concurrency.
>

Sounds interesting.

> In this case, servlet will not be the choice.
> An asynch. mode for serving this type of jobs will be necessary.
> I know that in asynch mode, it is not the same design for serving requests,
> but queueing and concurrency will have to be managed in a similar way.
>
> Starting many Ooo processes will also be necessary to increase service rate.
>
>

This feature will be (re)added soon - it's issue #24 in the tracker.

> Does the targeted JodConverter 3.0 will deal with this queue and concurrency
> requirements ?
>

My focus was mainly on web applications, so I didn't plan a queueing
mechanism specific for your scenario. But plans can be changed of
course.

Kind regards

Mirko