Problem using httpnio extension

[beobal]

Hi,

I'm having some issues using the httpnio extension for jclouds S3
interface. I'm using the low level, async apis I get the following
stacktrace whenever I try to put objects to S3. This is undoubtedly
down to something I'm doing/not doing, but I'd be very grateful for
any pointers.

Cheers,
Sam


28-Sep-2009 07:07:04 org.jclouds.logging.jdk.JDKLogger logWarn
WARNING:
org.jclouds.http.httpnio.pool.HttpNioFutureCommandConnectionPool@fcd4eca1
- saturated connection pool

28-Sep-2009 07:07:06 org.jclouds.logging.jdk.JDKLogger logWarn
WARNING:
org.jclouds.http.httpnio.pool.HttpNioFutureCommandConnectionPool@11ae6cab
- saturated connection pool
28-Sep-2009 07:07:07 org.jclouds.logging.jdk.JDKLogger logError
SEVERE: Error dispatching
org.apache.http.impl.nio.SSLClientIOEventDispatch@c3e967
org.apache.http.nio.reactor.IOReactorException: I/O dispatch worker
terminated abnormally
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute
(AbstractMultiworkerIOReactor.java:326)
at org.jclouds.http.httpnio.pool.HttpNioFutureCommandConnectionPool
$1.run(HttpNioFutureCommandConnectionPool.java:116)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at org.apache.http.nio.protocol.AsyncNHttpClientHandler.outputReady
(AsyncNHttpClientHandler.java:266)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.produceOutput
(DefaultNHttpClientConnection.java:213)
at org.apache.http.impl.nio.SSLClientIOEventDispatch.outputReady
(SSLClientIOEventDispatch.java:242)
at org.apache.http.impl.nio.reactor.BaseIOReactor.writable
(BaseIOReactor.java:177)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent
(AbstractIOReactor.java:317)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents
(AbstractIOReactor.java:294)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute
(AbstractIOReactor.java:256)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute
(BaseIOReactor.java:96)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor
$Worker.run(AbstractMultiworkerIOReactor.java:556)
... 1 more
28-Sep-2009 07:07:07 org.jclouds.logging.jdk.JDKLogger logInfo
INFO: FutureCommandConnectionPoolClient{status=SHUT_DOWN,
commandQueue=0, poolMap={https://talisplatformdev-eu.s3-
external-3.amazonaws.com:
443=org.jclouds.http.httpnio.pool.HttpNioFutureCommandConnect
ionPool@11ae6cab, https://s3.amazonaws.com:443=org.jclouds.http.httpnio.pool.HttpNioFutureCommandConnectionPool@fcd4eca1}}
28-Sep-2009 07:07:07 org.jclouds.logging.jdk.JDKLogger logInfo
INFO:
org.jclouds.http.httpnio.pool.HttpNioFutureCommandConnectionPool@fcd4eca1

[Adrian Cole]

Hi, Sam.

Sorry you are having problems with this.  Have you verified this is an issue only in the httpnio extension?  I recommend switching to the java one and noting the behavior.

Note that in our pending release of jclouds-beta3, we've tuned the standard http engine to the point where it is almost always faster and as scalable as the nio one.  In release beta-3, we will downgrade the httpnio engine to experimental status until someone stabilizes and tunes it.

Best regards,
-Adrian

[Sam Tunnicliffe]

Hi Adrian,

thanks for the super-fast response! Yes, I've tried the default java connector (and I read the tuning nio thread), and that works fine in light and serial usage. But, I've hit a different problem when using it to put a number of large objects concurrently, I seem to get OutOfMemory errors, which I haven't had chance to track down yet. From a (very) brief skim, I wondered if the object bodies were being buffered somewhere during the POST to S3. I've tried throttling the uploads, but tbh I would have expected the memory consumption caused by an upload to be unrelated to the size of the object itself. I appreciate this is a bit vague, so I'll try and spend some time isolating this issue a bit better.

Cheers,

Sam

[Adrian Cole]

No worries, Sam.

Let me know what you find on this.  Also, what size are your objects?  I can probably turn up the heat on our performance tests to see how they run. 

The nio engine is better in beta-3 as well.  Mostly logging is more useful.  I hope to have this out by wednesday.

Cheers,
-Adrian

P.S.  If you are interested in assisting, we'd love to have help knocking some of these kinks out.  There are test frameworks in jclouds that could be used to replicate your scenario.