Reusable connections when executing several bulk operations

[Henrik]

Hi

I'm running into some odd misbehavior in Ektorp:

If i do the following

localClient = new StdHttpClient.Builder()
.host(host)
.port(port)
.build();

instance = new StdCouchDbInstance(localClient);
conn = new StdCouchDbConnector(local_name, localInstance);

//fetch some rows from and old mysql - formatted for couch

List<CouchCompany> users = jdbcTemplate.getUsersStuff()

db.executeBulk(users); //This one usually works ok - but there seem to
be a race condition - if i breakpoint too long - it will fail as well

List<CouchCompany> companies = jdbcTemplate.getCompaniesStuff()

db.executeBulk(companies); //Always fails with a socket exception - as
shown below:

I can see in the couchdb log that the second request never touches
couch - and it seems to be the connection itself - getting closed but
not remove from whatever connection manager might be handling it.


[HttpRoute[{}->http://localhost:5984]] kept alive: 1, issued: 0,
allocated: 1 out of 20
Getting free connection [HttpRoute[{}->http://localhost:5984]][null]
Stale connection check
CookieSpec selected: best-match
Attempt 1 to execute request
Sending request: POST /cphse_cluster/_bulk_docs HTTP/1.1
>> "POST /cphse_cluster/_bulk_docs HTTP/1.1[EOL]"
>> "Transfer-Encoding: chunked[EOL]"
>> "Content-Type: application/json[EOL]"
>> "Host: localhost:5984[EOL]"
>> "Connection: Keep-Alive[EOL]"
>> "Expect: 100-Continue[EOL]"
>> "[EOL]"
>> POST /cphse_cluster/_bulk_docs HTTP/1.1
>> Transfer-Encoding: chunked
>> Content-Type: application/json
>> Host: localhost:5984
>> Connection: Keep-Alive
>> Expect: 100-Continue
>> "107[EOL]"
I/O error closing connection
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
Exception in thread "main" org.ektorp.DbAccessException:
org.apache.http.client.ClientProtocolException
at org.ektorp.util.Exceptions.propagate(Exceptions.java:19)
at
org.ektorp.http.StdHttpClient.executeRequest(StdHttpClient.java:105)
at org.ektorp.http.StdHttpClient.post(StdHttpClient.java:55)
at org.ektorp.http.RestTemplate.post(RestTemplate.java:57)
at
org.ektorp.impl.StdCouchDbConnector.executeBulk(StdCouchDbConnector.java:
431)
at
org.ektorp.impl.StdCouchDbConnector.executeBulk(StdCouchDbConnector.java:
393)
at dbmigration.Migrator.migrateCompanies(Migrator.java:129)
at dbmigration.Migrator.migrate(Migrator.java:186)
at dbmigration.Main.main(Main.java:15)
Caused by: org.apache.http.client.ClientProtocolException
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
643)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
576)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
554)
at
org.ektorp.http.StdHttpClient.executeRequest(StdHttpClient.java:97)
... 7 more
Caused by: org.apache.http.client.NonRepeatableRequestException:
Cannot retry request with a non-repeatable request entity. The cause
lists the reason the original request failed.
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:
470)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
641)
... 10 more
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:
136)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:
106)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.write(AbstractSessionOutputBuffer.java:
126)
at
org.apache.http.impl.conn.LoggingSessionOutputBuffer.write(LoggingSessionOutputBuffer.java:
64)
at
org.apache.http.impl.io.ChunkedOutputStream.flushCache(ChunkedOutputStream.java:
99)
at
org.apache.http.impl.io.ChunkedOutputStream.finish(ChunkedOutputStream.java:
131)
at
org.apache.http.impl.io.ChunkedOutputStream.close(ChunkedOutputStream.java:
185)
at
org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:
128)
at
org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:
253)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:
218)
at
org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:
249)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:
124)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:
483)
... 11 more
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:
136)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:
106)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:
113)
at
org.apache.http.impl.conn.LoggingSessionOutputBuffer.flush(LoggingSessionOutputBuffer.java:
85)
at
org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:
260)
at
org.apache.http.impl.SocketHttpClientConnection.close(SocketHttpClientConnection.java:
248)
at
org.apache.http.impl.conn.DefaultClientConnection.close(DefaultClientConnection.java:
156)
at
org.apache.http.impl.conn.AbstractPooledConnAdapter.close(AbstractPooledConnAdapter.java:
148)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:
130)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:
483)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
641)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
576)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
554)
at
org.ektorp.http.StdHttpClient.executeRequest(StdHttpClient.java:97)
at org.ektorp.http.StdHttpClient.post(StdHttpClient.java:55)
at org.ektorp.http.RestTemplate.post(RestTemplate.java:57)
at
org.ektorp.impl.StdCouchDbConnector.executeBulk(StdCouchDbConnector.java:
431)
at
org.ektorp.impl.StdCouchDbConnector.executeBulk(StdCouchDbConnector.java:
393)
at dbmigration.Migrator.migrateCompanies(Migrator.java:129)
at dbmigration.Migrator.migrate(Migrator.java:186)
at dbmigration.Main.main(Main.java:15)
Closing the connection.
Connection closed
I/O exception (java.net.SocketException) caught when processing
request: Broken pipe
Broken pipe
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:
136)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:
106)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.write(AbstractSessionOutputBuffer.java:
126)
at
org.apache.http.impl.conn.LoggingSessionOutputBuffer.write(LoggingSessionOutputBuffer.java:
64)
at
org.apache.http.impl.io.ChunkedOutputStream.flushCache(ChunkedOutputStream.java:
99)
at
org.apache.http.impl.io.ChunkedOutputStream.finish(ChunkedOutputStream.java:
131)
at
org.apache.http.impl.io.ChunkedOutputStream.close(ChunkedOutputStream.java:
185)
at
org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:
128)
at
org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:
253)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:
218)
at
org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:
249)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:
124)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:
483)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
641)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
576)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:
554)
at
org.ektorp.http.StdHttpClient.executeRequest(StdHttpClient.java:97)
at org.ektorp.http.StdHttpClient.post(StdHttpClient.java:55)
at org.ektorp.http.RestTemplate.post(RestTemplate.java:57)
at
org.ektorp.impl.StdCouchDbConnector.executeBulk(StdCouchDbConnector.java:
431)
at
org.ektorp.impl.StdCouchDbConnector.executeBulk(StdCouchDbConnector.java:
393)
at dbmigration.Migrator.migrateCompanies(Migrator.java:129)
at dbmigration.Migrator.migrate(Migrator.java:186)
at dbmigration.Main.main(Main.java:15)
Retrying request
Reopening the direct connection.
Cannot retry non-repeatable request
Connection shut down
Released connection is not reusable.
Releasing connection [HttpRoute[{}->http://localhost:5984]][null]
Notifying no-one, there are no waiting threads

Any help would be very appreciated.

Another thing - since im here :) Would be excellent to be able to
define map and reduce functions separately in a javascript file (code
completion and removing the need for escaping everything in json - if
it was build in)

Best regards
Henrik

[Henrik Lundgren]

We have seen this type of behaviour before, but I was hoping that the
changes made for issue #81:
http://code.google.com/p/ektorp/issues/detail?id=81 would fix this.

This CouchDB issue seems related to your problem:
https://issues.apache.org/jira/browse/COUCHDB-682

Can you create a test that reliably recreates this behaviour?