Broken Pipe

[Judson Stephenson]

I was doing a few tests to see the throughput of the FleetDB json api
(simple 10,000 ping request) and I got a "Broken pipe" error from the
server (though the request returned [0,pong]).

Here was the error:


Broken pipe
(Unknown Source)
java.net.SocketOutputStream.socketWrite0
SocketOutputStream.java:92
java.net.SocketOutputStream.socketWrite
SocketOutputStream.java:136
java.net.SocketOutputStream.write
StreamEncoder.java:202
sun.nio.cs.StreamEncoder.writeBytes
StreamEncoder.java:272
sun.nio.cs.StreamEncoder.implFlushBuffer
StreamEncoder.java:276
sun.nio.cs.StreamEncoder.implFlush
StreamEncoder.java:122
sun.nio.cs.StreamEncoder.flush
OutputStreamWriter.java:212
java.io.OutputStreamWriter.flush
BufferedWriter.java:236
java.io.BufferedWriter.flush
server.clj:52 fleetdb.server/write-
response
server.clj:77 fleetdb.server/handler
$fn[fn]
server.clj:66 fleetdb.server/
handler
server.clj:113 fleetdb.server/run
$fn[fn]
AFn.java:18 clojure.lang.AFn.call
FutureTask.java:303
java.util.concurrent.FutureTask$Sync.innerRun
FutureTask.java:138
java.util.concurrent.FutureTask.run
ThreadPoolExecutor.java:886
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
ThreadPoolExecutor.java:908
java.util.concurrent.ThreadPoolExecutor$Worker.run
Thread.java:662 java.lang.Thread.run

I am using my updated node-fleet client, available here:
https://github.com/Jud/node-fleet

Judson

[Mark McGranaghan]

Hi Judson,

Thanks for the report. Can you explain how I can reproduce your error?
I have installed and am somewhat familiar with Node, so maybe just
your FleetDB boot script and Node client test script?

Thanks,
Mark

[Judson Stephenson]

Hey Mark,

Not sure what you mean by FleetDB boot script, but I launch it with:

java -cp fleetdb-standalone.jar fleetdb.server -f btxc.fdb


And the test script:

var sys = require('sys'),
fleetdb = require('./inc-js/fleetdb');

var db = new fleetdb.Database();
db.open('http://127.0.0.1:3400');

for(var i=0; i<10000; i++)
{
db.query(['ping'], function(status, data) {
sys.puts('Data:' + JSON.stringify(data));
});
};

db.close();

*sorry about that first post formatting so poorly.

Judson

[Mark McGranaghan]

On Mon, Jan 31, 2011 at 3:17 PM, Judson Stephenson
<J...@trollingtowers.com> wrote:
> Hey Mark,
>
> Not sure what you mean by FleetDB boot script, but I launch it with:
>
> java -cp fleetdb-standalone.jar fleetdb.server -f btxc.fdb

Right, I just meant this. What version of FleetDB are you using by the way?

>
> And the test script:
>
> var sys = require('sys'),
>        fleetdb = require('./inc-js/fleetdb');
>
> var db = new fleetdb.Database();
>        db.open('http://127.0.0.1:3400');
>
> for(var i=0; i<10000; i++)
> {
>        db.query(['ping'], function(status, data) {
>          sys.puts('Data:' + JSON.stringify(data));
>        });
> };
>
> db.close();
>
> *sorry about that first post formatting so poorly.
>
> Judson

I will investigate this tonight.

Thanks,

- Mark

[Judson Stephenson]

Should be the latest, I just installed it yesterday.

Jud

[Mark McGranaghan]

Judson,

The error message might be due to exiting from Node before all
enqueued requests have been fully processes, which might cause the
client to exit mid-request. This variation on your script shows that
not all responses are received before the script exits:

https://gist.github.com/805775

The script below seems to work fine for me, except for a Node error at
the very end that I don't understand:

https://gist.github.com/805776

Let me know if this gets you any close to the root cause.

- Mark

On Mon, Jan 31, 2011 at 3:25 PM, Judson Stephenson

[Judson Stephenson]

Thanks Mark,

That should get me closer to the real issue.

Judson

[Judson Stephenson]

Fixed it Mark.

I had just forgotten about the async-ism's that come with node.

Once all the requests were pushed to the queue, it closed the DB
connection. I probably would have figured it out if I was doing
something other than 'pinging' the db, since random amounts of data
wouldn't have been there.

Thanks again!
Judson