"java.net.SocketException: Broken pipe" error when running in Network Mode.

195 views
Skip to first unread message

Tarandeep Gill

unread,
Dec 2, 2011, 11:44:11 AM12/2/11
to dedupfilesystem-...@googlegroups.com
Hello,

I am trying to run OpenDedup in network mode. There are 10 DSE services running on the local computer from ports 22000 to 22009. The chunks are divided evenly among the processes.

When I mount an SDFS volume, everything looks file, the servers are being added. But as soon as I try to copy a file into the SDFS volume, I get the "java.net.SocketException: Broken pipe" error. Nothing gets written into any of the chunkstores.

Config files for the 10 DSE processes, routing config file, and the volume config file are attached.

The output from the mount.sdfs command is given below:

root@test-server:/opt/sdfs-bin# ./mount.sdfs -m /mnt/test/ -r /opt/sdfs/config/routing-config.xml -vc /opt/sdfs/config/volumes/vol-0.xml

Running SDFS Version 1.1.2
 INFO [main] (SDFSService.java:34) - reading routing config file = /opt/sdfs/config/routing-config.xml
reading config file = /opt/sdfs/config/volumes/vol-0.xml
 INFO [main] (Config.java:165) - Running SDFS Version 1.1.2
 INFO [main] (Config.java:167) - Parsing subsystem-config version 1.1.2
 INFO [main] (Config.java:172) - parsing folder locations
 INFO [main] (Config.java:189) - Setting hash size to 16
 INFO [main] (Volume.java:66) - Mounting volume /opt/sdfs/volumes/vol-0/files
 INFO [main] (Volume.java:91) - Setting volume size to 2147483648
 INFO [main] (Volume.java:93) - Setting maximum capacity to 2147483648
 INFO [main] (MgmtWebServer.java:355) - ###################### SDFSCLI Management WebServer Started at localhost/127.0.0.1:6442 #########################
 INFO [main] (Config.java:367) - Parsing routing config /opt/sdfs/config/routing-config.xml
 INFO [main] (Config.java:386) - Parsing routing-config
 INFO [main] (Config.java:390) - parsing Servers
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server0
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server1
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server2
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server3
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server4
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server5
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server6
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server7
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server8
 INFO [main] (Config.java:393) - Connection to  Servers [10]
 INFO [main] (Config.java:417) - Added Server server9
 INFO [main] (SDFSGCScheduler.java:26) - Scheduling FDISK Jobs for SDFS
 INFO [main] (SDFSGCScheduler.java:34) - Garbage Collection Jobs Scheduled
 INFO [main] (MountSDFS.java:160) - Mount Option : -f
 INFO [main] (MountSDFS.java:160) - Mount Option : /mnt/test/
 INFO [main] (MountSDFS.java:160) - Mount Option : -o
 INFO [main] (MountSDFS.java:160) - Mount Option : direct_io,big_writes,allow_other,fsname=sdfs:/opt/sdfs/config/volumes/vol-0.xml:6442
 INFO [main] (SDFSFileSystem.java:47) - mounting /opt/sdfs/volumes/vol-0/files to /mnt/test/
08:24:12.263     main  INFO [fuse.FuseMount]: Mounted filesystem
FATAL [Thread-11] (HashClient.java:102) - unable to execute command
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at org.opendedup.sdfs.network.WriteHashCmd.executeCmd(WriteHashCmd.java:47)
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:95)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:111)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
 WARN [Thread-11] (HCServiceProxy.java:113) - unable to use hashclient
java.io.IOException: unable to execute command
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:103)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:111)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
FATAL [Thread-11] (HashClient.java:102) - unable to execute command
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at java.io.DataInputStream.readBoolean(DataInputStream.java:242)
at org.opendedup.sdfs.network.WriteHashCmd.executeCmd(WriteHashCmd.java:48)
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:95)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:116)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
FATAL [Thread-11] (HCServiceProxy.java:121) - Unable to write chunk [B@6fdb6621
java.io.IOException: unable to execute command
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:103)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:116)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
FATAL [Thread-17] (HashClient.java:102) - unable to execute command
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at java.io.DataInputStream.readBoolean(DataInputStream.java:242)
at org.opendedup.sdfs.network.WriteHashCmd.executeCmd(WriteHashCmd.java:48)
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:95)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:111)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
FATAL [Thread-11] (SparseDedupFile.java:314) - unable to add chunk [[B@6fb3cd4a] at position 131072
java.io.IOException: Unable to write chunk [B@6fdb6621
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:122)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
 WARN [Thread-17] (HCServiceProxy.java:113) - unable to use hashclient
java.io.IOException: unable to execute command
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:103)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:111)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
FATAL [Thread-17] (HashClient.java:102) - unable to execute command
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at java.io.DataInputStream.readBoolean(DataInputStream.java:242)
at org.opendedup.sdfs.network.WriteHashCmd.executeCmd(WriteHashCmd.java:48)
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:95)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:116)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
FATAL [Thread-17] (HCServiceProxy.java:121) - Unable to write chunk [B@dd86098
java.io.IOException: unable to execute command
at org.opendedup.sdfs.network.HashClient.executeCmd(HashClient.java:103)
at org.opendedup.sdfs.network.HashClient.writeChunk(HashClient.java:172)
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:116)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)
FATAL [Thread-17] (SparseDedupFile.java:314) - unable to add chunk [[B@346e2756] at position 0
java.io.IOException: Unable to write chunk [B@dd86098
at org.opendedup.sdfs.servers.HCServiceProxy.writeChunk(HCServiceProxy.java:122)
at org.opendedup.sdfs.io.SparseDedupFile.writeCache(SparseDedupFile.java:279)
at org.opendedup.sdfs.io.WritableCacheBuffer.close(WritableCacheBuffer.java:338)
at org.opendedup.util.PoolThread.run(PoolThread.java:31)




Any help would be highly appreciated :)

Thanks in advance.

Tarandeep Gill

unread,
Dec 2, 2011, 11:52:50 AM12/2/11
to dedupfilesystem-...@googlegroups.com
dse-0.xml

Tarandeep Gill

unread,
Dec 2, 2011, 11:53:25 AM12/2/11
to dedupfilesystem-...@googlegroups.com
Google won't let me upload multiple files at a time, so attaching them one at a time.
routing-config.xml

Tarandeep Gill

unread,
Dec 2, 2011, 11:53:46 AM12/2/11
to dedupfilesystem-...@googlegroups.com
volume config file.
vol-0.xml

Tarandeep Gill

unread,
Dec 3, 2011, 8:40:05 AM12/3/11
to dedupfilesystem-...@googlegroups.com
Ok, after some debugging here is what I found:

My DSEs are configured with page-size="131072", and in the volume config file, it says: <io chunk-size="128" .... >

So, ideally the client process should send 131072 sized chunks to the server process. And this is exactly what was happening. However, the server process was refusing these chunks because it was expecting 4096 sized chunks. The line numbers 108 of ClientThread.java was failing, because Main.CHUNK_LENGTH is declared as 4 * 1024.

108    if(len != Main.CHUNK_LENGTH)
109           throw new IOException("invalid chunk length " +len);

Changing Main.CHUNK_LENGTH (on line number 46 of Main.java) to 128 * 1024 solved my problem.
Reply all
Reply to author
Forward
0 new messages