WARN sdfs - Unable to flush FileChunkStore

[Morten Bohmer]

Hi

We are trying out SDFS 2.0 RC1 on FC18.

We mount catalog localy and present a file via iscsi to and Windows 2008R2 computer.

All seems to be working fine, but when we start to run for more than 10-15GB, the system halts on the Win2008R2 server, which in case seems to be due to iscsi/sdfs stops responding.

Also performance decreses dramaticly from we start operations to it halts, we can typicaly start out on 1200MB/s and it ticks down to everything halts.

In the logfile of the volume we get a lot of the following:

java.nio.channels.ClosedChannelException
    at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:99)
    at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:367)
    at org.opendedup.sdfs.filestore.FileChunkStore.sync(FileChunkStore.java:220)
    at org.opendedup.sdfs.filestore.FileChunkStore$SyncThread.run(FileChunkStore.java:461)
    at java.lang.Thread.run(Thread.java:744)
2014-03-28 12:51:26,591 [Thread-9] WARN sdfs  - Unable to flush FileChunkStore


The volume, anyone encountered similar issues ?


Morten

[Sam Silverberg]

Morton,

Can you provide the full log? I would like to see the full error report.

--
You received this message because you are subscribed to the Google Groups "dedupfilesystem-sdfs-user-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dedupfilesystem-sdfs-u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Morten Bohmer]

Full logs attached....

[Sam Silverberg]

Thanks Morten,

I am not sure what lead to this but fixed the issue with the log. Please validate and send me any other logs is issues arise.

I have attached the sdfs.jar with the fix. Please overwrite the one in /usr/share/sdfs/lib with this one and restart.

--

[Morten Bohmer]

Thank you!

Got the following in the log, anything to worry about ?

2014-03-28 15:16:12,110 [Thread-11] INFO sdfs  - Hashtable [/mnt/vol1/ExchangeArchive/chunkstore/hdb/hashstore-sdfs] closed
2014-03-28 15:16:12,112 [Thread-11] INFO sdfs  - SDFS is Shut Down
2014-03-28 15:16:12,112 [Thread-11] INFO sdfs  - All Data Flushed
2014-03-28 15:16:12,133 [Thread-11] INFO sdfs  - SDFS Shut Down Cleanly
2014-03-28 15:18:02,712 [main] INFO sdfs  - Running SDFS Version 2.0RC2
2014-03-28 15:18:02,716 [main] INFO sdfs  - Parsing volume subsystem-config version 2.0RC1
2014-03-28 15:18:02,716 [main] INFO sdfs  - parsing folder locations
2014-03-28 15:18:02,717 [main] INFO sdfs  - Setting hash engine to murmur3_128
2014-03-28 15:18:02,720 [main] INFO sdfs  - Mounting volume /mnt/vol1/ExchangeArchive/files
2014-03-28 15:18:02,723 [main] INFO sdfs  - Volume write threshold is 0.95
2014-03-28 15:18:02,724 [main] INFO sdfs  - Setting volume size to 10995116277760
2014-03-28 15:18:02,724 [main] INFO sdfs  - Setting maximum capacity to 10445360463872
2014-03-28 15:18:02,725 [main] INFO sdfs  - ######### Will allocate 10995116277760 in chunkstore ##############
2014-03-28 15:18:02,961 [main] INFO sdfs  - ###################### SDFSCLI SSL Management WebServer Started at localhost/127.0.0.1:6442 #########################
2014-03-28 15:18:03,311 [main] INFO sdfs  - Starting local chunkstore
2014-03-28 15:18:03,312 [main] INFO sdfs  - Opening Chunk Store
2014-03-28 15:18:03,313 [main] INFO sdfs  - Loading freebits bitset
2014-03-28 15:18:03,316 [main] INFO sdfs  - ChunkStore /mnt/vol1/ExchangeArchive/chunkstore/chunks/chunks.chk created
2014-03-28 15:18:03,319 [main] INFO sdfs  - Loading hashdb class org.opendedup.collections.FileBasedCSMap
2014-03-28 15:18:03,319 [main] INFO sdfs  - Maximum Number of Entries is 2684362560
2014-03-28 15:18:03,322 [main] INFO sdfs  - Loading freebits bitset
2014-03-28 15:18:17,727 [Thread-1] ERROR sdfs  - Unable to check if full.
java.lang.NullPointerException
        at org.opendedup.sdfs.servers.HCServiceProxy.getChunkStore(HCServiceProxy.java:194)
        at org.opendedup.sdfs.servers.HCServiceProxy.getDSECompressedSize(HCServiceProxy.java:163)
        at org.opendedup.sdfs.io.Volume.getCurrentSize(Volume.java:381)
        at org.opendedup.sdfs.io.VolumeFullThread.isFull(VolumeFullThread.java:67)
        at org.opendedup.sdfs.io.VolumeFullThread.run(VolumeFullThread.java:26)
        at java.lang.Thread.run(Thread.java:744)

On Friday, March 28, 2014 12:57:29 PM UTC+1, Morten Bohmer wrote:

[Sam Silverberg]

That error is expected. Will remove that error output before final release.

--