Java Issue - Opendedup on Centos 5.4

170 views
Skip to first unread message

Stevester

unread,
May 8, 2010, 1:17:32 PM5/8/10
to dedupfilesystem-sdfs-user-discuss
Good Day Everyone,

I am having an issue with running Opendedup on Centos 5.4. It seems
to be java related as the deduplication process doesnt seem to happen.
I have an NFS share exported at /srv and mounted to the SDFS volume.
Here is what get when perform the mount.sdfs command.


[root@kallenxp sdfs-bin]# export JAVA_HOME=/usr/local/src/jdk1.7.0/
jre[root@kallenxp sdfs-bin]# ./mount.sdfs -v deduped -m /srv -o
direct_io,big_writes,allow_other,fsname=deduped,nonempty
May 8, 2010 12:14:28 PM org.opendedup.sdfs.servers.SDFSService <init>
INFO: Running SDFS Version 0.9.2
May 8, 2010 12:14:28 PM org.opendedup.sdfs.servers.SDFSService <init>
INFO: reading config file = /etc/sdfs/deduped-volume-cfg.xml
May 8, 2010 12:14:29 PM org.opendedup.sdfs.Config parseSDFSConfigFile
INFO: Parsing subsystem-config version 0.9.2
May 8, 2010 12:14:29 PM org.opendedup.sdfs.Config parseSDFSConfigFile
INFO: parsing folder locations
May 8, 2010 12:14:29 PM org.opendedup.sdfs.io.Volume <init>
INFO: Mounting volume /opt/sdfs/deduped/files
May 8, 2010 12:14:29 PM org.opendedup.sdfs.io.Volume <init>
INFO: Setting volume size to 107374182400
May 8, 2010 12:14:29 PM org.opendedup.sdfs.Config parseSDFSConfigFile
INFO: ######### Will allocate 107374182400 in chunkstore
##############
May 8, 2010 12:14:31 PM org.opendedup.collections.CSByteArrayLongMap
setUp
INFO: This looks an existing hashtable will repopulate with [827200]
entries.
May 8, 2010 12:14:31 PM org.opendedup.collections.CSByteArrayLongMap
setUp
INFO: ##################### Loading Hash Database
#####################
Loading Done Loading
May 8, 2010 12:14:31 PM org.opendedup.collections.CSByteArrayLongMap
setUp
INFO: ########## Finished Loading Hash Database in [0] seconds
###########
May 8, 2010 12:14:31 PM org.opendedup.collections.CSByteArrayLongMap
setUp
INFO: loaded [0] into the hashtable [/opt/sdfs/deduped/chunkstore/hdb/
hashstore-sdfs] free slots available are [0] [0]
May 8, 2010 12:14:31 PM org.opendedup.sdfs.filestore.HashStore <init>
INFO: Total Entries 0
May 8, 2010 12:14:31 PM org.opendedup.sdfs.filestore.HashStore <init>
INFO: Added sdfs
May 8, 2010 12:14:31 PM
org.opendedup.sdfs.filestore.gc.ChunkStoreGCScheduler <init>
INFO: Scheduling Garbage Collection Jobs
12:14:31.225 main INFO [org.quartz.core.SchedulerSignalerImpl]:
Initialized Scheduler Signaller of type: class
org.quartz.core.SchedulerSignalerImpl
12:14:31.237 main INFO [org.quartz.core.QuartzScheduler]: Quartz
Scheduler v.1.7.3 created.
12:14:31.241 main INFO [org.quartz.simpl.RAMJobStore]:
RAMJobStore initialized.
12:14:31.242 main INFO [org.quartz.impl.StdSchedulerFactory]:
Quartz scheduler 'QuartzScheduler' initialized from an externally
provided properties instance.
12:14:31.243 main INFO [org.quartz.impl.StdSchedulerFactory]:
Quartz scheduler version: 1.7.3
12:14:31.246 main INFO [org.quartz.core.QuartzScheduler]:
Scheduler QuartzScheduler_$_NON_CLUSTERED started.
May 8, 2010 12:14:31 PM
org.opendedup.sdfs.filestore.gc.ChunkStoreGCScheduler <init>
INFO: Garbage Collection Jobs Scheduled
May 8, 2010 12:14:31 PM
org.opendedup.sdfs.filestore.gc.SDFSGCScheduler <init>
INFO: Scheduling Garbage Collection Jobs for SDFS
12:14:31.284 main INFO [org.quartz.core.QuartzScheduler]:
Scheduler QuartzScheduler_$_NON_CLUSTERED started.
May 8, 2010 12:14:31 PM
org.opendedup.sdfs.filestore.gc.SDFSGCScheduler <init>
INFO: Garbage Collection Jobs Scheduled
-f
/srv
-o
direct_io,big_writes,allow_other,fsname=deduped,nonempty
12:14:31.315 main INFO [fuse.SDFS.SDFSFileSystem]: mounting /opt/
sdfs/deduped/files to /srv
java.net.UnknownHostException: kallenxp.corp.callglobalcom.com:
kallenxp.corp.callglobalcom.com
at java.net.InetAddress.getLocalHost(InetAddress.java:1406)
at org.opendedup.util.RandomGUID.<clinit>(RandomGUID.java:34)
at fuse.SDFS.SDFSCmds.<clinit>(SDFSCmds.java:50)
at fuse.SDFS.SDFSFileSystem.<init>(SDFSFileSystem.java:51)
at fuse.SDFS.MountSDFS.main(MountSDFS.java:136)
Exception in thread "main" java.lang.UnsatisfiedLinkError: /usr/local/
src/sdfs-bin/bin/libjavafs.so: libfuse.so.2: cannot open shared object
file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1922)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1848)
at java.lang.Runtime.loadLibrary0(Runtime.java:840)
at java.lang.System.loadLibrary(System.java:1078)
at fuse.FuseMount.<clinit>(FuseMount.java:24)
at fuse.SDFS.MountSDFS.main(MountSDFS.java:135)

Sam Silverberg

unread,
May 8, 2010, 1:39:35 PM5/8/10
to dedupfilesystem-...@googlegroups.com
Stevester,
 
Thanks for the info. Let me take a look. We will be releasing a new version tomorrow or monday and I will try to get a fix in if it is needed.
 
Sam

Sam Silverberg

unread,
May 8, 2010, 1:41:27 PM5/8/10
to dedupfilesystem-...@googlegroups.com
also, it looks like you are missing libfuse.so.2 or it is not it your library path. Can you see if it is in /usr/lib/ or verify its location?

On Sat, May 8, 2010 at 10:17 AM, Stevester <sha...@firstcomm.com> wrote:

Steve Handy

unread,
May 8, 2010, 1:50:38 PM5/8/10
to dedupfilesystem-...@googlegroups.com
I don't seem to have that file after doing a find command. The file that I seem to have is libfuse.so.2.8.4

Also would you happen to have a virtual NAS appliance (64 bit) that has a virtual disk that's not 928GB large.

Thanks
Steve

From: dedupfilesystem-...@googlegroups.com <dedupfilesystem-...@googlegroups.com>
To: dedupfilesystem-...@googlegroups.com <dedupfilesystem-...@googlegroups.com>
Sent: Sat May 08 12:41:27 2010
Subject: Re: Java Issue - Opendedup on Centos 5.4

Steve Handy

unread,
May 8, 2010, 2:00:32 PM5/8/10
to dedupfilesystem-...@googlegroups.com
I will wait till you release a stable verison on either Red Hat or Centos. Honesty in the linux world, production should run on either one of those distros. The power of VMware attests to that.

Thanks
Steve

Sent: Sat May 08 12:39:35 2010

Subject: Re: Java Issue - Opendedup on Centos 5.4
Stevester,
 
Thanks for the info. Let me take a look. We will be releasing a new version tomorrow or monday and I will try to get a fix in if it is needed.
 
Sam

Sam Silverberg

unread,
May 8, 2010, 2:14:55 PM5/8/10
to dedupfilesystem-...@googlegroups.com
Stevester,
 
I agree we should have a package for RedHat and Centos. We will be releasing an RPM for Centos 5.4 next week. As far as the NAS appliance, what size would you prefer for the NAS Appliance?

Steve Handy

unread,
May 8, 2010, 2:34:31 PM5/8/10
to dedupfilesystem-...@googlegroups.com
How about a NAS appliance with a 2nd VMDK file of size 200GB. The filesystem should be XFS and the appliance 64 bit. I assume the appliance will have start up scripts, etc that will automatically set the correct Java path as well as perform the mount.sdfs command on startup or is that something the user should do?

Stevester

Sent: Sat May 08 13:14:55 2010

Sam Silverberg

unread,
May 8, 2010, 2:48:15 PM5/8/10
to dedupfilesystem-...@googlegroups.com
In the current NAS appliance there is nothing the user needs to do and I would assume the same for one based on XFS. BTW, I am curious why the current NAS appliance will not work? I know is 900 GB but its not pre-allocated and actually is about 2 GB on disk. The VM will only grow as you use the disk so it should stay fairly small unless you actually store 900 GB of data.

Steve Handy

unread,
May 8, 2010, 2:50:18 PM5/8/10
to dedupfilesystem-...@googlegroups.com
I believe its pre-allocated because when I try to import it via the vconvertor, its a fixed size requiring nearly a 1TB size.

Sent: Sat May 08 13:48:15 2010

Sam Silverberg

unread,
May 8, 2010, 2:57:26 PM5/8/10
to dedupfilesystem-...@googlegroups.com
Interresting. Its not preallocated vmware workstation. It must set it up for pre-allocation as part of the conversion process. Anyway, I will create one with a smaller (200 GB) volume based on XFS.

Steve Handy

unread,
May 8, 2010, 3:00:45 PM5/8/10
to dedupfilesystem-...@googlegroups.com
Okay. I assume it will be on Centos or Red Hat? I assume this will come early next week.

Stevester

Sent: Sat May 08 13:57:26 2010

Sam Silverberg

unread,
May 8, 2010, 3:11:19 PM5/8/10
to dedupfilesystem-...@googlegroups.com
The current NAS appliance is actually based on Ubuntu 9.10. RedHat and Centos uses fuse 2.7 which is about 1/2 as fast as fuse 2.8 for IO over NFS/CIFS. I think Centos/RH 5.5 will include fuse 2.8 but have not checked into this. There are fuse 2.8 rpms out in the wild but I am unclear on their stability.
 
I will be done with the 200 GB appliance today and upload it. The RPM packages for Centos and RedHat will be available later this week.

Steve Handy

unread,
May 8, 2010, 3:13:50 PM5/8/10
to dedupfilesystem-...@googlegroups.com
Okay. I ran fuse 2.8 (compiled, make) on Centos which is probably why I maybe experiencing issues. (Don't know).

Thanks
Steve

Sent: Sat May 08 14:11:19 2010

Sam Silverberg

unread,
May 8, 2010, 3:19:23 PM5/8/10
to dedupfilesystem-...@googlegroups.com
Could be. I bet it put libfuse.so.2 in /usr/local/lib . It should be in /usr/lib . Also make sure the host can resolve kallenxp.corp.callglobalcom.com you may want to add an entry to the hosts file for that name.

if you do

./configure --prefix=/usr
make
make install
for fuse that should put it in /usr/lib .

Steve Handy

unread,
May 8, 2010, 10:39:50 PM5/8/10
to dedupfilesystem-...@googlegroups.com

Sam: Has the edited virtual appliance been uploaded.

 

 

Thanks

Steve

Sam Silverberg

unread,
May 9, 2010, 4:40:59 AM5/9/10
to dedupfilesystem-...@googlegroups.com
Yes,

here is the link

https://docs.google.com/leaf?id=0B-MbPwBlCLzbMTBhNThkNDItNGEwYS00OWI4LTg2MmUtZjhkMzZkMjYxMzky&hl=en

Sam Silverberg

unread,
May 9, 2010, 4:43:58 AM5/9/10
to dedupfilesystem-...@googlegroups.com
The username is sdfs and the password is changeit.

Steve Handy

unread,
May 9, 2010, 10:58:58 AM5/9/10
to dedupfilesystem-...@googlegroups.com
Good Morning Everyone,

Is there a username and password for this virtual appliance? The username/password : sdfs/changeit is not working.

Thanks
Stevester

Sent: Sun May 09 03:43:58 2010

Sam Silverberg

unread,
May 9, 2010, 11:03:44 AM5/9/10
to dedupfilesystem-...@googlegroups.com
oops sorry. Its password is changeme

Steve Handy

unread,
May 9, 2010, 11:12:11 AM5/9/10
to dedupfilesystem-...@googlegroups.com
Okay. That worked. Now I can't get an ipaddress....lol.

Sent: Sun May 09 10:03:44 2010

Steve Handy

unread,
May 9, 2010, 11:51:47 AM5/9/10
to dedupfilesystem-...@googlegroups.com
Also having issues with keyboard issues too. When I push a key, it keeps registering.

Sent: Sun May 09 10:12:11 2010

Stevester

unread,
May 12, 2010, 1:15:33 PM5/12/10
to dedupfilesystem-sdfs-user-discuss

Good Day Everyone,

Is there a reasonable timetable of the RPM release of Opendedup?

Thanks
Stevester

Stevester

unread,
May 12, 2010, 1:17:38 PM5/12/10
to dedupfilesystem-sdfs-user-discuss
Sorry for the repeat question, but I didn't see my earlier question
posted on the Google User and thought no one saw this.


Thanks
Steve


Sam Silverberg

unread,
May 12, 2010, 9:46:13 PM5/12/10
to dedupfilesystem-...@googlegroups.com
Steve,

We are still working on it. We have a working Virtual Appliance but not an RPM yet.

Steve Handy

unread,
May 12, 2010, 9:49:10 PM5/12/10
to dedupfilesystem-...@googlegroups.com
Okay. Now I assume you have a Centos/Red Hat virtual appliance, is that right? If so, is there a link to it and can it have a small 2nd VMDK like before...lol. Sorry if I am asking for too much.

Stevester

Sent: Wed May 12 20:46:13 2010

Subject: Re: Java Issue - Opendedup on Centos 5.4
Reply all
Reply to author
Forward
0 new messages