Fw: Re: [Kosmosfs-users] Unable to load kfs_access native library in Hadoop

26 views
Skip to first unread message

Tung Nguyen

unread,
May 9, 2011, 4:43:28 PM5/9/11
to kfs-...@googlegroups.com
Hi,
Since we moved to google code. I forwarded my question here.

TT

--- On Mon, 5/9/11, Tung Nguyen <beta...@yahoo.com> wrote:

From: Tung Nguyen <beta...@yahoo.com>
Subject: Re: [Kosmosfs-users] Unable to load kfs_access native library in Hadoop
To: "Sriram Rao" <srira...@gmail.com>
Cc: "kosmosf...@lists.sourceforge.net" <kosmosf...@lists.sourceforge.net>
Date: Monday, May 9, 2011, 10:26 AM

Hi Sriram,
I copied all file in the kfs/build/lib dir (except the static folder) and issued 
#bin/hadoop fs -fs kfs://localhost:20000 -ls /
I got
ls: Can not create a Path from an empty string
Usage: java FsShell [-ls <path>]

But I did have kfs running:
 ./kfsshell -s localhost -p 20000
KfsShell> ls
dumpster
test
KfsShell>
TT


--- On Mon, 5/9/11, Sriram Rao <srira...@gmail.com> wrote:

From: Sriram Rao <srira...@gmail.com>
Subject: Re: [Kosmosfs-users] Unable to load kfs_access native library in Hadoop
To: "Tung Nguyen" <beta...@yahoo.com>
Cc: "kosmosf...@lists.sourceforge.net" <kosmosf...@lists.sourceforge.net>
Date: Monday, May 9, 2011, 12:42 AM

You are missing libkfs_access.so

can you copy this file over?


Sriram

On Sunday, May 8, 2011, Tung Nguyen <beta...@yahoo.com> wrote:
> Hi Sriram,
> Thank you for your reply. Actually, in the  http://sourceforge.net/apps/trac/kosmosfs/wiki/UsingWithHadoop, there's an instruction about LD_LIBRARY_PATH. I followed that so I have LD_LIBRARY_PATH point to kfs/lib.I also exported it out$ echo $LD_LIBRARY_PATH                            /home/mine/kfs-0.5/build/lib
> Anyway, I tried to copy that lib to the Hadoop native lib directory: ls lib/native/Linux-amd64-64/libhadoop.a   libhadoop.so    libhadoop.so.1.0.0libhadoop.la  libhadoop.so.1  libkfsClient.so
> I still got the same error: java.lang.UnsatisfiedLinkError: no kfs_access in java.library.path        at
>  java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)        at java.lang.Runtime.loadLibrary0(Runtime.java:823)        at java.lang.System.loadLibrary(System.java:1028)        at org.kosmix.kosmosfs.access.KfsAccess.<clinit>(KfsAccess.java:103)....ThanksTT
>
> --- On Sun, 5/8/11, Sriram Rao <srira...@gmail.com> wrote:
>
> From: Sriram Rao <srira...@gmail.com>
> Subject: Re: [Kosmosfs-users] Unable to load kfs_access native library in Hadoop
> To: "Tung Nguyen" <beta...@yahoo.com>
> Cc: kosmosf...@lists.sourceforge.net
> Date: Sunday, May 8, 2011, 2:46 PM
>
>
> The bit you missed:
> Update the LD_LIBRARY_PATH environment variable so that libkfsClient.so can be loaded
>
> The easiest way to fix this is to copy the libkfsaccess.so from build/lib
>  to: /home/mine/hadoop-0.20.2/bin/../lib/native/Linux-amd64-64
>
>
> Sriram
>
>
> On Sun, May 8, 2011 at 8:22 AM, Tung Nguyen <beta...@yahoo.com> wrote:
>
> Hi,I tried to use Hadoop with KFS. I followed the instruction at http://sourceforge.net/apps/trac/kosmosfs/wiki/UsingWithHadoop
> but when I issued #bin/hadoop fs -fs kfs://localhost:20000 -ls /I got the following error. I used hadoop-0.20.0 and KFS-0.5
> java.lang.UnsatisfiedLinkError: no kfs_access in java.library.path
>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)        at java.lang.Runtime.loadLibrary0(Runtime.java:823)        at java.lang.System.loadLibrary(System.java:1028)
>         at org.kosmix.kosmosfs.access.KfsAccess.<clinit>(KfsAccess.java:103)
>        at org.apache.hadoop.fs.kfs.KFSImpl.<init>(KFSImpl.java:45)
>        at org.apache.hadoop.fs.kfs.KosmosFileSystem.initialize(KosmosFileSystem.java:70)        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)        at org.apache.hadoop.fs.FsShell.init(FsShell.java:82)        at org.apache.hadoop.fs.FsShell.run(FsShell.java:1731)
>         at
>  org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)        at
>  org.apache.hadoop.fs.FsShell.main(FsShell.java:1880)
> Unable to load kfs_access native library: /home/mine/hadoop-0.20.2/bin/../lib/native/Linux-amd64-64
> I am sure that KFS and Hadoop are working fine. I used kfsshell to create dir, copy file to kfs and hdfs without any problem.
> In the  /home/mine/hadoop-0.20.2/lib/native/Linux-amd64-64 dir I have:-rw-r--r-- 1 mine mine 115416 2010-02-19 03:07 libhadoop.a-rw-r--r-- 1 mine mine    904 2010-02-19 03:07 libhadoop.la
> -rw-r--r-- 1 mine mine  75669 2010-02-19 03:07 libhadoop.so-rw-r--r-- 1 mine mine  75669 2010-02-19 03:07 libhadoop.so.1-rw-r--r-- 1 mine mine  75669 2010-02-19 03:07
>  libhadoop.so.1.0.0
> Please help me out.Thank youTT
> ------------------------------------------------------------------------------
>
> WhatsUp Gold - Download Free Network Management Software
>
> The most intuitive, comprehensive, and cost-effective
>  network
>
> management toolset available today.  Delivers lowest initial
>
> acquisition cost and overall TCO of any competing solution.
>
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
>
> Kosmosfs-users mailing list
>
> Kosmosf...@lists.sourceforge.net
>
> https://lists.sourceforge.net/lists/listinfo/kosmosfs-users
>
>
>
>
>

-----Inline Attachment Follows-----

------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today.  Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd

-----Inline Attachment Follows-----

_______________________________________________
Kosmosfs-users mailing list
Kosmosf...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kosmosfs-users

Stas Oskin

unread,
May 10, 2011, 4:01:56 PM5/10/11
to kfs-...@googlegroups.com
Hi.

A bit not related, but how I can unsubscribe from this list?

Thanks.

Tung Nguyen

unread,
May 15, 2011, 11:38:47 PM5/15/11
to kfs-...@googlegroups.com
Hi Sriram,
My bad. I forgot. Here, I forward my email to the google group.
Thanks
TT


--- On Sun, 5/15/11, Sriram Rao <srira...@gmail.com> wrote:

From: Sriram Rao <srira...@gmail.com>
Subject: Re: [Kosmosfs-users] Unable to load kfs_access native library in Hadoop
To: "Tung Nguyen" <beta...@yahoo.com>
Cc: "kosmosf...@lists.sourceforge.net" <kosmosf...@lists.sourceforge.net>
Date: Sunday, May 15, 2011, 11:30 PM


Awesome.  Thanks.  Can you please use the google-groups for this project?

Thanks.

Sriram

On Sun, May 15, 2011 at 3:34 PM, Tung Nguyen <beta...@yahoo.com> wrote:
Hi all,
I solved the problem. I forgot to replace the old version of kfs jar in the lib directory of hadoop. Basically, the kfs in hadoop is the old version (0.2.2). I used the new version of KFS. I hope this is helpful for someone who experiences the same problem.
Thanks
Reply all
Reply to author
Forward
0 new messages