Has anyone made this run with OSX?

[Christopher Schmidt]

Hi, I am executing the example at http://sploutsql.com/gettingstarted.html and getting the exception below.

I've put already all the SQLite native libs I found to the java.library.path

Any Idea?

Information: [sqlite] DB[1]: instantiated [/tmp/hadoop-FaKod/mapred/local/localRunner/FaKod/jobcache/job_local882794258_0001/attempt_local882794258_0001_r_000000_0/splout_task_attempt_local882794258_0001_r_000000_0.1]

14/02/13 07:26:44 WARN mapred.LocalJobRunner: job_local882794258_0001

java.io.IOException: com.almworks.sqlite4java.SQLiteException: [-91] cannot load library: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-0.282 in java.library.path

at com.splout.db.hadoop.TupleSQLite4JavaOutputFormat$TupleSQLRecordWriter.initSql(TupleSQLite4JavaOutputFormat.java:298)

at com.splout.db.hadoop.TupleSQLite4JavaOutputFormat$TupleSQLRecordWriter.write(TupleSQLite4JavaOutputFormat.java:313)

at com.splout.db.hadoop.TupleSQLite4JavaOutputFormat$TupleSQLRecordWriter.write(TupleSQLite4JavaOutputFormat.java:214)

at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:576)

at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)

at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)

at com.datasalt.pangool.tuplemr.TupleReducer$StaticCollector.write(TupleReducer.java:84)

at com.datasalt.pangool.tuplemr.IdentityTupleReducer.reduce(IdentityTupleReducer.java:32)

at com.datasalt.pangool.tuplemr.mapred.SimpleReducer.reduce(SimpleReducer.java:127)

at com.datasalt.pangool.tuplemr.mapred.SimpleReducer.reduce(SimpleReducer.java:43)

at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)

at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:645)

at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:405)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:445)

Caused by: com.almworks.sqlite4java.SQLiteException: [-91] cannot load library: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-0.282 in java.library.path

at com.almworks.sqlite4java.SQLite.loadLibrary(SQLite.java:97)

at com.almworks.sqlite4java.SQLiteConnection.open0(SQLiteConnection.java:1314)

at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:258)

at com.splout.db.hadoop.TupleSQLite4JavaOutputFormat$TupleSQLRecordWriter.initSql(TupleSQLite4JavaOutputFormat.java:269)

... 13 more

Caused by: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-0.282 in java.library.path

at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)

at java.lang.Runtime.loadLibrary0(Runtime.java:849)

at java.lang.System.loadLibrary(System.java:1088)

at com.almworks.sqlite4java.Internal.tryLoadFromSystemPath(Internal.java:349)

at com.almworks.sqlite4java.Internal.loadLibraryX(Internal.java:124)

at com.almworks.sqlite4java.SQLite.loadLibrary(SQLite.java:95)

... 16 more

[Iván de Prado]

Hi Christopher, 

The reason for the failure is that currently we are not distributing the native libraries for sqlite4java in the Splout distribution. But you can add them manually. Just follow the following steps:

1) Download Sqlite4Java version 282 (http://code.google.com/p/sqlite4java/downloads/detail?name=sqlite4java-282.zip) 

2) Copy all jnilib files into the native folder in the splout distribution

I have tried that in my mac and it works. 

Anyway, we don't recommend to use these native libraries for production as they will perform very slowly due to some not-well-tuned compilation parameters.

Iván

--
You received this message because you are subscribed to the Google Groups "sploutdb-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sploutdb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Iván de Prado

CEO & Co-founder

www.datasalt.com

[Pere Ferrera Bertran]

To clarify this a little bit: We do distribute native libs, but only for Linux which is the recommended production system for Splout. And these libs have been recompiled by us to enhance performance.

Maybe we should add default natives for Mac in the next release and advise not to use them in production.

[Christopher Schmidt]

Thanks for your answer.

I've done what you proposed, but still does not work for me. Same error.

Do I have to do something with my Hadoop installation?

[Iván de Prado]

Christopher, I suspect that this could be due to to use of the local Job Runner. Could you try with Hadoop installed in pseudo distributed mode?

Iván

[Christopher Schmidt]

My Hadoop (YARN) was not properly installed, obviously. I reinstalled it following this blogpost

http://www.ymc.ch/anleitung-setup-und-konfiguration-von-cdh-4-5-mit-mapreduce-2-und-yarn-auf-max-os-x

Unfortunately it's in German. After copying the native libs everything works fine.

Thanks Iván and Pere for your help...