pio train error related to spark

[Ilsa Baqai]

Hello,

I've deployed a template which built against an application successfully but when I run command to train the engine I get the following error. The error log is:

Exception in thread "main" java.lang.IllegalArgumentException: java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.2-7318f37a-3c9e-4b73-b087-480420806012-libsnappyjava.so: /tmp/snappy-1.1.2-7318f37a-3c9e-4b73-b087-480420806012-libsnappyjava.so: failed to map segment from shared object: Operation not permitted
        at org.apache.spark.io.SnappyCompressionCodec$.liftedTree1$1(CompressionCodec.scala:171)
        at org.apache.spark.io.SnappyCompressionCodec$.org$apache$spark$io$SnappyCompressionCodec$$version$lzycompute(CompressionCodec.scala:168)
        at org.apache.spark.io.SnappyCompressionCodec$.org$apache$spark$io$SnappyCompressionCodec$$version(CompressionCodec.scala:168)
        at org.apache.spark.io.SnappyCompressionCodec.<init>(CompressionCodec.scala:152)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at org.apache.spark.io.CompressionCodec$.createCodec(CompressionCodec.scala:72)
        at org.apache.spark.io.CompressionCodec$.createCodec(CompressionCodec.scala:65)
        at org.apache.spark.broadcast.TorrentBroadcast.org$apache$spark$broadcast$TorrentBroadcast$$setConf(TorrentBroadcast.scala:73)
        at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:80)
        at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
        at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:63)
        at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1326)
        at org.apache.spark.rdd.NewHadoopRDD.<init>(NewHadoopRDD.scala:75)
        at org.apache.spark.SparkContext$$anonfun$newAPIHadoopRDD$1.apply(SparkContext.scala:1134)
        at org.apache.spark.SparkContext$$anonfun$newAPIHadoopRDD$1.apply(SparkContext.scala:1129)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
        at org.apache.spark.SparkContext.withScope(SparkContext.scala:714)
        at org.apache.spark.SparkContext.newAPIHadoopRDD(SparkContext.scala:1129)
        at io.prediction.data.storage.hbase.HBPEvents.find(HBPEvents.scala:83)
        at io.prediction.data.store.PEventStore$.find(PEventStore.scala:80)
        at famglobal.DataSource.getRatings(DataSource.scala:35)
        at famglobal.DataSource.readTraining(DataSource.scala:62)
        at famglobal.DataSource.readTraining(DataSource.scala:22)
        at io.prediction.controller.PDataSource.readTrainingBase(PDataSource.scala:37)
        at io.prediction.controller.Engine$.train(Engine.scala:641)
        at io.prediction.controller.Engine.train(Engine.scala:174)
        at io.prediction.workflow.CoreWorkflow$.runTrain(CoreWorkflow.scala:65)
        at io.prediction.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)
        at io.prediction.workflow.CreateWorkflow.main(CreateWorkflow.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.2-7318f37a-3c9e-4b73-b087-480420806012-libsnappyjava.so: /tmp/snappy-1.1.2-7318f37a-3c9e-4b73-b087-480420806012-libsnappyjava.so: failed to map segment from shared object: Operation not permitted
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(Unknown Source)
        at java.lang.ClassLoader.loadLibrary(Unknown Source)
        at java.lang.Runtime.load0(Unknown Source)
        at java.lang.System.load(Unknown Source)
        at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:174)
        at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:152)
        at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
        at org.apache.spark.io.SnappyCompressionCodec$.liftedTree1$1(CompressionCodec.scala:169)
        ... 41 more


can anyone please help me with this? Thanks :)

[Pat Ferrel]

Do you have snappy installed? What OS are you using?

--
You received this message because you are subscribed to the Google Groups "actionml-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to actionml-use...@googlegroups.com.
To post to this group, send email to action...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/246f7c95-fc47-42fa-bc2a-4d3bcd44ef16%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ilsa Baqai]

Thanks for the response, Pat.

I have a Linux os. CentOS release 6.8.
I never installed snappy. is it necessary to have snappy installed?

Thanks.

[Pat Ferrel]

Try the new Apache PIO mailing list. Here’s how to singup: http://predictionio.incubator.apache.org/support/

Apache support is via the user mailing list.

To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/e2837649-d9ee-4471-a05b-5ecfe0b9f92c%40googlegroups.com.