[JIRA] (SPARKR-240) Failed with error: ‘invalid package name’ Error in as.name(name) : attempt to use zero-length variable name

25 views
Skip to first unread message

Antonio Piccolboni (JIRA)

unread,
Mar 26, 2015, 6:35:02 PM3/26/15
to spark...@googlegroups.com
Antonio Piccolboni created an issue
 
SparkR / Bug SPARKR-240
Failed with error: ‘invalid package name’ Error in as.name(name) : attempt to use zero-length variable name
Issue Type: Bug Bug
Assignee: Unassigned
Created: 26/Mar/15 3:34 PM
Priority: Major Major
Reporter: Antonio Piccolboni

Context: trying to interface SparkR with foreach. Foreach is an abstraction over several parallel backends. This would enable execution on spark cluster of > 50 R packages including very popular caret and plyr. Simple foreach examples work. caret unfortunately does not.

I have a repro but it is somewhat complex (it is the main example of model fitting in caret on their website though, not something made on purpose to make SparkR fail). If I find anything more straightforward, I will comment here. Reproduced in an R --vanilla session, but I can uninstall all of my packages, so I may have missed some deps.

Reproduce with:

install.packages(c("caret", "foreach", "devtools", "mlbench", "gbm", "survival", "splines"))

library(caret)
library(foreach)
library(devtools)

install_github("RevolutionAnalytics/doParallelSpark", subdir = "pkg")
library(doParallelSpark)
registerDoParallelSpark()
library(mlbench)
data(Sonar)

set.seed(998)
inTraining <- createDataPartition(Sonar$Class, p = .75, list = FALSE)
training <- Sonar[ inTraining,]
testing <- Sonar[-inTraining,]
fitControl <- trainControl(## 10-fold CV
method = "repeatedcv",
number = 10,

    1. repeated ten times
      repeats = 10)

set.seed(825)
gbmFit1 <- train(Class ~ ., data = training,
method = "gbm",
trControl = fitControl,

    1. This last option is actually one
    2. for gbm() that passes through
      verbose = FALSE)

Stack trace
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Error in as.name(name) : attempt to use zero-length variable name
Calls: source ... withVisible -> eval -> eval -> getNamespace -> as.name
Execution halted
15/03/26 14:32:30 ERROR Executor: Exception in task 0.0 in stage 3.0 (TID 5)
org.apache.spark.SparkException: R computation failed with
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Error in as.name(name) : attempt to use zero-length variable name
Calls: source ... withVisible -> eval -> eval -> getNamespace -> as.name
Execution halted
at edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:80)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
at edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:32)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:54)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
15/03/26 14:32:30 WARN TaskSetManager: Lost task 0.0 in stage 3.0 (TID 5, localhost): org.apache.spark.SparkException: R computation failed with
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Error in as.name(name) : attempt to use zero-length variable name
Calls: source ... withVisible -> eval -> eval -> getNamespace -> as.name
Execution halted
edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:80)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:32)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
org.apache.spark.scheduler.Task.run(Task.scala:54)
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
15/03/26 14:32:30 ERROR TaskSetManager: Task 0 in stage 3.0 failed 1 times; aborting job
collect on 33 failed with java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at edu.berkeley.cs.amplab.sparkr.SparkRBackendHandler.handleMethodCall(SparkRBackendHandler.scala:111)
at edu.berkeley.cs.amplab.sparkr.SparkRBackendHandler.channelRead0(SparkRBackendHandler.scala:58)
at edu.berkeley.cs.amplab.sparkr.SparkRBackendHandler.channelRead0(SparkRBackendHandler.scala:19)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 3.0 failed 1 times, most recent failure: Lost task 0.0 in stage 3.0 (TID 5, localhost): org.apache.spark.SparkException: R computation failed with
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Failed with error: ‘invalid package name’
Error in as.name(name) : attempt to use zero-length variable name
Calls: source ... withVisible -> eval -> eval -> getNamespace -> as.name
Execution halted
edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:80)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:32)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
org.apache.spark.scheduler.Task.run(Task.scala:54)
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688)
at org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1391)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
at akka.actor.ActorCell.invoke(ActorCell.scala:456)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Error: returnStatus == 0 is not TRUE

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4-OD-16-005#64014-sha1:3f74600)
Atlassian logo

Shivaram Venkataraman (JIRA)

unread,
Apr 9, 2015, 7:16:01 PM4/9/15
to spark...@googlegroups.com
Shivaram Venkataraman resolved as Duplicate
Change By: Shivaram Venkataraman
Resolution: Duplicate
Status: Open Resolved
This message was sent by Atlassian JIRA (v6.4-OD-16-006#64014-sha1:e47e3fa)
Atlassian logo

Shivaram Venkataraman (JIRA)

unread,
Apr 9, 2015, 7:16:01 PM4/9/15
to spark...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages