Error: java.lang.AssertionError: assertion failed: curTarget 0 is out of range 1 to 3
379 views
Skip to first unread message
tf...@foxmailbox.com
Nov 2, 2017, 8:51:39 PM11/2/17
to BigDL User Group
I'm a little confused as to what this error means:
Any ideas? Is it saying that labels cannot be value 0? I checked that and changed all my labels to nonzero values, but it didn't help the problem.
--------------------------------------------------------------------------- Py4JJavaError Traceback (most recent call last) <ipython-input-63-c648da557482> in <module>() ----> 1 get_ipython().run_cell_magic(u'time', u'', u'# Boot training process\ntrained_model = optimizer.optimize()\nprint("Optimization Done.")') /opt/conda/envs/py27/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_cell_magic(self, magic_name, line, cell) 2115 magic_arg_s = self.var_expand(line, stack_depth) 2116 with self.builtin_trap: -> 2117 result = fn(magic_arg_s, cell) 2118 return result 2119 <decorator-gen-60> in time(self, line, cell, local_ns) /opt/conda/envs/py27/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k) 186 # but it's overkill for just that one bit of state. 187 def magic_deco(arg): --> 188 call = lambda f, *a, **k: f(*a, **k) 189 190 if callable(arg): /opt/conda/envs/py27/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in time(self, line, cell, local_ns) 1191 else: 1192 st = clock2() -> 1193 exec(code, glob, local_ns) 1194 end = clock2() 1195 out = None <timed exec> in <module>() /tmp/spark-5f2f73e5-c767-4349-8b52-7bb05550d453/userFiles-f53f07c6-8e3a-4459-bb23-a854d1beebc0/bigdl-0.3.0-SNAPSHOT-python-api.zip/bigdl/optim/optimizer.py in optimize(self) 596 Do an optimization. 597 """ --> 598 jmodel = callJavaFunc(get_spark_context(), self.value.optimize) 599 from bigdl.nn.layer import Layer 600 return Layer.of(jmodel) /tmp/spark-5f2f73e5-c767-4349-8b52-7bb05550d453/userFiles-f53f07c6-8e3a-4459-bb23-a854d1beebc0/bigdl-0.3.0-SNAPSHOT-python-api.zip/bigdl/util/common.py in callJavaFunc(sc, func, *args) 487 """ Call Java Function """ 488 args = [_py2java(sc, a) for a in args] --> 489 result = func(*args) 490 return _java2py(sc, result) 491 /usr/local/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py in __call__(self, *args) 1131 answer = self.gateway_client.send_command(command) 1132 return_value = get_return_value( -> 1133 answer, self.gateway_client, self.target_id, self.name) 1134 1135 for temp_arg in temp_args: /usr/local/spark/python/pyspark/sql/utils.py in deco(*a, **kw) 61 def deco(*a, **kw): 62 try: ---> 63 return f(*a, **kw) 64 except py4j.protocol.Py4JJavaError as e: 65 s = e.java_exception.toString() /usr/local/spark/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name) 317 raise Py4JJavaError( 318 "An error occurred while calling {0}{1}{2}.\n". --> 319 format(target_id, ".", name), value) 320 else: 321 raise Py4JError( Py4JJavaError: An error occurred while calling o2050.optimize. : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 184.0 failed 1 times, most recent failure: Lost task 0.0 in stage 184.0 (TID 119, localhost, executor driver): java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: Boxed Error at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$11.apply(DistriOptimizer.scala:235) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$11.apply(DistriOptimizer.scala:235) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8.apply(DistriOptimizer.scala:235) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8.apply(DistriOptimizer.scala:175) at org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:108) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: Boxed Error at scala.concurrent.impl.Promise$.resolver(Promise.scala:55) at scala.concurrent.impl.Promise$.scala$concurrent$impl$Promise$$resolveTry(Promise.scala:47) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:244) at scala.concurrent.Promise$class.complete(Promise.scala:55) at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:153) at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:23) at com.intel.analytics.bigdl.utils.ThreadPool$$anon$1.execute(ThreadPool.scala:203) at scala.concurrent.impl.Future$.apply(Future.scala:31) at scala.concurrent.Future$.apply(Future.scala:494) at com.intel.analytics.bigdl.utils.ThreadPool.invoke(ThreadPool.scala:168) at com.intel.analytics.bigdl.nn.ClassNLLCriterion.updateOutput(ClassNLLCriterion.scala:107) at com.intel.analytics.bigdl.nn.ClassNLLCriterion.updateOutput(ClassNLLCriterion.scala:60) at com.intel.analytics.bigdl.nn.abstractnn.AbstractCriterion.forward(AbstractCriterion.scala:70) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$9$$anonfun$apply$2.apply$mcI$sp(DistriOptimizer.scala:224) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$9$$anonfun$apply$2.apply(DistriOptimizer.scala:216) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$9$$anonfun$apply$2.apply(DistriOptimizer.scala:216) at com.intel.analytics.bigdl.utils.ThreadPool$$anonfun$1$$anon$4.call(ThreadPool.scala:112) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: java.lang.AssertionError: assertion failed: curTarget 0 is out of range 1 to 3 at scala.Predef$.assert(Predef.scala:170) at com.intel.analytics.bigdl.nn.ClassNLLCriterion$$anonfun$updateOutput$5.apply(ClassNLLCriterion.scala:109) at com.intel.analytics.bigdl.nn.ClassNLLCriterion$$anonfun$updateOutput$5.apply(ClassNLLCriterion.scala:107) at com.intel.analytics.bigdl.utils.ThreadPool$$anonfun$invoke$2.apply(ThreadPool.scala:162) at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ... 15 more Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1499) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1487) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1486) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1486) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:814) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:814) at scala.Option.foreach(Option.scala:257) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:814) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1714) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1669) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1658) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:630) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2022) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2119) at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:1026) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.rdd.RDD.withScope(RDD.scala:362) at org.apache.spark.rdd.RDD.reduce(RDD.scala:1008) at com.intel.analytics.bigdl.optim.DistriOptimizer$.optimize(DistriOptimizer.scala:285) at com.intel.analytics.bigdl.optim.DistriOptimizer.optimize(DistriOptimizer.scala:795) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: Boxed Error at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$11.apply(DistriOptimizer.scala:235) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$11.apply(DistriOptimizer.scala:235) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8.apply(DistriOptimizer.scala:235) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8.apply(DistriOptimizer.scala:175) at org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:108) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: java.util.concurrent.ExecutionException: Boxed Error at scala.concurrent.impl.Promise$.resolver(Promise.scala:55) at scala.concurrent.impl.Promise$.scala$concurrent$impl$Promise$$resolveTry(Promise.scala:47) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:244) at scala.concurrent.Promise$class.complete(Promise.scala:55) at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:153) at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:23) at com.intel.analytics.bigdl.utils.ThreadPool$$anon$1.execute(ThreadPool.scala:203) at scala.concurrent.impl.Future$.apply(Future.scala:31) at scala.concurrent.Future$.apply(Future.scala:494) at com.intel.analytics.bigdl.utils.ThreadPool.invoke(ThreadPool.scala:168) at com.intel.analytics.bigdl.nn.ClassNLLCriterion.updateOutput(ClassNLLCriterion.scala:107) at com.intel.analytics.bigdl.nn.ClassNLLCriterion.updateOutput(ClassNLLCriterion.scala:60) at com.intel.analytics.bigdl.nn.abstractnn.AbstractCriterion.forward(AbstractCriterion.scala:70) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$9$$anonfun$apply$2.apply$mcI$sp(DistriOptimizer.scala:224) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$9$$anonfun$apply$2.apply(DistriOptimizer.scala:216) at com.intel.analytics.bigdl.optim.DistriOptimizer$$anonfun$8$$anonfun$9$$anonfun$apply$2.apply(DistriOptimizer.scala:216) at com.intel.analytics.bigdl.utils.ThreadPool$$anonfun$1$$anon$4.call(ThreadPool.scala:112) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: java.lang.AssertionError: assertion failed: curTarget 0 is out of range 1 to 3 at scala.Predef$.assert(Predef.scala:170) at com.intel.analytics.bigdl.nn.ClassNLLCriterion$$anonfun$updateOutput$5.apply(ClassNLLCriterion.scala:109) at com.intel.analytics.bigdl.nn.ClassNLLCriterion$$anonfun$updateOutput$5.apply(ClassNLLCriterion.scala:107) at com.intel.analytics.bigdl.utils.ThreadPool$$anonfun$invoke$2.apply(ThreadPool.scala:162) at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ... 15 more
Yiheng Wang
Nov 2, 2017, 9:45:39 PM11/2/17
to tf...@foxmailbox.com, BigDL User Group
Hi Tim
Yes. The root cause should be some label data is 0.
Can you post the code of preparing data and the csv file? I see some code in another thread but it looks not right.
Regards,
--
You received this message because you are subscribed to the Google Groups "BigDL User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigdl-user-group+unsubscribe@googlegroups.com.
To post to this group, send email to bigdl-user-group@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigdl-user-group/04be98ed-8d23-4eba-952f-743a610960c5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
tf...@foxmailbox.com
Nov 2, 2017, 10:40:57 PM11/2/17
to BigDL User Group
%matplotlib inline
import pandas
import datetime as dt
from bigdl.dataset.transformer import *
from bigdl.dataset.base import *
from bigdl.nn.layer import *
from bigdl.nn.criterion import *
from bigdl.optim.optimizer import *
from bigdl.util.common import *
from utils import *
from pyspark.sql.functions import col
init_engine()
learning_rate = 0.2
training_epochs = 15
batch_size = 16
display_step = 1
# Network Parameters
n_input = 4
n_classes = 3
n_hidden_1 = 10 # 1st layer number of features
n_hidden_2 = 10 # 2nd layer number of features
iris_training = spark.read.csv("../data/iris2/iris_training.csv", header=True, inferSchema="true", mode="DROPMALFORMED")
iris_test = spark.read.csv("../data/iris2/iris_test.csv", header=True, inferSchema="true", mode="DROPMALFORMED")
iris_training = iris_training.select([col(c).cast("double") for c in iris_training.columns])
iris_test = iris_test.select([col(c).cast("double") for c in iris_test.columns])
iris_test.filter('label == 0').show()
iris_k_train = iris_training.rdd.map(list)
iris_k_test = iris_test.rdd.map(list)
iris_k_train.take(29)
#convert data into RDD[Sample]
def array2rdd(ds):
#build Sample from ndarrays
def build_sample(c0,c1,c2,c3,prediction):
feature = np.array([c0,c1, c2, c3]).flatten()
label = np.array(prediction)
return Sample.from_ndarray(feature, label)
rdd = ds.map(lambda (c0,c1,c2,c3,prediction): build_sample(c0,c1,c2,c3,prediction))
return rdd
iris_rdd_train = array2rdd(iris_k_train)
iris_rdd_train.cache()
iris_rdd_train.count()
iris_rdd_test = array2rdd(iris_k_test)
iris_rdd_test.cache()
iris_rdd_test.count()
# Create model
def multilayer_perceptron(n_hidden_1, n_hidden_2, n_input, n_classes):
# Initialize a sequential container
model = Sequential()
# Hidden layer with ReLu activation
#model.add(Reshape([28*28]))
model.add(Linear(n_input, n_hidden_1).set_name('mlp_fc1'))
model.add(ReLU())
# Hidden layer with ReLu activation
model.add(Linear(n_hidden_1, n_hidden_2).set_name('mlp_fc2'))
model.add(ReLU())
# output layer
model.add(Linear(n_hidden_2, n_classes).set_name('mlp_fc3'))
model.add(LogSoftMax())
return model
model = multilayer_perceptron(n_hidden_1, n_hidden_2, n_input, n_classes)
# Create an Optimizer
optimizer = Optimizer(
model=model,
training_rdd=iris_rdd_train,
criterion=ClassNLLCriterion(),
optim_method=SGD(learningrate=learning_rate),
end_trigger=MaxEpoch(training_epochs),
batch_size=batch_size)
# Set the validation logic
optimizer.set_validation(
batch_size=batch_size,
val_rdd=iris_rdd_test,
trigger=EveryEpoch(),
val_method=[Top1Accuracy()]
)
app_name='multilayer_perceptron-'+dt.datetime.now().strftime("%Y%m%d-%H%M%S")
train_summary = TrainSummary(log_dir='/tmp/bigdl_summaries',
app_name=app_name)
train_summary.set_summary_trigger("Parameters", SeveralIteration(50))
val_summary = ValidationSummary(log_dir='/tmp/bigdl_summaries',
app_name=app_name)
optimizer.set_train_summary(train_summary)
optimizer.set_val_summary(val_summary)
print("saving logs to ",app_name)
%%time
# Boot training process
trained_model = optimizer.optimize()
print("Optimization Done.")
Li, Zhichao
Nov 3, 2017, 12:42:25 AM11/3/17
to tf...@foxmailbox.com, BigDL User Group
BigDL follow the convention of Torch that target is one-base index, so you need to add one to the label.
--
You received this message because you are subscribed to the Google Groups "BigDL User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
bigdl-user-gro...@googlegroups.com.
To post to this group, send email to
bigdl-us...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/bigdl-user-group/83cff00d-6262-46f0-a6a5-38f5e8ef041f%40googlegroups.com.
Yiheng Wang
Nov 3, 2017, 2:29:58 AM11/3/17
to Li, Zhichao, tf...@foxmailbox.com, BigDL User Group
I look through the data. The 64th row label is 'r', which is Nan...
To unsubscribe from this group and stop receiving emails from it, send an email to bigdl-user-group+unsubscribe@googlegroups.com.
To post to this group, send email to bigdl-user-group@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigdl-user-group/83cff00d-6262-46f0-a6a5-38f5e8ef041f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "BigDL User Group" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigdl-user-group/F03D1005CC201743BE1B3E1C1E14668B6AC73A6E%40SHSMSX101.ccr.corp.intel.com.To unsubscribe from this group and stop receiving emails from it, send an email to bigdl-user-group+unsubscribe@googlegroups.com.
To post to this group, send email to bigdl-user-group@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages