Mesos support?
266 views
Skip to first unread message
Jérémy Subtil
Dec 3, 2014, 11:02:38 AM12/3/14
to zeppelin-...@googlegroups.com
Hi there,
I wish to run Zeppelin on my Mesos cluster. Is it possible as of version 0.4.0?
Would I have to set any other variable than MASTER?
Cheers,
Jeremy Subtil
I wish to run Zeppelin on my Mesos cluster. Is it possible as of version 0.4.0?
Would I have to set any other variable than MASTER?
Cheers,
Jeremy Subtil
moon soo Lee
Dec 6, 2014, 10:03:38 PM12/6/14
to zeppelin-...@googlegroups.com
Hi, Jeremy Subtil
I highly recommend to use latest master branch than prebuilt binary 0.4.0 version. 0.4.0 is really old.
I'm not sure with Mesos, but basically, Zeppelin creates SparkContext in the same way Spark-Shell does. So any JVM property for Spark would work for Zeppelin, too.
Thanks,
moon
Jérémy Subtil
Jan 13, 2015, 11:31:21 AM1/13/15
to moon soo Lee, zeppelin-...@googlegroups.com
Hi,
Thanks for your confirmation moon. By setting the MESOS_NATIVE_LIBRARY and SPARK_EXECUTOR_URI env variables I managed to register Zeppelin as a framework on my Mesos cluster.
However, while running a Scala command is just fine (e.g. Seq(1,2,3)), running a Spark command (e.g. sc.parallelize(Seq(1,2,3)).count()) fails. The tasks are submitted to the Mesos cluster and then I get the following log message: ({MesosSchedulerBackend driver} Logging.scala[logInfo]:59) - driver.run() returned with code DRIVER_ABORTED
Any ideas? Should I fill a bug report?
Running a vanilla Spark Shell on my Mesos cluster with the same env variables works fine.
Full log:
WARN [2015-01-13 16:03:56,332] ({main} ZeppelinConfiguration.java[create]:77) - Failed to load configuration, proceeding with a default
INFO [2015-01-13 16:03:56,491] ({main} NotebookServer.java[creatingwebSocketServerLog]:40) - Create zeppelin websocket on port 8091
INFO [2015-01-13 16:03:56,783] ({main} ZeppelinServer.java[main]:82) - Start zeppelin server
INFO [2015-01-13 16:03:56,785] ({main} Server.java[doStart]:272) - jetty-8.1.14.v20131031
INFO [2015-01-13 16:03:57,226] ({main} InterpreterFactory.java[init]:57) - Reading /opt/zeppelin/interpreter/spark
INFO [2015-01-13 16:03:57,342] ({main} InterpreterFactory.java[init]:72) - Interpreter spark found. class=com.nflabs.zeppelin.spark.SparkInterpreter
INFO [2015-01-13 16:03:57,344] ({main} InterpreterFactory.java[init]:72) - Interpreter sql found. class=com.nflabs.zeppelin.spark.SparkSqlInterpreter
INFO [2015-01-13 16:03:57,356] ({main} InterpreterFactory.java[init]:57) - Reading /opt/zeppelin/interpreter/sh
INFO [2015-01-13 16:03:57,363] ({main} InterpreterFactory.java[init]:72) - Interpreter sh found. class=com.nflabs.zeppelin.shell.ShellInterpreter
INFO [2015-01-13 16:03:57,363] ({main} InterpreterFactory.java[init]:57) - Reading /opt/zeppelin/interpreter/md
INFO [2015-01-13 16:03:57,368] ({main} InterpreterFactory.java[init]:72) - Interpreter md found. class=com.nflabs.zeppelin.markdown.Markdown
INFO [2015-01-13 16:03:57,410] ({main} StdSchedulerFactory.java[instantiate]:1184) - Using default implementation for ThreadExecutor
INFO [2015-01-13 16:03:57,414] ({main} SimpleThreadPool.java[initialize]:268) - Job execution threads will use class loader of thread: main
INFO [2015-01-13 16:03:57,433] ({main} SchedulerSignalerImpl.java[<init>]:61) - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
INFO [2015-01-13 16:03:57,434] ({main} QuartzScheduler.java[<init>]:240) - Quartz Scheduler v.2.2.1 created.
INFO [2015-01-13 16:03:57,435] ({main} RAMJobStore.java[initialize]:155) - RAMJobStore initialized.
INFO [2015-01-13 16:03:57,439] ({main} QuartzScheduler.java[initialize]:305) - Scheduler meta-data: Quartz Scheduler (v2.2.1) 'DefaultQuartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.
INFO [2015-01-13 16:03:57,439] ({main} StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package:
'quartz.properties'
INFO [2015-01-13 16:03:57,439] ({main} StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version: 2.2.1
INFO [2015-01-13 16:03:57,440] ({main} QuartzScheduler.java[start]:575) - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
INFO [2015-01-13 16:03:57,443] ({main} Notebook.java[loadAllNotes]:113) - Loading note from 2A9X1XSSN
INFO [2015-01-13 16:03:57,448] ({main} Note.java[load]:284) - Load note 2A9X1XSSN from /var/lib/zeppelin/notebook/2A9X1XSSN/note.json
INFO [2015-01-13 16:03:57,669] ({main} ServerImpl.java[initDestination]:94) - Setting the server's publish address to be /
INFO [2015-01-13 16:03:57,830] ({main} StandardDescriptorProcessor.java[visitServlet]:284) - NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
INFO [2015-01-13 16:03:58,798] ({main} StandardDescriptorProcessor.java[visitServlet]:284) - NO JSP Support for /docs, did not find org.apache.jasper.servlet.JspServlet
INFO [2015-01-13 16:03:58,810] ({main} AbstractConnector.java[doStart]:338) - Started SocketC...@0.0.0.0:8090
INFO [2015-01-13 16:03:58,810] ({main} ZeppelinServer.java[main]:84) - Started
INFO [2015-01-13 16:04:07,795] ({WebSocketWorker-8} NotebookServer.java[onOpen]:63) - New connection from localhost.localdomain : 38142
INFO [2015-01-13 16:04:07,835] ({WebSocketWorker-8} NotebookServer.java[onMessage]:75) - RECEIVE << LIST_NOTES
INFO [2015-01-13 16:04:07,836] ({WebSocketWorker-8} Notebook.java[getAllNotes]:128) - 1
INFO [2015-01-13 16:04:07,882] ({WebSocketWorker-8} NotebookServer.java[onMessage]:75) - RECEIVE << GET_NOTE
INFO [2015-01-13 16:04:11,078] ({WebSocketWorker-8} NotebookServer.java[onMessage]:75) - RECEIVE << RUN_PARAGRAPH
INFO [2015-01-13 16:04:11,081] ({WebSocketWorker-8} Note.java[persist]:262) - Persist note 2A9X1XSSN into /var/lib/zeppelin/notebook/2A9X1XSSN/note.json
INFO [2015-01-13 16:04:11,089] ({WebSocketWorker-8} NotebookServer.java[broadcast]:204) - SEND >> NOTE
INFO [2015-01-13 16:04:11,093] ({WebSocketWorker-8} InterpreterFactory.java[createRepl]:97) - find repl class spark = com.nflabs.zeppelin.spark.SparkInterpreter
INFO [2015-01-13 16:04:11,093] ({WebSocketWorker-8} InterpreterFactory.java[createRepl]:105) - Create repl com.nflabs.zeppelin.spark.SparkInterpreter from spark
INFO [2015-01-13 16:04:11,095] ({WebSocketWorker-8} NotebookServer.java[broadcast]:204) - SEND >> NOTE
INFO [2015-01-13 16:04:11,102] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:94) - Job paragraph_1418045534609_-1316659758 started by scheduler com.nflabs.zeppelin.spark.SparkInterpreter1457599292
INFO [2015-01-13 16:04:11,103] ({pool-2-thread-2} NotebookServer.java[broadcast]:204) - SEND >> NOTE
INFO [2015-01-13 16:04:11,108] ({pool-2-thread-2} Paragraph.java[jobRun]:164) - run paragraph 20141208-133214_32958749 using null com.nflabs.zeppelin.interpreter.LazyOpenInterpreter@7532157
INFO [2015-01-13 16:04:12,419] ({Thread-25} Logging.scala[logInfo]:59) - Changing view acls to: root
INFO [2015-01-13 16:04:12,420] ({Thread-25} Logging.scala[logInfo]:59) - Changing modify acls to: root
INFO [2015-01-13 16:04:12,421] ({Thread-25} Logging.scala[logInfo]:59) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
INFO [2015-01-13 16:04:12,422] ({Thread-25} Logging.scala[logInfo]:59) - Starting HTTP Server
INFO [2015-01-13 16:04:12,427] ({Thread-25} Server.java[doStart]:272) - jetty-8.1.14.v20131031
INFO [2015-01-13 16:04:12,430] ({Thread-25} AbstractConnector.java[doStart]:338) - Started SocketC...@0.0.0.0:33340
INFO [2015-01-13 16:04:12,432] ({Thread-25} Logging.scala[logInfo]:59) - Successfully started service 'HTTP class server' on port 33340.
WARN [2015-01-13 16:04:15,850] ({Thread-25} Logging.scala[logWarning]:71) -
SPARK_CLASSPATH was detected (set to ':/opt/zeppelin/zeppelin-server/target/lib/log4j-1.2.17.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-configuration-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/xmlschema-core-2.0.3.jar:/opt/zeppelin/zeppelin-server/target/lib/quartz-2.2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.el-2.2.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/mimepull-1.6.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.activation-1.1.0.v201105071233.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-bindings-xml-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-client-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-server-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-jaxrs_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/json4s-native_2.10-3.2.4.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-annotations-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/jetty-jsp-8.1.14.v20131031.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-jersey-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/zeppelin-zengine-0.5.0-SNAPSHOT.jar:/opt/zeppelin/zeppelin-server/target/lib/jetty-all-server-8.1.14.v20131031.jar:/opt/zeppelin/zeppelin-server/target/lib/joda-time-2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-runtime-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-logging-1.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-jersey-jaxrs_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/json4s-ext_2.10-3.2.4.jar:/opt/zeppelin/zeppelin-server/target/lib/Java-WebSocket-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet-3.0.0.v201112011016.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.security.auth.message-1.0.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/guava-15.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-core-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet.jsp.jstl-1.2.0.v201105211821.jar:/opt/zeppelin/zeppelin-server/target/lib/jsr311-api-1.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/woodstox-core-asl-4.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-frontend-jaxrs-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/joda-convert-1.2.jar:/opt/zeppelin/zeppelin-server/target/lib/org.apache.taglibs.standard.glassfish-1.2.0.v201112081803.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-servlet-1.18.1.jar:/opt/zeppelin/zeppelin-server/target/lib/slf4j-api-1.7.5.jar:/opt/zeppelin/zeppelin-server/target/lib/geronimo-servlet_3.0_spec-1.0.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet.jsp-2.2.0.v201112011158.jar:/opt/zeppelin/zeppelin-server/target/lib/geronimo-javamail_1.4_spec-1.7.1.jar:/opt/zeppelin/zeppelin-server/target/lib/asm-3.1.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-core_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/org.eclipse.jdt.core-3.7.1.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.annotation-1.1.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-multipart-1.13.jar:/opt/zeppelin/zeppelin-server/target/lib/dom4j-1.6.1.jar:/opt/zeppelin/zeppelin-server/target/lib/stax2-api-3.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-api-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jaxb-impl-2.2.6.jar:/opt/zeppelin/zeppelin-server/target/lib/javassist-3.12.1.GA.jar:/opt/zeppelin/zeppelin-server/target/lib/com.sun.el-2.2.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-annotations_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-transports-http-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.5.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.ws.rs-api-2.0-m10.jar:/opt/zeppelin/zeppelin-server/target/lib/c3p0-0.9.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-lang-2.5.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.mail.glassfish-1.4.1.v201005082020.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-jaxrs-json-provider-2.0.0.jar:/opt/zeppelin/zeppelin-server/target/lib/wsdl4j-1.6.3.jar:/opt/zeppelin/zeppelin-server/target/lib/reflections-0.9.8.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-module-jaxb-annotations-2.0.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-transports-http-jetty-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-module-jsonSchema-2.1.0.jar:/opt/zeppelin/zeppelin-server/target/lib/gson-2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/scala-library-2.10.4.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-core-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-io-2.4.jar:/opt/zeppelin/zeppelin-zengine/target/lib/log4j-1.2.17.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-configuration-1.9.jar:/opt/zeppelin/zeppelin-zengine/target/lib/quartz-2.2.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/xml-apis-1.4.01.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-logging-1.1.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/guava-15.0.jar:/opt/zeppelin/zeppelin-zengine/target/lib/slf4j-api-1.7.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/dom4j-1.6.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/javassist-3.12.1.GA.jar:/opt/zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/c3p0-0.9.1.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-lang-2.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/reflections-0.9.8.jar:/opt/zeppelin/zeppelin-zengine/target/lib/gson-2.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-io-2.4.jar::/opt/zeppelin/conf:/opt/zeppelin/zeppelin-zengine/target/classes:/opt/zeppelin/zeppelin-server/target/classes').
This is deprecated in Spark 1.0+.
Please instead use:
- ./spark-submit with --driver-class-path to augment the driver classpath
- spark.executor.extraClassPath to augment the executor classpath
WARN [2015-01-13 16:04:15,851] ({Thread-25} Logging.scala[logWarning]:71) - Setting 'spark.executor.extraClassPath' to ':/opt/zeppelin/zeppelin-server/target/lib/log4j-1.2.17.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-configuration-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/xmlschema-core-2.0.3.jar:/opt/zeppelin/zeppelin-server/target/lib/quartz-2.2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.el-2.2.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/mimepull-1.6.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.activation-1.1.0.v201105071233.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-bindings-xml-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-client-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-server-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-jaxrs_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/json4s-native_2.10-3.2.4.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-annotations-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/jetty-jsp-8.1.14.v20131031.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-jersey-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/zeppelin-zengine-0.5.0-SNAPSHOT.jar:/opt/zeppelin/zeppelin-server/target/lib/jetty-all-server-8.1.14.v20131031.jar:/opt/zeppelin/zeppelin-server/target/lib/joda-time-2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-runtime-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-logging-1.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-jersey-jaxrs_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/json4s-ext_2.10-3.2.4.jar:/opt/zeppelin/zeppelin-server/target/lib/Java-WebSocket-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet-3.0.0.v201112011016.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.security.auth.message-1.0.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/guava-15.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-core-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet.jsp.jstl-1.2.0.v201105211821.jar:/opt/zeppelin/zeppelin-server/target/lib/jsr311-api-1.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/woodstox-core-asl-4.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-frontend-jaxrs-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/joda-convert-1.2.jar:/opt/zeppelin/zeppelin-server/target/lib/org.apache.taglibs.standard.glassfish-1.2.0.v201112081803.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-servlet-1.18.1.jar:/opt/zeppelin/zeppelin-server/target/lib/slf4j-api-1.7.5.jar:/opt/zeppelin/zeppelin-server/target/lib/geronimo-servlet_3.0_spec-1.0.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet.jsp-2.2.0.v201112011158.jar:/opt/zeppelin/zeppelin-server/target/lib/geronimo-javamail_1.4_spec-1.7.1.jar:/opt/zeppelin/zeppelin-server/target/lib/asm-3.1.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-core_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/org.eclipse.jdt.core-3.7.1.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.annotation-1.1.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-multipart-1.13.jar:/opt/zeppelin/zeppelin-server/target/lib/dom4j-1.6.1.jar:/opt/zeppelin/zeppelin-server/target/lib/stax2-api-3.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-api-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jaxb-impl-2.2.6.jar:/opt/zeppelin/zeppelin-server/target/lib/javassist-3.12.1.GA.jar:/opt/zeppelin/zeppelin-server/target/lib/com.sun.el-2.2.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-annotations_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-transports-http-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.5.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.ws.rs-api-2.0-m10.jar:/opt/zeppelin/zeppelin-server/target/lib/c3p0-0.9.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-lang-2.5.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.mail.glassfish-1.4.1.v201005082020.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-jaxrs-json-provider-2.0.0.jar:/opt/zeppelin/zeppelin-server/target/lib/wsdl4j-1.6.3.jar:/opt/zeppelin/zeppelin-server/target/lib/reflections-0.9.8.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-module-jaxb-annotations-2.0.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-transports-http-jetty-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-module-jsonSchema-2.1.0.jar:/opt/zeppelin/zeppelin-server/target/lib/gson-2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/scala-library-2.10.4.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-core-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-io-2.4.jar:/opt/zeppelin/zeppelin-zengine/target/lib/log4j-1.2.17.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-configuration-1.9.jar:/opt/zeppelin/zeppelin-zengine/target/lib/quartz-2.2.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/xml-apis-1.4.01.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-logging-1.1.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/guava-15.0.jar:/opt/zeppelin/zeppelin-zengine/target/lib/slf4j-api-1.7.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/dom4j-1.6.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/javassist-3.12.1.GA.jar:/opt/zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/c3p0-0.9.1.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-lang-2.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/reflections-0.9.8.jar:/opt/zeppelin/zeppelin-zengine/target/lib/gson-2.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-io-2.4.jar::/opt/zeppelin/conf:/opt/zeppelin/zeppelin-zengine/target/classes:/opt/zeppelin/zeppelin-server/target/classes' as a work-around.
WARN [2015-01-13 16:04:15,852] ({Thread-25} Logging.scala[logWarning]:71) - Setting 'spark.driver.extraClassPath' to ':/opt/zeppelin/zeppelin-server/target/lib/log4j-1.2.17.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-configuration-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/xmlschema-core-2.0.3.jar:/opt/zeppelin/zeppelin-server/target/lib/quartz-2.2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.el-2.2.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/mimepull-1.6.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.activation-1.1.0.v201105071233.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-bindings-xml-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-client-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-server-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-jaxrs_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/json4s-native_2.10-3.2.4.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-annotations-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/jetty-jsp-8.1.14.v20131031.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-jersey-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/zeppelin-zengine-0.5.0-SNAPSHOT.jar:/opt/zeppelin/zeppelin-server/target/lib/jetty-all-server-8.1.14.v20131031.jar:/opt/zeppelin/zeppelin-server/target/lib/joda-time-2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/atmosphere-runtime-2.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-logging-1.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-jersey-jaxrs_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/json4s-ext_2.10-3.2.4.jar:/opt/zeppelin/zeppelin-server/target/lib/Java-WebSocket-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet-3.0.0.v201112011016.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.security.auth.message-1.0.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/guava-15.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-core-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet.jsp.jstl-1.2.0.v201105211821.jar:/opt/zeppelin/zeppelin-server/target/lib/jsr311-api-1.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/woodstox-core-asl-4.2.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-frontend-jaxrs-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/joda-convert-1.2.jar:/opt/zeppelin/zeppelin-server/target/lib/org.apache.taglibs.standard.glassfish-1.2.0.v201112081803.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-servlet-1.18.1.jar:/opt/zeppelin/zeppelin-server/target/lib/slf4j-api-1.7.5.jar:/opt/zeppelin/zeppelin-server/target/lib/geronimo-servlet_3.0_spec-1.0.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.servlet.jsp-2.2.0.v201112011158.jar:/opt/zeppelin/zeppelin-server/target/lib/geronimo-javamail_1.4_spec-1.7.1.jar:/opt/zeppelin/zeppelin-server/target/lib/asm-3.1.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-core_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/org.eclipse.jdt.core-3.7.1.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.annotation-1.1.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-multipart-1.13.jar:/opt/zeppelin/zeppelin-server/target/lib/dom4j-1.6.1.jar:/opt/zeppelin/zeppelin-server/target/lib/stax2-api-3.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-api-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jaxb-impl-2.2.6.jar:/opt/zeppelin/zeppelin-server/target/lib/javassist-3.12.1.GA.jar:/opt/zeppelin/zeppelin-server/target/lib/com.sun.el-2.2.0.v201108011116.jar:/opt/zeppelin/zeppelin-server/target/lib/swagger-annotations_2.10-1.3.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-transports-http-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.5.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.ws.rs-api-2.0-m10.jar:/opt/zeppelin/zeppelin-server/target/lib/c3p0-0.9.1.1.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-lang-2.5.jar:/opt/zeppelin/zeppelin-server/target/lib/javax.mail.glassfish-1.4.1.v201005082020.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-jaxrs-json-provider-2.0.0.jar:/opt/zeppelin/zeppelin-server/target/lib/wsdl4j-1.6.3.jar:/opt/zeppelin/zeppelin-server/target/lib/reflections-0.9.8.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-module-jaxb-annotations-2.0.0.jar:/opt/zeppelin/zeppelin-server/target/lib/cxf-rt-transports-http-jetty-2.7.7.jar:/opt/zeppelin/zeppelin-server/target/lib/jackson-module-jsonSchema-2.1.0.jar:/opt/zeppelin/zeppelin-server/target/lib/gson-2.1.jar:/opt/zeppelin/zeppelin-server/target/lib/scala-library-2.10.4.jar:/opt/zeppelin/zeppelin-server/target/lib/jersey-core-1.9.jar:/opt/zeppelin/zeppelin-server/target/lib/commons-io-2.4.jar:/opt/zeppelin/zeppelin-zengine/target/lib/log4j-1.2.17.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-configuration-1.9.jar:/opt/zeppelin/zeppelin-zengine/target/lib/quartz-2.2.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/xml-apis-1.4.01.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-logging-1.1.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/guava-15.0.jar:/opt/zeppelin/zeppelin-zengine/target/lib/slf4j-api-1.7.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/dom4j-1.6.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/javassist-3.12.1.GA.jar:/opt/zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/c3p0-0.9.1.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-lang-2.5.jar:/opt/zeppelin/zeppelin-zengine/target/lib/reflections-0.9.8.jar:/opt/zeppelin/zeppelin-zengine/target/lib/gson-2.1.jar:/opt/zeppelin/zeppelin-zengine/target/lib/commons-io-2.4.jar::/opt/zeppelin/conf:/opt/zeppelin/zeppelin-zengine/target/classes:/opt/zeppelin/zeppelin-server/target/classes' as a work-around.
INFO [2015-01-13 16:04:15,865] ({Thread-25} Logging.scala[logInfo]:59) - Changing view acls to: root
INFO [2015-01-13 16:04:15,866] ({Thread-25} Logging.scala[logInfo]:59) - Changing modify acls to: root
INFO [2015-01-13 16:04:15,866] ({Thread-25} Logging.scala[logInfo]:59) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
INFO [2015-01-13 16:04:16,500] ({sparkDriver-akka.actor.default-dispatcher-2} Slf4jLogger.scala[applyOrElse]:80) - Slf4jLogger started
INFO [2015-01-13 16:04:16,569] ({sparkDriver-akka.actor.default-dispatcher-2} Slf4jLogger.scala[apply$mcV$sp]:74) - Starting remoting
INFO [2015-01-13 16:04:16,800] ({sparkDriver-akka.actor.default-dispatcher-5} Slf4jLogger.scala[apply$mcV$sp]:74) - Remoting started; listening on addresses :[akka.tcp://sparkDriver@lab:50242]
INFO [2015-01-13 16:04:16,808] ({sparkDriver-akka.actor.default-dispatcher-5} Slf4jLogger.scala[apply$mcV$sp]:74) - Remoting now listens on addresses: [akka.tcp://sparkDriver@lab:50242]
INFO [2015-01-13 16:04:16,821] ({Thread-25} Logging.scala[logInfo]:59) - Successfully started service 'sparkDriver' on port 50242.
INFO [2015-01-13 16:04:16,844] ({Thread-25} Logging.scala[logInfo]:59) - Registering MapOutputTracker
INFO [2015-01-13 16:04:16,853] ({Thread-25} Logging.scala[logInfo]:59) - Registering BlockManagerMaster
INFO [2015-01-13 16:04:16,878] ({Thread-25} Logging.scala[logInfo]:59) - Created local directory at /tmp/spark-local-20150113160416-c9ab
INFO [2015-01-13 16:04:17,052] ({Thread-25} Logging.scala[logInfo]:59) - Successfully started service 'Connection manager for block manager' on port 47827.
INFO [2015-01-13 16:04:17,055] ({Thread-25} Logging.scala[logInfo]:59) - Bound socket to port 47827 with id = ConnectionManagerId(lab,47827)
INFO [2015-01-13 16:04:17,061] ({Thread-25} Logging.scala[logInfo]:59) - MemoryStore started with capacity 530.0 MB
INFO [2015-01-13 16:04:17,068] ({Thread-25} Logging.scala[logInfo]:59) - Trying to register BlockManager
INFO [2015-01-13 16:04:17,071] ({sparkDriver-akka.actor.default-dispatcher-3} Logging.scala[logInfo]:59) - Registering block manager lab:47827 with 530.0 MB RAM, BlockManagerId(<driver>, lab, 47827, 0)
INFO [2015-01-13 16:04:17,076] ({Thread-25} Logging.scala[logInfo]:59) - Registered BlockManager
INFO [2015-01-13 16:04:17,100] ({Thread-25} Logging.scala[logInfo]:59) - HTTP File server directory is /tmp/spark-1d1f13c3-f5ac-424d-959c-d192df6ba85c
INFO [2015-01-13 16:04:17,101] ({Thread-25} Logging.scala[logInfo]:59) - Starting HTTP Server
INFO [2015-01-13 16:04:17,101] ({Thread-25} Server.java[doStart]:272) - jetty-8.1.14.v20131031
INFO [2015-01-13 16:04:17,105] ({Thread-25} AbstractConnector.java[doStart]:338) - Started SocketC...@0.0.0.0:34006
INFO [2015-01-13 16:04:17,106] ({Thread-25} Logging.scala[logInfo]:59) - Successfully started service 'HTTP file server' on port 34006.
INFO [2015-01-13 16:04:17,382] ({Thread-25} Server.java[doStart]:272) - jetty-8.1.14.v20131031
INFO [2015-01-13 16:04:17,397] ({Thread-25} AbstractConnector.java[doStart]:338) - Started SelectChann...@0.0.0.0:4040
INFO [2015-01-13 16:04:17,398] ({Thread-25} Logging.scala[logInfo]:59) - Successfully started service 'SparkUI' on port 4040.
INFO [2015-01-13 16:04:17,403] ({Thread-25} Logging.scala[logInfo]:59) - Started SparkUI at http://lab:4040
INFO [2015-01-13 16:04:17,832] ({Thread-25} Logging.scala[logInfo]:59) - Created default pool default, schedulingMode: FIFO, minShare: 0, weight: 1
INFO [2015-01-13 16:04:17,910] ({Thread-59} Logging.scala[logInfo]:59) - Registered as framework ID 20150113-141001-1002677692-5050-8322-0010
INFO [2015-01-13 16:04:21,814] ({Thread-25} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:22,315] ({Thread-25} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:22,816] ({Thread-25} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:22,882] ({pool-2-thread-2} Paragraph.java[jobRun]:182) - RUN : Seq(1,2,3)
INFO [2015-01-13 16:04:23,317] ({Thread-25} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:23,835] ({Thread-25} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:23,924] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:456) - Job 20141208-133214_32958749 is finished
INFO [2015-01-13 16:04:23,924] ({pool-2-thread-2} Note.java[persist]:262) - Persist note 2A9X1XSSN into /var/lib/zeppelin/notebook/2A9X1XSSN/note.json
INFO [2015-01-13 16:04:23,929] ({pool-2-thread-2} NotebookServer.java[broadcast]:204) - SEND >> NOTE
INFO [2015-01-13 16:04:23,931] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:99) - Job paragraph_1418045534609_-1316659758 finished by scheduler com.nflabs.zeppelin.spark.SparkInterpreter1457599292
INFO [2015-01-13 16:04:26,084] ({WebSocketWorker-8} NotebookServer.java[onMessage]:75) - RECEIVE << RUN_PARAGRAPH
INFO [2015-01-13 16:04:26,084] ({WebSocketWorker-8} Note.java[persist]:262) - Persist note 2A9X1XSSN into /var/lib/zeppelin/notebook/2A9X1XSSN/note.json
INFO [2015-01-13 16:04:26,091] ({WebSocketWorker-8} NotebookServer.java[broadcast]:204) - SEND >> NOTE
INFO [2015-01-13 16:04:26,092] ({WebSocketWorker-8} NotebookServer.java[broadcast]:204) - SEND >> NOTE
INFO [2015-01-13 16:04:26,094] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:94) - Job paragraph_1418045525633_351611473 started by scheduler com.nflabs.zeppelin.spark.SparkInterpreter1457599292
INFO [2015-01-13 16:04:26,094] ({pool-2-thread-2} NotebookServer.java[broadcast]:204) - SEND >> NOTE
INFO [2015-01-13 16:04:26,097] ({pool-2-thread-2} Paragraph.java[jobRun]:164) - run paragraph 20141208-133205_1705458623 using null com.nflabs.zeppelin.interpreter.LazyOpenInterpreter@7532157
INFO [2015-01-13 16:04:26,097] ({Thread-66} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:26,340] ({pool-2-thread-2} Paragraph.java[jobRun]:182) - RUN : sc.parallelize(Seq(1,2,3)).count()
INFO [2015-01-13 16:04:26,598] ({Thread-66} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:26,834] ({pool-2-thread-2} Logging.scala[logInfo]:59) - Starting job: count at <console>:20
INFO [2015-01-13 16:04:26,852] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Got job 0 (count at <console>:20) with 8 output partitions (allowLocal=false)
INFO [2015-01-13 16:04:26,852] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Final stage: Stage 0(count at <console>:20)
INFO [2015-01-13 16:04:26,853] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Parents of final stage: List()
INFO [2015-01-13 16:04:26,859] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Missing parents: List()
INFO [2015-01-13 16:04:26,864] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Submitting Stage 0 (ParallelCollectionRDD[0] at parallelize at <console>:20), which has no missing parents
INFO [2015-01-13 16:04:26,984] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - ensureFreeSpace(1088) called with curMem=0, maxMem=555755765
INFO [2015-01-13 16:04:26,986] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Block broadcast_0 stored as values in memory (estimated size 1088.0 B, free 530.0 MB)
INFO [2015-01-13 16:04:27,023] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - ensureFreeSpace(800) called with curMem=1088, maxMem=555755765
INFO [2015-01-13 16:04:27,024] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Block broadcast_0_piece0 stored as bytes in memory (estimated size 800.0 B, free 530.0 MB)
INFO [2015-01-13 16:04:27,028] ({sparkDriver-akka.actor.default-dispatcher-13} Logging.scala[logInfo]:59) - Added broadcast_0_piece0 in memory on lab:47827 (size: 800.0 B, free: 530.0 MB)
INFO [2015-01-13 16:04:27,029] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Updated info of block broadcast_0_piece0
INFO [2015-01-13 16:04:27,056] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Submitting 8 missing tasks from Stage 0 (ParallelCollectionRDD[0] at parallelize at <console>:20)
INFO [2015-01-13 16:04:27,058] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Adding task set 0.0 with 8 tasks
INFO [2015-01-13 16:04:27,068] ({sparkDriver-akka.actor.default-dispatcher-4} Logging.scala[logInfo]:59) - Added task set TaskSet_0 tasks to pool default
INFO [2015-01-13 16:04:27,086] ({Thread-67} Logging.scala[logInfo]:59) - Starting task 0.0 in stage 0.0 (TID 0, localhost, PROCESS_LOCAL, 1137 bytes)
INFO [2015-01-13 16:04:27,090] ({Thread-67} Logging.scala[logInfo]:59) - Starting task 1.0 in stage 0.0 (TID 1, localhost, PROCESS_LOCAL, 1137 bytes)
INFO [2015-01-13 16:04:27,091] ({Thread-67} Logging.scala[logInfo]:59) - Starting task 2.0 in stage 0.0 (TID 2, localhost, PROCESS_LOCAL, 1141 bytes)
INFO [2015-01-13 16:04:27,092] ({Thread-67} Logging.scala[logInfo]:59) - Starting task 3.0 in stage 0.0 (TID 3, localhost, PROCESS_LOCAL, 1137 bytes)
INFO [2015-01-13 16:04:27,093] ({Thread-67} Logging.scala[logInfo]:59) - Starting task 4.0 in stage 0.0 (TID 4, localhost, PROCESS_LOCAL, 1137 bytes)
INFO [2015-01-13 16:04:27,108] ({Thread-66} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
INFO [2015-01-13 16:04:27,120] ({MesosSchedulerBackend driver} Logging.scala[logInfo]:59) - driver.run() returned with code DRIVER_ABORTED
INFO [2015-01-13 16:04:27,609] ({Thread-66} NotebookServer.java[broadcast]:204) - SEND >> PROGRESS
Cheers,
Jeremy Subtil
--
You received this message because you are subscribed to a topic in the Google Groups "zeppelin-developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zeppelin-developers/i7oXKHYfB38/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zeppelin-develo...@googlegroups.com.
To post to this group, send email to zeppelin-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages