h2o.table fails in distributed mode
110 views
Skip to first unread message
Denis Perevalov
Aug 31, 2016, 12:00:31 PM8/31/16
to H2O Open Source Scalable Machine Learning - h2ostream
I have a problem using h2o.table in distributed mode. I use h2o 3.10.0.3 R API
I execute the following code:
#N <- 100000 # Succeed
N <- 1000000 # Fail
x0 <- -1 + 0.0
x_sigma <- 0.55
k0 <- 0.001
k1 <- 0.035
gauss_sigma <- 0.02
xmin = -2.5
xmax = 1
seed = 1
set.seed(seed)
x <- rnorm(N, x0, x_sigma)
y <- 0.1 * (x^3 + 2*x^2) + 0.4
y <- y + rnorm(N, 0, gauss_sigma)
y <- ifelse(y > 0.1, y, 0.1)
y <- ifelse(y > 0.8, 0.8, y)
y_binomial <- rbinom(length(y), 1, y)
df <- data.frame(x=x, y=y, y_binomial=y_binomial)
df.hex = as.h2o(df)
h2o.table(df.hex[, c("x")])
Notice I have different N value in the beginning. My execution succeeds in the following cases:
1. N <- 100000. Regardless of 1-node or 2-node cluster
2. 1-node H2O cluster regardless of N
However if I have both
a. N <- 1000000
b. 2-node cluster
my execution fails with a message
Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = page, : ERROR MESSAGE: DistributedException from /10.2.91.178:5777
Erin LeDell
Aug 31, 2016, 10:09:23 PM8/31/16
to Denis Perevalov, H2O Open Source Scalable Machine Learning - h2ostream
Hi Denis,
Thanks for the report. Do you happen to have your h2o logs? You can post them here or send them to my email.
It's unclear at this point whether this is a bug in h2o.table, or
maybe you ran out of memory? What size cluster did you have?
-Erin
--
You received this message because you are subscribed to the Google Groups "H2O Open Source Scalable Machine Learning - h2ostream" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2ostream+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
-- Erin LeDell Ph.D. Statistician & Machine Learning Scientist | H2O.ai
Denis Perevalov
Aug 31, 2016, 10:23:22 PM8/31/16
to H2O Open Source Scalable Machine Learning - h2ostream, deni...@gmail.com
Hi Erin
OK. I'll try to recreate it tomorrow. I doubt that it's a memory issue since it works on a single machine. When I add an extra machine to the cluster with just as much memory, it doesn't work. Also, other operations on that H2OFrame like model fitting work just fine. But who knows.
OK. I'll try to recreate it tomorrow. I doubt that it's a memory issue since it works on a single machine. When I add an extra machine to the cluster with just as much memory, it doesn't work. Also, other operations on that H2OFrame like model fitting work just fine. But who knows.
Where would I find those logs you are referring to? Just those from the console that runs the H2O application?
Denis Perevalov
Sep 6, 2016, 11:00:28 AM9/6/16
to H2O Open Source Scalable Machine Learning - h2ostream, deni...@gmail.com
Hi Erin,
Here is the log that I get:
09-06 07:55:53.599 10.2.91.178:5777 18904 FJ-1-173 INFO: ** Note: Dataset is not well distributed, consider rebalancing ** 09-06 07:55:53.666 10.2.91.178:5777 18904 #32337-27 INFO: GET /3/Frames/df, parms: {row_count=10} 09-06 07:55:54.551 10.2.91.178:5777 18904 #32337-31 INFO: POST /99/Rapids, parms: {ast=(tmp= RTMP_sid_b867_3282 (table (cols df [0]) TRUE)), session_id=_sid_b867} 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: DistributedException from /10.2.91.178:5777, caused by water.exceptions.H2OIllegalArgumentException: unimplemented 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.getResult(MRTask.java:477) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.getResult(MRTask.java:485) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.doAll(MRTask.java:389) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.doAll(MRTask.java:376) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.doAll(MRTask.java:375) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable.slow_table(AstTable.java:144) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable.apply(AstTable.java:65) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.AstExec.exec(AstExec.java:74) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.assign.AstTmpAssign.apply(AstTmpAssign.java:35) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.assign.AstTmpAssign.apply(AstTmpAssign.java:15) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.AstExec.exec(AstExec.java:74) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.Session.exec(Session.java:75) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.Rapids.exec(Rapids.java:81) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RapidsHandler.exec(RapidsHandler.java:41) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at java.lang.reflect.Method.invoke(Unknown Source) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.Handler.handle(Handler.java:56) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RequestServer.serve(RequestServer.java:394) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RequestServer.doGeneric(RequestServer.java:245) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RequestServer.doPost(RequestServer.java:199) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.Server.handle(Server.java:370) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at java.lang.Thread.run(Unknown Source) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: Caused by: water.exceptions.H2OIllegalArgumentException: unimplemented 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.H2O.unimpl(H2O.java:921) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable$SlowCnt.reduce(AstTable.java:308) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable$SlowCnt.reduce(AstTable.java:264) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.reduce4(MRTask.java:743) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.reduce3(MRTask.java:730) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.postLocal0(MRTask.java:698) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.onCompletion(MRTask.java:670) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.CountedCompleter.__tryComplete(CountedCompleter.java:425) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.RPC$2.compute2(RPC.java:623) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1196) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 09-06 07:55:55.652 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: DistributedException from /10.2.91.178:5777, caused by water.exceptions.H2OIllegalArgumentException: unimplemented 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.getResult(MRTask.java:477) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.getResult(MRTask.java:485) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.doAll(MRTask.java:389) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.doAll(MRTask.java:376) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.doAll(MRTask.java:375) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable.slow_table(AstTable.java:144) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable.apply(AstTable.java:65) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.AstExec.exec(AstExec.java:74) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.assign.AstTmpAssign.apply(AstTmpAssign.java:35) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.assign.AstTmpAssign.apply(AstTmpAssign.java:15) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.AstExec.exec(AstExec.java:74) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.Session.exec(Session.java:75) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.Rapids.exec(Rapids.java:81) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RapidsHandler.exec(RapidsHandler.java:41) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at java.lang.reflect.Method.invoke(Unknown Source) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.Handler.handle(Handler.java:56) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RequestServer.serve(RequestServer.java:394) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RequestServer.doGeneric(RequestServer.java:245) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.api.RequestServer.doPost(RequestServer.java:199) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.Server.handle(Server.java:370) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at java.lang.Thread.run(Unknown Source) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: Caused by: water.exceptions.H2OIllegalArgumentException: unimplemented 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.H2O.unimpl(H2O.java:921) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable$SlowCnt.reduce(AstTable.java:308) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.rapids.ast.prims.advmath.AstTable$SlowCnt.reduce(AstTable.java:264) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.reduce4(MRTask.java:743) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.reduce3(MRTask.java:730) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.postLocal0(MRTask.java:698) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.MRTask.onCompletion(MRTask.java:670) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.CountedCompleter.__tryComplete(CountedCompleter.java:425) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.RPC$2.compute2(RPC.java:623) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1196) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 09-06 07:55:55.659 10.2.91.178:5777 18904 #32337-31 ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) 09-06 07:55:55.660 10.2.91.178:5777 18904 #32337-31 ERRR: Caught exception: 09-06 07:55:55.660 10.2.91.178:5777 18904 #32337-31 ERRR: 09-06 07:55:55.660 10.2.91.178:5777 18904 #32337-31 ERRR: ERROR MESSAGE: 09-06 07:55:55.660 10.2.91.178:5777 18904 #32337-31 ERRR: 09-06 07:55:55.660 10.2.91.178:5777 18904 #32337-31 ERRR: DistributedException from /10.2.91.178:5777 09-06 07:55:55.660 10.2.91.178:5777 18904 #32337-31 ERRR: 09-06 07:55:55.660 10.2.91.178:5777 18904 #32337-31 ERRR: ; Stacktrace: [DistributedException from /10.2.91.178:5777, caused by water.exceptions.H2OIllegalArgumentException: unimplemented, water.MRTask.getResult(MRTask.java:477), water.MRTask.getResult(MRTask.java:485), water.MRTask.doAll(MRTask.java:389), water.MRTask.doAll(MRTask.java:376), water.MRTask.doAll(MRTask.java:375), water.rapids.ast.prims.advmath.AstTable.slow_table(AstTable.java:144), water.rapids.ast.prims.advmath.AstTable.apply(AstTable.java:65), water.rapids.ast.AstExec.exec(AstExec.java:74), water.rapids.ast.prims.assign.AstTmpAssign.apply(AstTmpAssign.java:35), water.rapids.ast.prims.assign.AstTmpAssign.apply(AstTmpAssign.java:15), water.rapids.ast.AstExec.exec(AstExec.java:74), water.rapids.Session.exec(Session.java:75), water.rapids.Rapids.exec(Rapids.java:81), water.api.RapidsHandler.exec(RapidsHandler.java:41), sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source), sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source), java.lang.reflect.Method.invoke(Unknown Source), water.api.Handler.handle(Handler.java:56), water.api.RequestServer.serve(RequestServer.java:394), water.api.RequestServer.doGeneric(RequestServer.java:245), water.api.RequestServer.doPost(RequestServer.java:199), javax.servlet.http.HttpServlet.service(HttpServlet.java:755), javax.servlet.http.HttpServlet.service(HttpServlet.java:848), org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684), org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503), org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137), org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557), org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231), org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086), org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429), org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193), org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020), org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135), org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154), org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116), org.eclipse.jetty.server.Server.handle(Server.java:370), org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494), org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53), org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982), org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043), org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865), org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240), org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72), org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264), org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608), org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543), java.lang.Thread.run(Unknown Source), Caused by:water.exceptions.H2OIllegalArgumentException: unimplemented, water.H2O.unimpl(H2O.java:921), water.rapids.ast.prims.advmath.AstTable$SlowCnt.reduce(AstTable.java:308), water.rapids.ast.prims.advmath.AstTable$SlowCnt.reduce(AstTable.java:264), water.MRTask.reduce4(MRTask.java:743), water.MRTask.reduce3(MRTask.java:730), water.MRTask.postLocal0(MRTask.java:698), water.MRTask.onCompletion(MRTask.java:670), jsr166y.CountedCompleter.__tryComplete(CountedCompleter.java:425), water.RPC$2.compute2(RPC.java:623), water.H2O$H2OCountedCompleter.compute(H2O.java:1196), jsr166y.CountedCompleter.exec(CountedCompleter.java:468), jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263), jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974), jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477), jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)] 09-06 07:57:14.972 10.2.91.178:5777 18904 #32337-32 INFO: GET /3/Logs/nodes/-1/files/info, parms: {} 09-06 07:57:15.127 10.2.91.178:5777 18904 #32337-32 INFO: GET /3/Logs/nodes/0/files/info, parms: {}
Reply all
Reply to author
Forward
0 new messages