Presto 0.147 ERROR

891 views
Skip to first unread message

soman...@gmail.com

unread,
Jun 15, 2016, 10:50:17 AM6/15/16
to Presto
The Presto query fails always with ERRORS like below :

2016-06-13T13:02:26.790Z WARN http-client-shared-171 com.facebook.presto.metadata.RemoteNodeState Error fetching node state from http://10.0.40.199:8889/v1/info/state: Server refused connection: http://10.0.40.199:8889/v1/info/state
2016-06-13T13:02:26.790Z WARN http-client-shared-124 com.facebook.presto.metadata.RemoteNodeState Error fetching node state from http://10.0.40.72:8889/v1/info/state: Server refused connection: http://10.0.40.72:8889/v1/info/state
2016-06-13T13:02:26.790Z WARN http-client-shared-124 com.facebook.presto.metadata.RemoteNodeState Error fetching node state from http://10.0.40.83:8889/v1/info/state: Server refused connection: http://10.0.40.83:8889/v1/info/state

2016-06-13T13:02:28.148Z WARN http-client-shared-48 com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.0.40.72:8889/v1/memory: Server refused connection: http://10.0.40.72:8889/v1/memory

2016-06-13T13:04:10.258Z ERROR remote-task-callback-25 com.facebook.presto.execution.StageStateMachine Stage 20160613_130335_00003_6biuf.1 failed
com.facebook.presto.spi.PrestoException: Could not communicate with the remote task. The node may have crashed or be under too much load. This is probably a transient issue, so please retry your query in a few minutes.
at com.facebook.presto.server.remotetask.ContinuousTaskStatusFetcher.updateTaskStatus(ContinuousTaskStatusFetcher.java:234)
.
.
2016-06-13T13:04:10.262Z ERROR exchange-client-5 com.facebook.presto.operator.HttpPageBufferClient Request to delete http://10.0.40.55:8889/v1/task/20160613_130335_00003_6biuf.0.0/results/output.buffer.id failed io.airlift.http.client.RuntimeIOException: Server refused connection: http://10.0.40.55:8889/v1/task/20160613_130335_00003_6biuf.0.0/results/output.buffer.id
.
2016-06-13T13:07:32.394Z WARN http-client-shared-scheduler com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.0.40.199:8889/v1/memory: java.net.SocketTimeoutException: Connect Timeout
2016-06-13T13:07:32.395Z WARN http-client-shared-scheduler com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.0.40.119:8889/v1/memory: java.net.SocketTimeoutException: Connect Timeout
2016-06-13T13:07:32.395Z WARN http-client-shared-scheduler com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.0.40.83:8889/v1/memory: java.net.SocketTimeoutException: Connect Timeout

What causes this ? Though there are some WARN messages, but I want to understand the reason

soman...@gmail.com

unread,
Jun 15, 2016, 10:51:10 AM6/15/16
to Presto, soman...@gmail.com
FULL ERROR Message:
==========================
2016-06-13T13:04:10.231Z WARN http-client-shared-171 com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://10.0.40.83:8889/v1/memory: Server refused connection: http://10.0.40.83:8889/v1/memory

2016-06-13T13:04:10.258Z ERROR remote-task-callback-25 com.facebook.presto.execution.StageStateMachine Stage 20160613_130335_00003_6biuf.1 failed
com.facebook.presto.spi.PrestoException: Could not communicate with the remote task. The node may have crashed or be under too much load. This is probably a transient issue, so please retry your query in a few minutes.
at com.facebook.presto.server.remotetask.ContinuousTaskStatusFetcher.updateTaskStatus(ContinuousTaskStatusFetcher.java:234)
at com.facebook.presto.server.remotetask.ContinuousTaskStatusFetcher.success(ContinuousTaskStatusFetcher.java:168)
at com.facebook.presto.server.remotetask.ContinuousTaskStatusFetcher.success(ContinuousTaskStatusFetcher.java:52)
at com.facebook.presto.server.remotetask.SimpleHttpResponseHandler.onSuccess(SimpleHttpResponseHandler.java:49)
at com.facebook.presto.server.remotetask.SimpleHttpResponseHandler.onSuccess(SimpleHttpResponseHandler.java:27)
at com.google.common.util.concurrent.Futures$6.run(Futures.java:1319)
at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Zhu Li

unread,
Jun 28, 2016, 3:24:13 PM6/28/16
to Presto, soman...@gmail.com
I had the same error too. How did you solve it?

Anup Ash

unread,
Aug 5, 2016, 5:28:33 AM8/5/16
to Presto, soman...@gmail.com


how did you solve it

soman...@gmail.com

unread,
Aug 5, 2016, 5:33:02 AM8/5/16
to Presto, soman...@gmail.com
I have not yet resolved it.
But, I just increased the cluster size. But, still could not find the root cause.

Rickman, Brian T

unread,
Aug 5, 2016, 11:14:11 AM8/5/16
to presto...@googlegroups.com, soman...@gmail.com
The most likely explanation is that some nodes ran out of memory while executing the query.

We'd be able to offer better advice if you provided information about your cluster (number of nodes, amount of memory), size of table(s), and the query.
--
You received this message because you are subscribed to the Google Groups "Presto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to presto-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages