tajo query 실행 시 Progress 진행 되지 않습니다.
44 views
Skip to first unread message
kbel...@gmail.com
May 17, 2017, 5:01:34 AM5/17/17
to Apache Tajo 한국 사용자 그룹
테이블 크기는 300 row 정도 되고
쿼리는 아래와 같이 실행하였습니다.
select * from kma where temp > 10 // 기상청 데이터에서 온도가 10 이상인 것
처음 몇번은 정상적으로 잘 동작하다가 3~4번 쯤 progress; 0% 에서 증가하지 않는 현상이 발생합니다.
아래 처럼 zsql 터미널에 아래와 같이 표시됩니다
Progress: 0%, response time: 151.064 sec
Progress: 0%, response time: 152.067 sec
Progress: 0%, response time: 153.069 sec
Progress: 0%, response time: 154.072 sec
Progress: 0%, response time: 155.075 sec
Progress: 0%, response time: 156.078 sec
Progress: 0%, response time: 157.081 sec
Progress: 0%, response time: 158.085 sec
Progress: 0%, response time: 159.087 sec
Progress: 0%, response time: 160.091 sec
Progress: 0%, response time: 161.094 sec
Progress: 0%, response time: 162.097 sec
Progress: 0%, response time: 163.101 sec
Progress: 0%, response time: 152.067 sec
Progress: 0%, response time: 153.069 sec
Progress: 0%, response time: 154.072 sec
Progress: 0%, response time: 155.075 sec
Progress: 0%, response time: 156.078 sec
Progress: 0%, response time: 157.081 sec
Progress: 0%, response time: 158.085 sec
Progress: 0%, response time: 159.087 sec
Progress: 0%, response time: 160.091 sec
Progress: 0%, response time: 161.094 sec
Progress: 0%, response time: 162.097 sec
Progress: 0%, response time: 163.101 sec
마스터 로그 파일입니다
- tajo-root-master-master.log
2017-05-17 17:56:05,436 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:07,436 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:09,437 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:11,438 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:13,437 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:15,440 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:17,439 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:19,440 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:21,441 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:23,441 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:25,441 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:27,442 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:07,436 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:09,437 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:11,438 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:13,437 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:15,440 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:17,439 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:19,440 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:21,441 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:23,441 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:25,441 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 17:56:27,442 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495011124518_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=master
워커 로그 파일입니다
- tajo-root-worker-master.log
=======================================================
Block Id: eb_1495011124518_0001_000002 [TERMINAL]
=======================================================
Block Id: eb_1495011124518_0001_000002 [TERMINAL]
=======================================================
2017-05-17 17:53:14,668 INFO org.apache.tajo.querymaster.Query:
=======================================================
The order of execution:
=======================================================
The order of execution:
1: eb_1495011124518_0001_000001
2: eb_1495011124518_0001_000002
=======================================================
2017-05-17 17:53:14,669 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.QueryEventType for class org.apache.tajo.querymaster.Query
2017-05-17 17:53:14,707 INFO org.apache.tajo.querymaster.Query: Processing q_1495011124518_0001 of type START
2017-05-17 17:53:14,710 INFO org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Initial executable blocks [eb_1495011124518_0001_000001]
2017-05-17 17:53:14,721 INFO org.apache.tajo.querymaster.Query: q_1495011124518_0001 Query Transitioned from QUERY_NEW to QUERY_RUNNING
2017-05-17 17:53:14,728 INFO org.apache.tajo.querymaster.Stage: org.apache.tajo.querymaster.DefaultTaskScheduler is chosen for the task scheduling for eb_1495011124518_0001_000001
2017-05-17 17:53:14,859 INFO org.apache.tajo.storage.FileTablespace: Total input paths to process : 1
2017-05-17 17:53:15,035 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1
2017-05-17 17:53:15,096 INFO org.apache.tajo.querymaster.Stage: 1 objects are scheduled
2017-05-17 17:53:15,097 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Start TaskScheduler
2017-05-17 17:53:15,111 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved master to /default-rack
2017-05-17 17:53:15,113 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave02 to /default-rack
2017-05-17 17:53:15,113 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave01 to /default-rack
2017-05-17 17:53:15,134 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Assigned host : master, Unknown Volume : -1, Concurrency : 1
2017-05-17 17:53:15,831 INFO org.apache.tajo.worker.TaskManager: QueryMaster Address:master:28093
2017-05-17 17:53:16,027 INFO org.apache.tajo.worker.ExecutionBlockContext: Tajo Root Dir: hdfs://master:9000/tajo
2017-05-17 17:53:16,027 INFO org.apache.tajo.worker.ExecutionBlockContext: Worker Local Dir: /tmp/tajo-root/tmpdir
2017-05-17 17:53:16,054 INFO org.apache.tajo.worker.TaskManager: Running ExecutionBlocks: 1, running tasks:0, availableResource: (Memory:2944, Disks:1, vCores:0)
2: eb_1495011124518_0001_000002
=======================================================
2017-05-17 17:53:14,669 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.QueryEventType for class org.apache.tajo.querymaster.Query
2017-05-17 17:53:14,707 INFO org.apache.tajo.querymaster.Query: Processing q_1495011124518_0001 of type START
2017-05-17 17:53:14,710 INFO org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Initial executable blocks [eb_1495011124518_0001_000001]
2017-05-17 17:53:14,721 INFO org.apache.tajo.querymaster.Query: q_1495011124518_0001 Query Transitioned from QUERY_NEW to QUERY_RUNNING
2017-05-17 17:53:14,728 INFO org.apache.tajo.querymaster.Stage: org.apache.tajo.querymaster.DefaultTaskScheduler is chosen for the task scheduling for eb_1495011124518_0001_000001
2017-05-17 17:53:14,859 INFO org.apache.tajo.storage.FileTablespace: Total input paths to process : 1
2017-05-17 17:53:15,035 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1
2017-05-17 17:53:15,096 INFO org.apache.tajo.querymaster.Stage: 1 objects are scheduled
2017-05-17 17:53:15,097 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Start TaskScheduler
2017-05-17 17:53:15,111 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved master to /default-rack
2017-05-17 17:53:15,113 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave02 to /default-rack
2017-05-17 17:53:15,113 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave01 to /default-rack
2017-05-17 17:53:15,134 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Assigned host : master, Unknown Volume : -1, Concurrency : 1
2017-05-17 17:53:15,831 INFO org.apache.tajo.worker.TaskManager: QueryMaster Address:master:28093
2017-05-17 17:53:16,027 INFO org.apache.tajo.worker.ExecutionBlockContext: Tajo Root Dir: hdfs://master:9000/tajo
2017-05-17 17:53:16,027 INFO org.apache.tajo.worker.ExecutionBlockContext: Worker Local Dir: /tmp/tajo-root/tmpdir
2017-05-17 17:53:16,054 INFO org.apache.tajo.worker.TaskManager: Running ExecutionBlocks: 1, running tasks:0, availableResource: (Memory:2944, Disks:1, vCores:0)
- tajo-site.xml 파일입니다
<property>
<name>tajo.rootdir</name>
<value>hdfs://master:9000/tajo</value>
</property>
<name>tajo.rootdir</name>
<value>hdfs://master:9000/tajo</value>
</property>
<property>
<name>tajo.master.umbilical-rpc.address</name>
<value>master:26001</value>
</property>
<name>tajo.master.umbilical-rpc.address</name>
<value>master:26001</value>
</property>
<property>
<name>tajo.master.client-rpc.address</name>
<value>master:26002</value>
</property>
<name>tajo.master.client-rpc.address</name>
<value>master:26002</value>
</property>
<property>
<name>tajo.resource-tracker.rpc.address</name>
<value>master:26004</value>
</property>
<name>tajo.resource-tracker.rpc.address</name>
<value>master:26004</value>
</property>
<property>
<name>tajo.catalog.client-rpc.address</name>
<value>master:26005</value>
</property>
<name>tajo.catalog.client-rpc.address</name>
<value>master:26005</value>
</property>
<property>
<name>tajo.worker.resource.dfs-dir-aware</name>
<value>true</value>
</property>
<name>tajo.worker.resource.dfs-dir-aware</name>
<value>true</value>
</property>
Jinho Kim
May 17, 2017, 5:24:45 AM5/17/17
to Apache Tajo 한국 사용자 그룹
worker cpu core 가 1개인것 같은데 아래와같이 올려보세요
<name>tajo.worker.resource.cpu-cores</name>
<value>2</value>
</property>
2017년 5월 17일 수요일 오후 6시 1분 34초 UTC+9, kbel...@gmail.com 님의 말:
kbel...@gmail.com
May 17, 2017, 7:42:22 AM5/17/17
to Apache Tajo 한국 사용자 그룹
바꿔보았는데 여전 같은 상태가 발생합니다.
정상적으로 성공하였을 경우
tajo-root-master-master.log 입니다 /
2017-05-17 20:31:37,763 INFO org.apache.tajo.master.GlobalEngine: Query: select * from kma where temp > '10'
2017-05-17 20:31:39,110 INFO org.apache.tajo.master.GlobalEngine: Non Optimized Query:
2017-05-17 20:31:39,110 INFO org.apache.tajo.master.GlobalEngine: Non Optimized Query:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
-----------------------------
PROJECTION(2)
=> Targets: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
SELECTION(1)
=> Search Cond: energy.kma.temp (TEXT) > 10
SCAN(0) on energy.kma
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), energy.kma.temp (TEXT) > 10 as ?greaterthan
=> out schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> Targets: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
SELECTION(1)
=> Search Cond: energy.kma.temp (TEXT) > 10
SCAN(0) on energy.kma
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), energy.kma.temp (TEXT) > 10 as ?greaterthan
=> out schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
2017-05-17 20:31:39,166 INFO org.apache.tajo.master.GlobalEngine: =============================================
2017-05-17 20:31:39,166 INFO org.apache.tajo.master.GlobalEngine: Optimized Query:
2017-05-17 20:31:39,166 INFO org.apache.tajo.master.GlobalEngine: Optimized Query:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
[LogicalPlan]
> ProjectionNode is eliminated.
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
[LogicalPlan]
> ProjectionNode is eliminated.
-----------------------------
SCAN(0) on energy.kma
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
2017-05-17 20:31:39,166 INFO org.apache.tajo.master.GlobalEngine: =============================================
2017-05-17 20:31:39,226 INFO org.apache.tajo.master.exec.QueryExecutor: Query q_1495020655111_0001,select * from kma where temp > '10', is forwarded to :0
2017-05-17 20:31:39,306 INFO org.apache.tajo.master.QueryInProgress: Try to connect to QueryMaster:slave02/192.168.1.176:28093
2017-05-17 20:31:39,468 INFO org.apache.tajo.master.QueryInProgress: Initializing QueryInProgress for QueryID=q_1495020655111_0001
2017-05-17 20:31:39,470 INFO org.apache.tajo.master.scheduler.SimpleScheduler: Running Queries: 1
2017-05-17 20:31:39,470 INFO org.apache.tajo.master.QueryInProgress: Call executeQuery to :slave02:28093,q_1495020655111_0001
2017-05-17 20:31:40,202 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_NOT_ASSIGNED,progress=0.0, queryMaster=slave02
2017-05-17 20:31:42,140 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_NOT_ASSIGNED,progress=0.0, queryMaster=slave02
2017-05-17 20:31:44,144 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=slave02
2017-05-17 20:31:46,142 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=slave02
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_SUCCEEDED,progress=1.0, queryMaster=slave02
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryManager: Stop QueryInProgress:q_1495020655111_0001
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryInProgress: =========================================================
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryInProgress: Stop query:q_1495020655111_0001
2017-05-17 20:31:46,937 INFO org.apache.tajo.storage.FileTablespace: Total input paths to process : 1
2017-05-17 20:31:47,129 INFO org.apache.tajo.util.history.HistoryWriter: Create query history file: hdfs://master:9000/tmp/tajo-root/staging/history/20170517/query-list/query-list-203147.hist
2017-05-17 20:31:48,115 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1
2017-05-17 20:31:51,358 INFO org.apache.tajo.master.exec.NonForwardQueryResultFileScanner: "Sent result to client for 51de466a-8c7e-4d59-b6c0-53c5b36124cf, queryId: q_1495020655111_0001 rows: 300
2017-05-17 20:31:39,226 INFO org.apache.tajo.master.exec.QueryExecutor: Query q_1495020655111_0001,select * from kma where temp > '10', is forwarded to :0
2017-05-17 20:31:39,306 INFO org.apache.tajo.master.QueryInProgress: Try to connect to QueryMaster:slave02/192.168.1.176:28093
2017-05-17 20:31:39,468 INFO org.apache.tajo.master.QueryInProgress: Initializing QueryInProgress for QueryID=q_1495020655111_0001
2017-05-17 20:31:39,470 INFO org.apache.tajo.master.scheduler.SimpleScheduler: Running Queries: 1
2017-05-17 20:31:39,470 INFO org.apache.tajo.master.QueryInProgress: Call executeQuery to :slave02:28093,q_1495020655111_0001
2017-05-17 20:31:40,202 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_NOT_ASSIGNED,progress=0.0, queryMaster=slave02
2017-05-17 20:31:42,140 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_NOT_ASSIGNED,progress=0.0, queryMaster=slave02
2017-05-17 20:31:44,144 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=slave02
2017-05-17 20:31:46,142 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_RUNNING,progress=0.0, queryMaster=slave02
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0001,state=QUERY_SUCCEEDED,progress=1.0, queryMaster=slave02
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryManager: Stop QueryInProgress:q_1495020655111_0001
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryInProgress: =========================================================
2017-05-17 20:31:46,434 INFO org.apache.tajo.master.QueryInProgress: Stop query:q_1495020655111_0001
2017-05-17 20:31:46,937 INFO org.apache.tajo.storage.FileTablespace: Total input paths to process : 1
2017-05-17 20:31:47,129 INFO org.apache.tajo.util.history.HistoryWriter: Create query history file: hdfs://master:9000/tmp/tajo-root/staging/history/20170517/query-list/query-list-203147.hist
2017-05-17 20:31:48,115 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1
2017-05-17 20:31:51,358 INFO org.apache.tajo.master.exec.NonForwardQueryResultFileScanner: "Sent result to client for 51de466a-8c7e-4d59-b6c0-53c5b36124cf, queryId: q_1495020655111_0001 rows: 300
Process 가 0%에서 증가 하지 않았을 떄의
tajo-root-master-master.log 로그입니다
2017-05-17 20:35:22,780 INFO org.apache.tajo.master.GlobalEngine: Query: select * from kma where temp > '10'
2017-05-17 20:35:22,801 INFO org.apache.tajo.master.GlobalEngine: Non Optimized Query:
2017-05-17 20:35:22,801 INFO org.apache.tajo.master.GlobalEngine: Non Optimized Query:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
-----------------------------
PROJECTION(2)
=> Targets: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
SELECTION(1)
=> Search Cond: energy.kma.temp (TEXT) > 10
SCAN(0) on energy.kma
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), energy.kma.temp (TEXT) > 10 as ?greaterthan
=> out schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> Targets: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
SELECTION(1)
=> Search Cond: energy.kma.temp (TEXT) > 10
SCAN(0) on energy.kma
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), energy.kma.temp (TEXT) > 10 as ?greaterthan
=> out schema: {(13) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT), ?greaterthan (BOOLEAN)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
2017-05-17 20:35:22,803 INFO org.apache.tajo.master.GlobalEngine: =============================================
2017-05-17 20:35:22,803 INFO org.apache.tajo.master.GlobalEngine: Optimized Query:
2017-05-17 20:35:22,803 INFO org.apache.tajo.master.GlobalEngine: Optimized Query:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
[LogicalPlan]
> ProjectionNode is eliminated.
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
[LogicalPlan]
> ProjectionNode is eliminated.
-----------------------------
SCAN(0) on energy.kma
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
2017-05-17 20:35:22,803 INFO org.apache.tajo.master.GlobalEngine: =============================================
2017-05-17 20:35:22,804 INFO org.apache.tajo.master.QueryInProgress: Try to connect to QueryMaster:master/192.168.1.174:28093
2017-05-17 20:35:22,811 INFO org.apache.tajo.master.QueryInProgress: Initializing QueryInProgress for QueryID=q_1495020655111_0005
2017-05-17 20:35:22,811 INFO org.apache.tajo.master.scheduler.SimpleScheduler: Running Queries: 1
2017-05-17 20:35:22,811 INFO org.apache.tajo.master.QueryInProgress: Call executeQuery to :master:28093,q_1495020655111_0005
2017-05-17 20:35:22,825 INFO org.apache.tajo.master.exec.QueryExecutor: Query q_1495020655111_0005,select * from kma where temp > '10', is forwarded to master:28093
2017-05-17 20:35:24,011 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:26,012 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:28,012 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:30,014 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:32,014 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:34,015 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:36,016 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:38,016 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:40,019 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:22,804 INFO org.apache.tajo.master.QueryInProgress: Try to connect to QueryMaster:master/192.168.1.174:28093
2017-05-17 20:35:22,811 INFO org.apache.tajo.master.QueryInProgress: Initializing QueryInProgress for QueryID=q_1495020655111_0005
2017-05-17 20:35:22,811 INFO org.apache.tajo.master.scheduler.SimpleScheduler: Running Queries: 1
2017-05-17 20:35:22,811 INFO org.apache.tajo.master.QueryInProgress: Call executeQuery to :master:28093,q_1495020655111_0005
2017-05-17 20:35:22,825 INFO org.apache.tajo.master.exec.QueryExecutor: Query q_1495020655111_0005,select * from kma where temp > '10', is forwarded to master:28093
2017-05-17 20:35:24,011 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:26,012 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:28,012 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:30,014 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:32,014 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:34,015 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:36,016 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:38,016 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
2017-05-17 20:35:40,019 INFO org.apache.tajo.master.QueryInProgress: Received QueryMaster heartbeat:q_1495020655111_0005,state=QUERY_RUNNING,progress=0.0, queryMaster=master
Process 가 0%에서 증가 하지 않았을 떄의
tajo-root-worker-master.log 로그입니다
017-05-17 20:35:22,817 INFO org.apache.tajo.querymaster.QueryMasterManagerService: Receive executeQuery request:q_1495020655111_0005
2017-05-17 20:35:22,817 INFO org.apache.tajo.querymaster.QueryMaster: Start QueryStartEventHandler:q_1495020655111_0005
2017-05-17 20:35:22,817 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.StageEventType for class org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.TaskEventType for class org.apache.tajo.querymaster.QueryMasterTask$TaskEventDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.TaskAttemptEventType for class org.apache.tajo.querymaster.QueryMasterTask$TaskAttemptEventDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.QueryMasterQueryCompletedEvent$EventType for class org.apache.tajo.querymaster.QueryMasterTask$QueryFinishEventHandler
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.TaskSchedulerEvent$EventType for class org.apache.tajo.querymaster.QueryMasterTask$TaskSchedulerDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.LocalTaskEventType for class org.apache.tajo.querymaster.QueryMasterTask$LocalTaskEventHandler
2017-05-17 20:35:22,818 INFO org.apache.tajo.querymaster.QueryMasterTask: INDEX_ENABLED : false
2017-05-17 20:35:23,057 INFO org.apache.tajo.querymaster.QueryMasterTask: The staging dir 'hdfs://master:9000/tmp/tajo-root/staging/q_1495020655111_0005' is created.
2017-05-17 20:35:23,057 INFO org.apache.tajo.engine.planner.global.GlobalPlanner:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
[LogicalPlan]
> ProjectionNode is eliminated.
-----------------------------
2017-05-17 20:35:22,817 INFO org.apache.tajo.querymaster.QueryMaster: Start QueryStartEventHandler:q_1495020655111_0005
2017-05-17 20:35:22,817 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.StageEventType for class org.apache.tajo.querymaster.QueryMasterTask$StageEventDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.TaskEventType for class org.apache.tajo.querymaster.QueryMasterTask$TaskEventDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.TaskAttemptEventType for class org.apache.tajo.querymaster.QueryMasterTask$TaskAttemptEventDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.QueryMasterQueryCompletedEvent$EventType for class org.apache.tajo.querymaster.QueryMasterTask$QueryFinishEventHandler
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.TaskSchedulerEvent$EventType for class org.apache.tajo.querymaster.QueryMasterTask$TaskSchedulerDispatcher
2017-05-17 20:35:22,818 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.LocalTaskEventType for class org.apache.tajo.querymaster.QueryMasterTask$LocalTaskEventHandler
2017-05-17 20:35:22,818 INFO org.apache.tajo.querymaster.QueryMasterTask: INDEX_ENABLED : false
2017-05-17 20:35:23,057 INFO org.apache.tajo.querymaster.QueryMasterTask: The staging dir 'hdfs://master:9000/tmp/tajo-root/staging/q_1495020655111_0005' is created.
2017-05-17 20:35:23,057 INFO org.apache.tajo.engine.planner.global.GlobalPlanner:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
[LogicalPlan]
> ProjectionNode is eliminated.
-----------------------------
SCAN(0) on energy.kma
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
2017-05-17 20:35:23,058 INFO org.apache.tajo.engine.planner.global.GlobalPlanner:
Non-optimized master plan
-------------------------------------------------------------------------------
Execution Block Graph (TERMINAL - eb_1495020655111_0005_000002)
-------------------------------------------------------------------------------
|-eb_1495020655111_0005_000002
|-eb_1495020655111_0005_000001
-------------------------------------------------------------------------------
Order of Execution
-------------------------------------------------------------------------------
1: eb_1495020655111_0005_000001
2: eb_1495020655111_0005_000002
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Execution Block Graph (TERMINAL - eb_1495020655111_0005_000002)
-------------------------------------------------------------------------------
|-eb_1495020655111_0005_000002
|-eb_1495020655111_0005_000001
-------------------------------------------------------------------------------
Order of Execution
-------------------------------------------------------------------------------
1: eb_1495020655111_0005_000001
2: eb_1495020655111_0005_000002
-------------------------------------------------------------------------------
=======================================================
Block Id: eb_1495020655111_0005_000001 [ROOT]
=======================================================
Block Id: eb_1495020655111_0005_000001 [ROOT]
=======================================================
SCAN(0) on energy.kma
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=======================================================
Block Id: eb_1495020655111_0005_000002 [TERMINAL]
=======================================================
Block Id: eb_1495020655111_0005_000002 [TERMINAL]
=======================================================
2017-05-17 20:35:23,059 INFO org.apache.tajo.engine.planner.global.GlobalPlanner:
Optimized master plan
-------------------------------------------------------------------------------
Execution Block Graph (TERMINAL - eb_1495020655111_0005_000002)
-------------------------------------------------------------------------------
|-eb_1495020655111_0005_000002
|-eb_1495020655111_0005_000001
-------------------------------------------------------------------------------
Order of Execution
-------------------------------------------------------------------------------
1: eb_1495020655111_0005_000001
2: eb_1495020655111_0005_000002
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Execution Block Graph (TERMINAL - eb_1495020655111_0005_000002)
-------------------------------------------------------------------------------
|-eb_1495020655111_0005_000002
|-eb_1495020655111_0005_000001
-------------------------------------------------------------------------------
Order of Execution
-------------------------------------------------------------------------------
1: eb_1495020655111_0005_000001
2: eb_1495020655111_0005_000002
-------------------------------------------------------------------------------
=======================================================
Block Id: eb_1495020655111_0005_000001 [ROOT]
=======================================================
Block Id: eb_1495020655111_0005_000001 [ROOT]
=======================================================
SCAN(0) on energy.kma
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> filter: energy.kma.temp (TEXT) > 10
=> target list: energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)
=> out schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=> in schema: {(12) energy.kma.location_code (TEXT), energy.kma.tm (TEXT), energy.kma.time_stamp (TEXT), energy.kma.temp (TEXT), energy.kma.tmx (TEXT), energy.kma.tmn (TEXT), energy.kma.pty (TEXT), energy.kma.pop (TEXT), energy.kma.sky (TEXT), energy.kma.wd (TEXT), energy.kma.ws (TEXT), energy.kma.reh (TEXT)}
=======================================================
Block Id: eb_1495020655111_0005_000002 [TERMINAL]
=======================================================
Block Id: eb_1495020655111_0005_000002 [TERMINAL]
=======================================================
2017-05-17 20:35:23,059 INFO org.apache.tajo.querymaster.Query:
=======================================================
The order of execution:
1: eb_1495020655111_0005_000001
2: eb_1495020655111_0005_000002
=======================================================
2017-05-17 20:35:23,059 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.QueryEventType for class org.apache.tajo.querymaster.Query
2017-05-17 20:35:23,066 INFO org.apache.tajo.querymaster.Query: Processing q_1495020655111_0005 of type START
2017-05-17 20:35:23,067 INFO org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Initial executable blocks [eb_1495020655111_0005_000001]
2017-05-17 20:35:23,067 INFO org.apache.tajo.querymaster.Query: q_1495020655111_0005 Query Transitioned from QUERY_NEW to QUERY_RUNNING
2017-05-17 20:35:23,068 INFO org.apache.tajo.querymaster.Stage: org.apache.tajo.querymaster.DefaultTaskScheduler is chosen for the task scheduling for eb_1495020655111_0005_000001
2017-05-17 20:35:23,122 INFO org.apache.tajo.storage.FileTablespace: Total input paths to process : 1
2017-05-17 20:35:23,140 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1
2017-05-17 20:35:23,140 INFO org.apache.tajo.querymaster.Stage: 1 objects are scheduled
2017-05-17 20:35:23,141 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Start TaskScheduler
2017-05-17 20:35:23,144 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved master to /default-rack
2017-05-17 20:35:23,144 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave02 to /default-rack
2017-05-17 20:35:23,144 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave01 to /default-rack
2017-05-17 20:35:23,153 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Assigned host : master, Unknown Volume : -1, Concurrency : 1
2017-05-17 20:35:23,205 INFO org.apache.tajo.worker.TaskManager: QueryMaster Address:master:28093
2017-05-17 20:35:23,292 INFO org.apache.tajo.worker.ExecutionBlockContext: Tajo Root Dir: hdfs://master:9000/tajo
2017-05-17 20:35:23,292 INFO org.apache.tajo.worker.ExecutionBlockContext: Worker Local Dir: /tmp/tajo-root/tmpdir
2017-05-17 20:35:23,367 INFO org.apache.tajo.worker.TaskManager: Running ExecutionBlocks: 1, running tasks:0, availableResource: (Memory:2944, Disks:1, vCores:0)
2: eb_1495020655111_0005_000002
=======================================================
2017-05-17 20:35:23,059 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.event.QueryEventType for class org.apache.tajo.querymaster.Query
2017-05-17 20:35:23,066 INFO org.apache.tajo.querymaster.Query: Processing q_1495020655111_0005 of type START
2017-05-17 20:35:23,067 INFO org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Initial executable blocks [eb_1495020655111_0005_000001]
2017-05-17 20:35:23,067 INFO org.apache.tajo.querymaster.Query: q_1495020655111_0005 Query Transitioned from QUERY_NEW to QUERY_RUNNING
2017-05-17 20:35:23,068 INFO org.apache.tajo.querymaster.Stage: org.apache.tajo.querymaster.DefaultTaskScheduler is chosen for the task scheduling for eb_1495020655111_0005_000001
2017-05-17 20:35:23,122 INFO org.apache.tajo.storage.FileTablespace: Total input paths to process : 1
2017-05-17 20:35:23,140 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1
2017-05-17 20:35:23,140 INFO org.apache.tajo.querymaster.Stage: 1 objects are scheduled
2017-05-17 20:35:23,141 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Start TaskScheduler
2017-05-17 20:35:23,144 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved master to /default-rack
2017-05-17 20:35:23,144 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave02 to /default-rack
2017-05-17 20:35:23,144 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved slave01 to /default-rack
2017-05-17 20:35:23,153 INFO org.apache.tajo.querymaster.DefaultTaskScheduler: Assigned host : master, Unknown Volume : -1, Concurrency : 1
2017-05-17 20:35:23,205 INFO org.apache.tajo.worker.TaskManager: QueryMaster Address:master:28093
2017-05-17 20:35:23,292 INFO org.apache.tajo.worker.ExecutionBlockContext: Tajo Root Dir: hdfs://master:9000/tajo
2017-05-17 20:35:23,292 INFO org.apache.tajo.worker.ExecutionBlockContext: Worker Local Dir: /tmp/tajo-root/tmpdir
2017-05-17 20:35:23,367 INFO org.apache.tajo.worker.TaskManager: Running ExecutionBlocks: 1, running tasks:0, availableResource: (Memory:2944, Disks:1, vCores:0)
귀찮게해서 죄송합니다 ㅜㅜ
tajo-site.xml 파일에 말씀하신 프로퍼티 추가하였습니다 (master , slave01, slave02)
conf/workers 파일에 master , slave01, slave02 설정해주었는데 이것 때문에 문제가 발생할수도 있을까요?
kbel...@gmail.com
May 17, 2017, 8:47:47 PM5/17/17
to Apache Tajo 한국 사용자 그룹
해결한 것 같습니다.
Tajo Master 가 돌아가는 서버에 Tajo Worker 도 동시에 돌려서 생긴 문제 같습니다
conf/workers 파일에
master
slave01
slave02
이렇게 설정되어있던 것에서 master 를 제거 하고 타조를 재 실행해보니 Progress가 0%에서 증가하지 않는 현상이 사라졌습니다
혹시 왜 그런지 이유를 알수 있을 까요???
Jinho Kim
May 17, 2017, 10:01:51 PM5/17/17
to Apache Tajo 한국 사용자 그룹
slave 노드들이 다 죽고 master 노드에 띄운 worker 리소스가 작게 잡혀있는 경우를 제외하고 못본것 같습니다.
혹시 다른 에러로그 같은건 없었는지요?
(Memory:2944, Disks:1, vCores:0) 이것으로 보면 QueryMaster 하나실행되고 cpu가 0인것을 보면 재시작이 안된건지도 의심 해봐야할것 같고요
Reply all
Reply to author
Forward
0 new messages