์ฟผ๋ฆฌ ์ˆ˜ํ–‰ ์‹œ ๋ฌดํ•œ ๋Œ€๊ธฐ ์ƒํƒœ ์งˆ๋ฌธ๋“œ๋ฆฝ๋‹ˆ๋‹ค. (Resourcetracker, Query master ๋ฌธ์ œ)

256 views
Skip to first unread message

Mellisa

unread,
Aug 26, 2015, 5:08:25โ€ฏAM8/26/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”

ํƒ€์กฐ ์ฝ”๋“œ๋ฅผ ๋ฐ›์•„ ๋ฐ”์ด๋„ˆ๋ฆฌ๋ฅผ ๋งŒ๋“  ํ›„
ํƒ€์กฐ๋ฅผ ์‹คํ–‰ํ•˜์˜€์Šต๋‹ˆ๋‹ค.

ํ˜„์žฌ ํด๋Ÿฌ์Šคํ„ฐ ๊ตฌ์„ฑ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
host1 : tajo master, namenode
host2,3,4 : tajo worker, datanode

๋งˆ์Šคํ„ฐ์ธ host1์—์„œ ๊ฐ„๋‹จํ•œ ์ฟผ๋ฆฌ select count(*) from lineitem; ์„ ํ–ˆ์„ ๊ฒฝ์šฐ์—

ํ„ฐ๋ฏธ๋„์ƒ์—์„œ ๋ฌดํ•œ ๋Œ€๊ธฐ ์ƒํƒœ๊ฐ€ ๋ฐœ์ƒํ•˜์˜€์Šต๋‹ˆ๋‹ค.

์ด์—, ๋กœ๊ทธ๋ฅผ ํ™•์ธ ํ•ด ๋ณธ ๊ฒฐ๊ณผ

๋งˆ์Šคํ„ฐ์ธ host1์˜ ๋กœ๊ทธ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

2015-08-26 17:40:43,815 INFO org.apache.tajo.master.schedulr: No Available Resources for QueryMaster :q_1440576503051_051_0001,state=QUERY_MASTER_INIT,progress=0.0, queryMaster=

์›Œ์ปค์˜ ๋กœ๊ทธ๋Š” ์•„๋ž˜ ๋ฌธ๊ตฌ๊ฐ€ ๋ฐ˜๋ณต์ ์œผ๋กœ ์ฐํžˆ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.

2015-08-26 17:47:59,309 WARN: org.apache.tajo.rpc.NettyClientBase (doReconnect(198)) -
Exception [org.apache.tajo.ipc.TajoResourceTrackerProtocol(localhost/127.0.0.1:26003)]: ConnectException: ์—ฐ๊ฒฐ์ด ๊ฑฐ๋ถ€ ๋จ: localhost/127.0.0.1:26003 Try to reconnect : localhost/127.0.0.1:26003

์›น ul ์—์„œ๋Š” ํ•ด๋‹น ์ฟผ๋ฆฌ๊ฐ€ query init()์—์„œ ๋ฌดํ•œ ๋Œ€๊ธฐ ์ƒํƒœ๋กœ progress 0% ์ž…๋‹ˆ๋‹ค.

ํฌํŠธ ๋ฌธ์ œ์ธ๊ฐ€ ์‹ถ์–ด์„œ master์˜ 26003์˜ ํฌํŠธ๋ฅผ ํ™•์ธํ•ด ๋ณธ ๊ฒฐ๊ณผ listen ์ƒํƒœ์ž„์„ ํ™•์ธํ•˜์˜€์Šต๋‹ˆ๋‹ค.

์ง€๋‚œ ๊ฒŒ์‹œ๊ธ€์„ ์ฐธ๊ณ ํ•˜๋‹ˆ ๋งŒ์•ฝ ํƒ€์กฐ๋ฅผ ๊ตฌ๋™ํ•˜๋Š” ์‹œ์ž‘ ๋ถ€๋ถ„์—์„œ
์›Œ์ปค์—TajoResourceTrackerProtocol ์—ฐ๊ฒฐ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•œ๋‹ค๋ฉด ๋ฌธ์ œ๊ฐ€ ๋˜์ง€ ์•Š๋Š”๋‹ค๊ณ  ํ•ด์„œ
ํ™•์ธํ•ด๋ณด๋‹ˆ ์ฒ˜์Œ์— ๋ฐœ์ƒํ•˜๊ธด ํ•ฉ๋‹ˆ๋‹ค.

ํ•˜์ง€๋งŒ ์œ„์˜ ์ฟผ๋ฆฌ๋งˆ์Šคํ„ฐ ์ž์› ํ• ๋‹น ๋ถ€๋ถ„์—์„œ ๋ฉˆ์ถฐ ์žˆ๋Š” ๊ฒƒ์„ ๋ณด๋‹ˆ TajoResourceTrackerProtocol๊ณผ ๊ด€๋ จ์ด ์žˆ๋Š” ๊ฒƒ ๊ฐ™์•„ ์งˆ๋ฌธ๋“œ๋ฆฝ๋‹ˆ๋‹ค.


๊ด€๋ จํ•ด์„œ ์กฐ์–ธ ์ฃผ์‹œ๋ฉด ๊ฐ์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค!

Jinho Kim

unread,
Aug 26, 2015, 5:18:09โ€ฏAM8/26/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”.

๋กœ๊ทธ์— ์ฐํžŒ ๋ถ€๋ถ„์„ ๋ณด์‹œ๋ฉด worker ๊ฐ€ localhost ๋กœ master ์— ์—ฐ๊ฒฐํ•˜๋ ค๊ณ  ํ•˜๋Š”๋ฐ master ๋Š” localhost ๊ฐ€ ์•„๋‹๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.
๋จผ์ € cluster ์„ค์ •์„ ํ•˜์‹ ํ›„ ๊ฐ tajo worker ๋กœ ๋ณต์‚ฌํ•˜์‹œ๊ณ  ์‚ฌ์šฉํ•˜์‹œ๊ธฐ ๋ฐ”๋ž๋‹ˆ๋‹ค.

2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:

์ •๊พธ

unread,
Aug 26, 2015, 6:35:10โ€ฏAM8/26/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์™ ์ง€ ๋งˆ์Šคํ„ฐ๋…ธ๋“œ์— tajo-site.xml ์„ค์ •์„ ์•ˆํ•ด์„œ localhost๋กœ ๋œฌ๊ฒƒ์€ ์•„๋‹Œ๊ฐ€ ์‹ถ๋„ค์š”.
tajo-site.xml์„ค์ •ํ›„ ๋ชจ๋“  ๋…ธ๋“œ์— ๋ณต์‚ฌํ•ด์„œ ์žฌ๊ธฐ๋™ ํ•˜์„ธ์š”.

hyejong jeong

unread,
Aug 26, 2015, 12:20:42โ€ฏPM8/26/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
ย ์•ˆ๋…•ํ•˜์„ธ์š”!

์šฐ์„ ย Try to reconnect ๋ฌธ์ œ ๋ถ€ํ„ฐ ์ œ ๊ฒฝํ—˜์„ ๋ฐ”ํƒ•์œผ๋กœ ์„ค๋ช…๋“œ๋ฆฌ๊ฒ ์Šต๋‹ˆ๋‹ค.

host1 ย : tajo master, namenode : tajo-site.xml ์„ค์ •ํ•˜๊ธฐ - ์ œ๊ฐ€ ์‚ฝ์งˆํ–ˆ๋˜ ๊ฐ€์žฅํฐ ์‹ค์ˆ˜๋Š” Tajo Master์— tajo-site.xml์„ ์„ค์ •ํ•˜์ง€ ์•Š์€ ๊ฒƒ์ด์˜€์Šต๋‹ˆ๋‹ค.

host2,3,4 : tajo worker, datanodeย : tajo-site.xml ๋ณต์‚ฌํ•˜๊ธฐ

1. tajo-site.xml ํŒŒ์ผ์€ ๋ชจ๋“  ๋…ธ๋“œ์— ์กด์žฌํ•ด์•ผํ•ฉ๋‹ˆ๋‹ค. (host1, 2, 3, 4 ๋ชจ๋‘์— ์กด์žฌํ•ด์•ผํ•จ)

2. Master ๋…ธ๋“œ์— ๋Œ€ํ•œ ํ˜ธ์ŠคํŠธ or IP๋ฅผ ์ •์˜ํ•ด์•ผ ย ๋งˆ์Šคํ„ฐ ๋…ธ๋“œ๋กœ ๋ถ™์„์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

ex) Tajo Master Node IP :ย 172.31.22.99 ( ์ € ๊ฐ™์€ ๊ฒฝ์šฐ ์•„๋งˆ์กด ์›น ์„œ๋น„์Šค(aws) ์‚ฌ์šฉํ•˜์—ฌ ํ”„๋ผ์ด๋น— ์•„์ดํ”ผ๋ฅผ ์‚ฌ์šฉํ•˜์˜€์Šต๋‹ˆ๋‹ค.)

<property>

ย  <name>tajo.resource-tracker.rpc.address</name>

ย  <value>172.31.22.99:26003</value>

ย  <description>TajoMaster binding address between master and workers.</description>

</property>


์•„๋ž˜์™€ ๊ฐ™์€ ์ปค๋„ฅ์…˜ ์˜ค๋ฅ˜๋Š” ํƒ€์กฐ ์›Œ์ปค(worker)์—๋Š” tajo-site.xml๋ฅผ ์„ค์ •ํ•˜์˜€์ง€๋งŒ, ํƒ€์กฐ ๋งˆ์Šคํ„ฐ(master)์—๋Š” tajo-site.xml ํŒŒ์ผ์ด ์—†๊ฑฐ๋‚˜ ์ˆ˜์ •ํ•˜์ง€ ์•Š์•˜์„ ๊ฒฝ์šฐ์ผ ๊ฐ€๋Šฅ์„ฑ์ด ๋†’์Šต๋‹ˆ๋‹ค.


a. tajo master๋ฅผ tajo-site.xml์— ์„ค์ •ํ•˜์ง€ ์•Š์€๊ฒฝ์šฐ.



b. tajo master๋ฅผ tajo-site.xml์— ์„ค์ •ํ•œ ๊ฒฝ์šฐ.


์ •์„ ์ƒ์˜ ๋ธ”๋กœ๊ทธ( http://tost.tistory.com/192 )๋ฅผ ์ฐธ๊ณ ํ•˜์—ฌ ์„ค๋ช…๋“œ๋ฆฝ๋‹ˆ๋‹ค.

2015-08-26 17:47:59,309 WARN: org.apache.tajo.rpc.NettyClientBase (doReconnect(198)) -ย 
Exception [org.apache.tajo.ipc.TajoResourceTrackerProtocol(localhost/127.0.0.1:26003)]: ConnectException: ์—ฐ๊ฒฐ์ด ๊ฑฐ๋ถ€ ย  ย  ย  ย  ย  ย  ย  ย  ย  ย  ย  ย  ย  ๋จ: localhost/127.0.0.1:26003ย Try to reconnect : localhost/127.0.0.1:26003

๋‘๋ฒˆ์งธ๋Š” ํƒ€์กฐ ์›Œ์ปค๊ฐ€ ์–ด๋– ํ•œ ํŒŒ์ผ์„ ๊ฐ€์ง€๊ณ  ์žˆ์–ด์•ผ ํ•˜๋Š” ๋ฌธ์ œ์ž…๋‹ˆ๋‹ค.

-->ย ย tajo-site.xml , tajo-env.sh ํŒŒ์ผ๋ฅผ worker ๋…ธ๋“œ์„œ๋ฒ„์— ๋ณต์‚ฌ

์ •ํ˜œ์ข… ๋“œ๋ฆผ.


2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

Mellisa

unread,
Aug 26, 2015, 2:02:16โ€ฏPM8/26/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
๋‹ต๋ณ€ ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค.ย 

์šฐ์„  ๋ง์”€ํ•ด์ฃผ์…จ๋˜ย 
ย localhost/127.0.0.1:26003ย Try to reconnect : localhost/127.0.0.1:26003ย ย 
๋ถ€๋ถ„์—์„œ localhost๋กœ ํ‘œ๊ธฐ๋˜๋Š” ๋ถ€๋ถ„์ด master์˜ ip๊ฐ€ ๋œจ๋„๋ก ์„ค์ •์„ ์ˆ˜์ •ํ•˜๊ณ  ์›น ui์—์„œ ํ™•์ธํ•˜์˜€์Šต๋‹ˆ๋‹ค.ย 



ํ•˜์ง€๋งŒ, ์—ฌ์ „ํžˆ ์•„๋ž˜์˜ ์บก์ณ๋œ ์›Œ์ปค ๋กœ๊ทธ ๋‚ด์šฉ๊ณผ ๊ฐ™์ด resourcetracker ์—ฐ๊ฒฐ ๊ฑฐ๋ถ€ ์—๋Ÿฌ๊ฐ€ ๋ฐœ์ƒํ•˜๊ณ ย 
select count(*) from tablename; ์ฟผ๋ฆฌ๋Š” query master init()์ƒํƒœ์—์„œ progress 0% ๋กœ ์ •์ง€ํ•ด์žˆ์Šต๋‹ˆ๋‹ค.


๋งˆ์Šคํ„ฐ์™€ ์›Œ์ปค๊ฐ„์— ํ†ต์‹ ์ด ์•ˆ ๋˜๋Š” ๋ฌธ์ œ๋ผ๊ณ  ์˜ˆ์ƒ๋˜๋Š”๋ฐ์š”ย 

ํ˜„์žฌ tajo-0.11.0-snapshot binary ์‚ฌ์šฉ์ค‘์ด๋ฉฐ,

master : 227
worker : 229, 236, 237ย 


tajo-env.sh ์„ค์ •์€ ์•„๋ž˜์™€ ๊ฐ™์Šต๋‹ˆ๋‹ค.ย 

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51-2.4.5.5.el7.x86_64/
export HADOOP_HOME=/home/team_system/hadoop-2.7.1
export TAJO_CLASSPATH=$HADOOP_HOME/clinet/*:$HADOOP_HOME/lib/*

tajo-site.xml ์„ค์ •์€ ์•„๋ž˜์™€ ๊ฐ™์Šต๋‹ˆ๋‹ค.ย 

<configuration>

ย <property>
ย  <name>tajo.rootdir</name>
ย  <value>hdfs://165.132.122.227:9000/home/team_system/tajo</value>
ย  <description>Base directory including system directories.</description>
</property>

<property>
ย  <name>tajo.master.umbilical-rpc.address</name>
ย  <value>165.132.122.227:26001</value>
ย  <description>TajoMaster binding address between master and workers.</description>
</property>

<property>
ย  <name>tajo.master.client-rpc.address</name>
ย  <value>165.132.122.227:26002</value>
ย  <description>TajoMaster binding address between master and clients.</description>
</property>

<property>
ย  <name>tajo.resource-tracker.rpc.address</name>
ย  <value>165.132.122.227:26003</value>
</property>

<property>
ย  <name>tajo.catalog.client-rpc.address</name>
ย  <value>165.132.122.227:26005</value>
ย  <description>CatalogServer binding address between catalog server and workers.</description>
</property>

</configuration>


ย 
ย 



์กฐ์–ธ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค.


2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

Jihoon Son

unread,
Aug 26, 2015, 10:48:17โ€ฏPM8/26/15
to Mellisa, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”
์ฒซ๋ฒˆ์งธ ์Šค๋ƒ…์ƒท์„ ๋ณด๋ฉด ๋…ธ๋“œ ์ฃผ์†Œ๊ฐ€ localhost๋กœ ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. ์›Œ์ปค ์„ค์ •์ด ์ž˜ ๋˜์ง€ ์•Š์€ ๊ฒƒ์ฒ˜๋Ÿผ ๋ณด์ด๋„ค์š”.
ํ˜น์‹œ ์›Œ์ปค๋“ค์—๋„ ๋™์ผํ•œ tajo-site.xml์œผ๋กœ ์„ค์ •ํ•˜์…จ๋‚˜์š”?

์†์ง€ํ›ˆ ๋“œ๋ฆผ

2015๋…„ 8์›” 27์ผ (๋ชฉ) ์˜ค์ „ 3:02, Mellisa <goodga...@gmail.com>๋‹˜์ด ์ž‘์„ฑ:

Mellisa

unread,
Aug 27, 2015, 1:43:58โ€ฏAM8/27/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
๋‹ต๋ณ€ ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค.ย 

๋งˆ์Šคํ„ฐ์— ์žˆ๋Š” tajo-site.xml์„ ๋ณต์‚ฌํ•ด์„œ ์›Œ์ปค๋กœ ์˜ฎ๊ฒจ์„œ ์‚ฌ์šฉํ•ด๋„ ๊ฐ™์€ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.ย 
๊ทธ๋ž˜๋„ ์—ฌ์ „ํžˆ ์›น ui์˜ ๋…ธ๋“œ๋ถ€๋ถ„์— localhost๋ผ๊ณ  ๋ช…์‹œ๋ฉ๋‹ˆ๋‹ค.ย 

workers ์„ค์ •์ด๋ผ๊ณ  ํ•˜์‹œ๋ฉด,ย 
๋งˆ์Šคํ„ฐ ์„œ๋ฒ„์— workers ํŒŒ์ผ์— ๋งˆ์Šคํ„ฐ๋ฅผ ์ œ์™ธํ•œ ์›Œ์ปค์˜ ip์„ ์ ๋Š” ๊ฒƒ ์ด์™ธ์—๋„ ๋‹ค๋ฅธ ๊ฒƒ์ด ์žˆ๋‚˜์š”~?! ย 

2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

Mellisa

unread,
Aug 27, 2015, 2:34:35โ€ฏAM8/27/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์ถ”๊ฐ€๋กœย 
master log๋ฅผ ์‚ดํŽด๋ณด๋‹ˆย 
ERROR org.apache.tajo.master.rm.NodeStatus: Invalid event STATE_UPDATE on NodeStatus ย -1944742237
์—๋Ÿฌ ๋กœ๊ทธ๊ฐ€ ์ฐํ˜€์žˆ์—ˆ์Šต๋‹ˆ๋‹ค.ย 

์กฐ์–ธ๋ฐ›๋Š”๋ฐ ๋„์›€์ด ๋ ๊นŒ ํ•˜์—ฌ ์˜ฌ๋ ค๋ด…๋‹ˆ๋‹ค.ย 

๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค.


2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

์ •๊พธ

unread,
Aug 27, 2015, 3:01:05โ€ฏAM8/27/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์ œ๊ฐ€ ํ…Œ์ŠคํŠธํ•˜๋‹ค๋ณด๋ฉด ์ข…์ข… ์˜ค๋™์ž‘ํ• ๋•Œ ย stop-tajo ๋กœ ์ข…๋ฃŒํ–ˆ๋Š”๋ฐ ํ”„๋กœ์„ธ์Šค๊ฐ€ ์ข…๋ฃŒ๋˜์ง€ ์•Š๋Š” ๊ฒฝ์šฐ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.
stop-tajo ํ•˜์‹œ๊ณ  ๋งˆ์Šคํ„ฐ๋ž‘ ์›Œ์ปค๋…ธ๋“œ ๋ชจ๋‘ ๋Œ๋ฉด์„œ ์ˆ˜๋™์œผ๋กœ

ps -ef | grep tajo ํ•ด์„œ ํ”„๋กœ์„ธ์Šค ์ฒดํฌํ•œ๋ฒˆ ํ•ด๋ณด์‹œ๊ฒ ์–ด์š”?

Mellisa

unread,
Aug 27, 2015, 3:50:18โ€ฏAM8/27/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
๋‹ต๋ณ€ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค.ย 

๊ฐ ๋…ธ๋“œ์—์„œ ํ”„๋กœ์„ธ์Šค ํ™•์ธํ•ด๋ณด์•˜๋Š”๋ฐ ๋”ฐ๋กœ ๋Œ๊ณ ์žˆ๋Š” ๊ฒƒ์€ ์—†๋Š” ๊ฒƒ์„ ํ™•์ธํ–ˆ์Šต๋‹ˆ๋‹ค.ย 

๊ธฐ์กด์— ์„ค์น˜๋˜์–ด ์žˆ๋˜ tajo 0.10.1 ๋ฒ„์ „์œผ๋กœ ์‹คํ–‰ํ•ด๋ณด๋‹ˆย 
์›Œ์ปค์—์„œ ย 127.0.0.1:26003ย ์œผ๋กœ ์—ฐ๊ฒฐ๋˜๋Š” ๋ฌธ์ œ๋Š” ํ•ด๊ฒฐ์ด ๋œ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค ( ๊ธฐ์กด์— ์„ค์น˜ํ–ˆ๋˜ ๋ฒ„์ „๊ณผ ๋‹ค๋ฅธ ๊ฒƒ์ด ์—†๋Š”๋ฐ ํ•ด๊ฒฐ๋˜์„œ ์˜๋ฌธ์ด๊ธด ํ•ฉ๋‹ˆ๋‹ค๋งŒ..)ย 

ํ•˜์ง€๋งŒ, select count(*) from customer; ย ์ฟผ๋ฆฌ ์ˆ˜ํ–‰ ์‹œ ์›น ul์—์„œ ์—ฌ์ „ํžˆ query init() ๋ถ€๋ถ„์—์„œ ๋ฉˆ์ถฐ์žˆ๊ณ ย 
๊ฒฐ๊ณผ๊ฐ€ ๋‚˜์˜ค์ง€ ์•Š๋Š”๋ฐย 
์ด ๋ฌธ์ œ ๋˜ํ•œ ํ™˜๊ฒฝ์…‹ํŒ… ๋ฌธ์ œ์ผ ํ™•๋ฅ ์ด ๋†’์€๊ฐ€์š”?ย 

๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค!ย 


2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

Jihoon Son

unread,
Aug 27, 2015, 6:40:01โ€ฏAM8/27/15
to Mellisa, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”

์—ฌ์ „ํžˆ ์„ค์ •์— ๋ฌธ์ œ๊ฐ€ ์žˆ๋Š” ๊ฒƒ์ฒ˜๋Ÿผ ๋ณด์ž…๋‹ˆ๋‹ค.

๋งˆ์Šคํ„ฐ์™€ ์›Œ์ปค๊ฐ€ ๋™์ผํ•˜๊ฒŒ ์„ค์ •๋œ tajo-site.xml ์„ ๊ฐ–๊ณ  ์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋ฆฌ๊ณ  tajo-site.xml ํŒŒ์ผ์—๋Š” resolving์ด ๊ฐ€๋Šฅํ•œ ๋„คํŠธ์›Œํฌ ์ด๋ฆ„์ด ๋“ค์–ด์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ํƒ€์กฐ ์„ค์ • ํ›„์—๋Š” ์ „์ฒด ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ์žฌ์‹œ์ž‘ํ•˜์—ฌ์•ผํ•ฉ๋‹ˆ๋‹ค.

์†์ง€ํ›ˆ ๋“œ๋ฆผ
2015๋…„ 8์›” 27์ผ (๋ชฉ) ์˜คํ›„ 4:50, Mellisa <goodga...@gmail.com>๋‹˜์ด ์ž‘์„ฑ:

hyejong jeong

unread,
Aug 27, 2015, 6:42:59โ€ฏAM8/27/15
to Mellisa, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”!

2. ๊ธฐ์กด ํ”„๋กœ์„ธ์Šค๊ฐ€ ์ฃฝ์ง€ ์•Š๊ณ  ์‚ด์•„ ์žˆ์„๊ฒฝ์šฐ

์ •ํ˜œ์ข… ๋“œ๋ฆผ

2015๋…„ 8์›” 27์ผ ๋ชฉ์š”์ผ, Mellisa<goodga...@gmail.com>๋‹˜์ด ์ž‘์„ฑํ•œ ๋ฉ”์‹œ์ง€:

Mellisa

unread,
Aug 27, 2015, 1:29:34โ€ฏPM8/27/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์กฐ์–ธ ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค !!!!ย 

์„ค์ •์ด ๊ผฌ์ธ ๋ฌธ์ œ์ธ ๊ฒƒ ๊ฐ™์•„ ๋ชจ๋‘ ์ง€์šด ํ›„ย 
๋‹ค์‹œ ์„ค์น˜ํ•ด๋ณด์•˜์Šต๋‹ˆ๋‹ค.ย 

1. tajo trunk -0.11.0-SNAPSHOT ๋ฒ„์ „
2. tajo-0.10.1 ๋ฒ„์ „ย 

์œ„์˜ ๋‘ ๊ฐ€์ง€ ๋ฒ„์ „์— ๋Œ€ํ•ด ์…‹ํŒ… ํ›„ ์‹คํ–‰ํ•ด ๋ณธ ๊ฒฝ๊ณผย 

1๋ฒˆ์˜ ๊ฒฝ์šฐ์—๋Š” ์—ฌ์ „ํžˆ ์›Œ์ปค์—์„œ 26003 ํฌํŠธ ์—ฐ๊ฒฐ ์˜ค๋ฅ˜ ๋ฐœ์ƒ, select count(*) from table ์งˆ์˜ ์ˆ˜ํ–‰ ์‹œ ๋ฌดํ•œ ๋Œ€๊ธฐย 
2๋ฒˆ์€ ์ดˆ๊ธฐ์— ์—ฐ๊ฒฐ์˜ค๋ฅ˜ ๋ฐœ์ƒํ•˜๋‹ค๊ฐ€ ์ •์ƒ ์—ฐ๊ฒฐ ๋œ ๊ฒƒ์„ ํ™•์ธ, select count(*) from table ์งˆ์˜ ์ˆ˜ํ–‰ ์‹œ ๋ฌดํ•œ ๋Œ€๊ธฐย 

๋งˆ์Šคํ„ฐ์—์„œ ์˜ค๋ฅ˜ ๋ฉ”์‹œ์ง€๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.ย 
2015-08-27 22:16:25,714 INFO org.apache.tajo.master.rm.NodeStatus: -1944742237 Node Transitioned from NEW to RUNNING
2015-08-27 22:16:25,715 ERROR org.apache.tajo.master.rm.NodeStatus: Can't handle this event at current state, eventType:STATE_UPDATE, oldState:NEW, nextState:NEW
org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: STATE_UPDATE at NEW
ย  ย  ย  ย  at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
ย  ย  ย  ย  at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
ย  ย  ย  ย  at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
ย  ย  ย  ย  at org.apache.tajo.master.rm.NodeStatus.handle(NodeStatus.java:282)
ย  ย  ย  ย  at org.apache.tajo.master.rm.TajoResourceManager$WorkerEventDispatcher.handle(TajoResourceManager.java:135)
ย  ย  ย  ย  at org.apache.tajo.master.rm.TajoResourceManager$WorkerEventDispatcher.handle(TajoResourceManager.java:120)
ย  ย  ย  ย  at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
ย  ย  ย  ย  at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
ย  ย  ย  ย  at java.lang.Thread.run(Thread.java:744)
2015-08-27 22:16:25,716 ERROR org.apache.tajo.master.rm.NodeStatus: Invalid event STATE_UPDATE on NodeStatus ย -1944742237

ย 

๋กœ๊ทธ๋ฅผ ๋ณด์•˜์„ ๋•Œ ์—ฌ์ „ํžˆ ์›Œ์ปค์™€์˜ ํ†ต์‹ ์ด ์ž˜ ์•ˆ๋˜๋Š” ๋“ฏ ๋ณด์ž…๋‹ˆ๋‹ค.ย 

๊ทธ๋Ÿฐ๋ฐ ์—ฌ๊ธฐ์„œ ย ๊ถ๊ธˆ์ฆ์ด ์ƒ๊ฒจ ์งˆ๋ฌธ๋“œ๋ฆฝ๋‹ˆ๋‹ค.ย 

์งˆ๋ฌธ 1. ๋งŒ์•ฝ hdfs์˜ ๋ฐ์ดํ„ฐ๋…ธ๋“œ ํ”„๋กœ์„ธ์Šค๊ฐ€ ๋– ์žˆ๊ธด ํ•˜์ง€๋งŒ ์ •์ƒ์ ์œผ๋กœ ย ๋“ฑ๋ก๋˜์ง€ ์•Š์€ ๊ฒฝ์šฐ๋ผ๋ฉดย 
ํƒ€์กฐ์˜ ๊ธฐ๋ณธ ์„ค์ •์ด ์ œ๋Œ€๋กœ ๋˜์–ด ์žˆ์–ด๋„ย 
./start-tajo.sh ์Šคํฌ๋ฆฝํŠธ ์ˆ˜ํ–‰ํ–ˆ์„ ๋•Œ ์œ„์˜ ์ œ์‹œ๋œ ํƒ€์กฐ ์›Œ์ปค ์—ฐ๊ฒฐ ์˜ค๋ฅ˜, master ๋กœ๊ทธ์™€ ๊ฐ™์€ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•  ์—ฌ์ง€๊ฐ€ ์žˆ๋Š” ๊ฑด๊ฐ€์š”? (๋ฐ์ดํ„ฐ๋…ธ๋“œ์™€ ์ƒ๊ด€์—†์ด ํƒ€์กฐ ์‹คํ–‰ ์ž์ฒด๋Š” ์ •์ƒ์ ์œผ๋กœ ๋Œ์•„๊ฐ€์•ผ ํ•œ๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ๋Š”๋ฐ , ์ž์› ๊ด€๋ จํ•ด์„œ ๊ด€๊ณ„๊ฐ€ ์žˆ๋‚˜ ์‹ถ์–ด์„œ์š”..)ย 

์งˆ๋ฌธ 2. ์˜คํ”ˆ ์†Œ์Šค ๊ฐœ๋ฐœ ์ดˆ๋ณด์ž์ž…๋‹ˆ๋‹ค. ย ํƒ€์กฐ ์†Œ์Šค ์ฝ”๋“œ๋ฅผ ๋ฐ›์•„์„œ ์ˆ˜์ • ํ–ˆ์„ ๊ฒฝ์šฐ์— ์ˆ˜์ • ๋ถ€๋ถ„ ๊ฒ€์ฆ์„ ์œ„ํ•ด์„œ๋Š” ์ปดํŒŒ์ผ ํ™•์ธ ํ›„ ๋ฐ”์ด๋„ˆ๋ฆฌ๋กœ ๋งŒ๋“ค์–ด ์ง์ ‘ ์‹คํ–‰ํ•ด๋ณด๋Š” ๋ฐฉ๋ฒ• ์ด์™ธ์— ๋‹ค๋ฅธ ๋ฐฉ์•ˆ์ด ์žˆ์„๊นŒ์š”? (์ฝ”๋“œ ๋‚ด์—์„œ ์ „์ฒด ํ”„๋กœ๊ทธ๋žจ์„ ํ…Œ์ŠคํŠธ ํ•ด๋ณด๊ธฐ ๋“ฑ๋“ฑ,, ) ๊ฐœ๋ฐœ ํ™˜๊ฒฝ์ด ๋ฏธ์ˆ™ํ•ด์„œ ์กฐ์–ธ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค ย 

๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค ! ย 

์ด์ „ ๊ฒŒ์‹œ๊ธ€์—์„œย 

2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

hyejong jeong

unread,
Aug 27, 2015, 3:38:35โ€ฏPM8/27/15
to Mellisa, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š” ^^

์˜ค๋Š˜ ํƒ€์กฐ 8์›” ๋ฐ‹์—…์—์„œ ์†์ง€ํ›ˆ ๋ฐ•์‚ฌ๋‹˜์„ ๋งŒ๋‚˜ ๋’ทํ’€์ด ์‹œ๊ฐ„์—ย ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜์˜€์Šต๋‹ˆ๋‹ค.ย 

์ € ๊ฐ™์€ ๊ฒฝ์šฐ๋Š” aws 1cpu๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฆฌ์†Œ์Šค ๋ถ€์กฑ์ธ case ์˜€์Šต๋‹ˆ๋‹ค.ย 

ํƒ€์กฐ ๋งˆ์Šคํ„ฐ๊ฐ€ 1cpu๋ฅผ ํƒ€์ €์›Œ์ปค์˜ ์ฟผ๋ฆฌ ๋งˆ์Šคํ„ฐ๊ฐ€ 1cpu๋ฅผ, ํƒ€์กฐ ์›Œ์ปค๊ฐ€ 1cpu๋ฅผ ์‚ฌ์šฉํ•˜๊ธฐ ๋•Œ๋ฌธ์—
๋ฆฌ์†Œ์Šค๊ฐ€ ๋ถ€์กฑํ•œ ๊ฒฝ์šฐ์˜€์Šต๋‹ˆ๋‹ค.ย 

๊ทธ๋ž˜์„œ ์ €๋Š” ํƒ€์กฐ ๋งˆ์Šคํ„ฐ ์™€ ์›Œ์ปค 2๊ฐœ์˜ ์„œ๋ฒ„๋ฅผ ์ด์šฉํ•˜์—ฌ ์•„๋ž˜ 3๋ฒˆ ์ฒ˜๋Ÿผ ์ง„ํ–‰์„ ํ•˜์˜€์Šต๋‹ˆ๋‹ค.ย 

์—ฌ์ „ํžˆ ๊ฐ™์€ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•˜์˜€์ง€๋งŒ, ๋’ทํ’€์ด์—์„œ ์†๋ฐ•์‚ฌ๋‹˜์˜ ๊ธฐ์ˆ ์ง€์›(์˜๊ด‘์Šค๋Ÿฝ๊ฒŒ๋„)์„ ๋ฐ›์•„ ๋ฆฌ์†Œ์Šค ๋ฌธ์ œ์ž„์„ ์•Œ๊ฒŒ ๋˜๊ณ  conf/workers ํŒŒ์ผ์— ํƒ€์กฐ ๋งˆ์Šคํ„ฐ ์›Œ์ปค ๋…ธ๋“œ๋ฅผ ์ถ”๊ฐ€ํ•˜์—ฌ ์‹คํ–‰ํ•˜๋‹ˆ, ๋ฌธ์ œ๊ฐ€ ํ•ด๊ฒฐ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.ย 

์‚ฌ์ง„์„ ํ•จ๊ป˜ ์ฒจ๋ถ€ํ•ฉ๋‹ˆ๋‹ค.ย 

์ฟผ๋ฆฌ ๊ฒฐ๊ณผ์— ์˜ค๋ฅ˜๊ฐ€ ๋‚˜๋Š”๊ฑด ๋”๋น„๋””๋น„๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋‚˜๋Š” ๋ฌธ์ œ์ž…๋‹ˆ๋‹ค.ย 


1. tajo-0.11.0-desktop ๋ฒ„์ „์€ ์ •์ƒ์ ์œผ๋กœ ์ž˜ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋ž˜์„œ ๋ฆฌ์†Œ์Šค๋Š” ์ƒ๊ด€์ด ์—†๋Š”์ค„ ์•Œ์•˜์Šต๋‹ˆ๋‹ค.ย 


2.ย tajo-0.11.0-SNAPSHOT ๋ฒ„์ „์œผ๋กœ ์Šคํƒ ๋“œ์–ผ๋ก (Standalone)์œผ๋กœ ํ…Œ์ŠคํŠธ ํ•˜๋‹ค๊ฐ€ Mellisa๋‹˜ ๊ณผ ๊ฐ™์€ ํ˜„์ƒ์ด ๋ฐœ์ƒํ•˜์—ฌ, ํƒ€์กฐ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน์— ๋ฌธ์˜ํ•˜๋‹ค๊ฐ€
์•„๋ž˜ 3๋ฒˆ๊ณผ ๊ฐ™์ด ํ…Œ์ŠคํŠธ ํ™˜๊ฒฝ์„ ๋ณ€๊ฒฝํ•˜์˜€์Šต๋‹ˆ๋‹ค.ย 


3. tajo-0.12.0-SNAPSHOT ๋ฒ„์ „์œผ๋กœ ๋ณ€๊ฒฝํ•˜์—ฌย 
๋งˆ์Šคํ„ฐ ์™€ ์›Œ์ปค 2๋Œ€์˜ ์„œ๋ฒ„๋ฅผ ์‚ฌ์šฉํ•˜์˜€์Šต๋‹ˆ๋‹ค.ย 


2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ, Mellisa<goodga...@gmail.com>๋‹˜์ด ์ž‘์„ฑํ•œ ๋ฉ”์‹œ์ง€:
IMG_2655.jpg
IMG_2656.jpg
IMG_2657.jpg
IMG_2658.jpg
IMG_2659.jpg
IMG_2660.jpg

Jinho Kim

unread,
Aug 27, 2015, 9:52:07โ€ฏPM8/27/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”.

1๋ฒˆ ์งˆ๋ฌธ์€ hadoop ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์™€ ์„ค์ •์„ tajo ๊ฐ€ ์ฐธ์กฐํ•˜๋Š”๊ฒƒ ์™ธ์—๋Š” ๋ณ„๋„๋กœ ์‹คํ–‰๋ฉ๋‹ˆ๋‹ค.
๊ทธ๋Ÿฌ๋‚˜ HDFS ๊ฐ€ ์ •์ƒ ์‹คํ–‰๋˜์ง€ ์•Š๋Š”๋‹ค๋ฉด tajo ์ฟผ๋ฆฌ์˜ ๊ฒฐ๊ณผ ํŒŒ์ผ์„ ์ €์žฅํ• ์ˆ˜ ์—†๊ธฐ๋•Œ๋ฌธ์— ์‚ฌ์šฉํ• ์ˆ˜ ์—†๊ณ 
HDFS ์—†์ด ๋กœ์ปฌํŒŒ์ผ๋กœ 1๋Œ€๋งŒ ์‹คํ–‰ํ•˜์‹ค์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

2๋ฒˆ์€ย BUILDING ํŒŒ์ผ์„ ๋ณด์‹œ๋ฉด ์ƒ์„ธํ•˜๊ฒŒ ์ ํ˜€์žˆ์Šต๋‹ˆ๋‹ค. ๊ฐ„๋‹จํ•˜๊ฒŒ mvn clean install ๋งŒ ํ•˜์…”๋„ ์ „์ฒด ย test ๋ฅผ ๋Œ๋ ค๋ณด์‹ค์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

tajo-site.xml, tajo-env.sh, workers, /etc/hosts ๋ฅผ ๋งŒ๊ฐํ•œ ๋ถ€๋ถ„(ip ๋˜๋Š” ํ˜ธ์ŠคํŠธ๋„ค์ž„) ์€ xx ๋กœ ํ•˜์…”์„œ ํ•œ๋ฒˆ ์ฒจ๋ถ€ํ•ด ์ฃผ์‹œ๋ฉด ์‚ดํŽด๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.
Meetup์— ํ•œ๋ฒˆ ์˜ค์…”์„œ ๊ถ๊ธˆํ•˜์‹ ๋ถ€๋ถ„ ์งˆ๋ฌธ ํ•˜์‹œ๋Š”๊ฒƒ๋„ ์ข‹์„๋“ฏ ํ•ฉ๋‹ˆ๋‹ค.

2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ ์˜ค์ „ 2์‹œ 29๋ถ„ 34์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:

Jinho Kim

unread,
Aug 27, 2015, 10:16:34โ€ฏPM8/27/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน, goodga...@gmail.com
์•ˆ๋…•ํ•˜์„ธ์š”.

์„ค์ •ํ•˜์‹  ๋ถ€๋ถ„์ค‘ ๋กœ์ปฌํŒŒ์ผ์‹œ์Šคํ…œ ์€ ๋ถ„์‚ฐ๋ชจ๋“œ์—์„œ ์‚ฌ์šฉํ•˜์‹ค์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.ย 
s3 ๋ฅผ ์‚ฌ์šฉํ•˜์‹ ๋‹ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์ด ๋ฐ”๊ฟ”์ฃผ์…”์•ผ ์ •์ƒ์ ์œผ๋กœ ๊ฒฐ๊ณผ๋ฅผ ์ €์žฅํ•˜์‹ค์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
alter tablespace set location 's3://location/tajo/warehouse';ย 

๊ทธ๋ฆฌ๊ณ  ์„ค์ •์ค‘ ์ธ์Šคํ„ด์Šค๊ฐ€ cpu1 ๊ฐœ๋ฉด ์•„๋ž˜ ์„ค์ •์œผ๋กœ ๊ฐ•์ œ๋กœ ๋ณ€๊ฒฝํ•˜์‹ค์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
tajo.worker.resource.cpu-cores
์ž์›์„ ์ตœ๋Œ€ํ•œ ํ™œ์šฉํ•˜์‹ค๋ ค๋ฉด tajo-env.sh ์— worker heap ๋„ ์„ค์ •ํ•˜์‹œ๋Š”๊ฒƒ์ด ์ข‹์Šต๋‹ˆ๋‹ค.

2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ ์˜ค์ „ 4์‹œ 38๋ถ„ 35์ดˆ UTC+9, hyejong jeong ๋‹˜์˜ ๋ง:

hyejong jeong

unread,
Aug 27, 2015, 10:19:36โ€ฏPM8/27/15
to Jinho Kim, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š” ^^

1. HDFS ํŒŒ์ผ ์—†์ด ๋กœ์ปฌ ํŒŒ์ผ๋กœ ๋กœ์ปฌ ์„œ๋ฒ„ 1๋Œ€๋กœ
Apache Tajo๋ฅผ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ๋‹ค๋Š” ๊ฒƒ์€ย 

1-1. ์„œ๋ฒ„ ์‚ฌ์–‘ :ย 2์ฝ”์–ด์ด์ƒ์˜ ์„œ๋ฒ„ 1๋Œ€
ย  ย  ย  ย  (2์ฝ”์–ด ์ด์ƒ์˜ ์„œ๋ฒ„๊ฐ€ ์—†์„๋•Œ๋Š” ํ™˜๊ฒฝ์„ค์ •์—์„œ ์ฝ”์–ด์ˆ˜๋ฅผ 2๋กœ ์…‹ํŒ…ํ•˜๋Š” ํŽธ๋ฒ•์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. )

1-2. ๋””๋น„๋Š” derby db๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.ย 

1-3. Workers ํŒŒ์ผ์— ๋กœ์ปฌํ˜ธ์ŠคํŠธ๋ฅผ ์ถ”๊ฐ€ํ•œ๋‹ค.ย 

์œ„์™€ ๊ฐ™์ด ํ…Œ์ŠคํŠธํ•ด๋ณด๊ณ  ๊ฒฐ๊ณผ๋ฅผ ํ”ผ๋“œ๋ฐฑ ๋“œ๋ฆฌ๊ฒ ์Šต๋‹ˆ๋‹ค.ย 

์—ฌ๊ธฐ์„œ ์ž ๊น) ํƒ€์กฐ๋Š” ์ฒจ๋ถ€ํ•œ ํŒŒ์ผ๊ณผ ๊ฐ™์ดย 
Tajo Master
Query Master
Node๋กœ ๊ตฌ์„ฑ๋˜์–ด ์žˆ๊ธฐ ๋•Œ๋ฌธ์— ์ตœ์†Œ 2๊ฐœ์˜ cpu๊ฐ€ ํ•„์š”ํ•˜๋‹ค. (์ž์„ธํ•œ ์„ค๋ช…์€ ๋ฐ‹์—…์—์„œ ์ง„ํ˜ธ๋‹˜์—๊ฒŒ ๋“ฃ๋Š”๋‹ค ^^)

์ •ํ˜œ์ข… ๋“œ๋ฆผ

2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ, Jinho Kim<jin...@gmail.com>๋‹˜์ด ์ž‘์„ฑํ•œ ๋ฉ”์‹œ์ง€:
IMG_2656.jpg

hyejong jeong

unread,
Aug 28, 2015, 12:20:10โ€ฏAM8/28/15
to Jinho Kim, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน, ๊น€๋ฏผ๊ฒฝ
์•ˆ๋…•ํ•˜์„ธ์š”!

๊น€์ง„ํ˜ธ๋‹˜์ด ๋ง์”€ํ•ด์ฃผ์‹  ๊ฒƒ์ฒ˜๋Ÿผ ์ธ์Šคํ„ด์Šค(worker)์˜ cpu ๋ฅผ 2๋กœ ์„ค์ •ํ•˜๊ณ  ์žฌ์‹œ์ž‘ํ•˜์˜€๋”๋‹ˆ ์•„๋ž˜์™€ ๊ฐ™์ด ์ฟผ๋ฆฌ์˜ ๊ฒฐ๊ณผ๋ฅผ ๊ฐ€์ง€๊ณ  ์™”์Šต๋‹ˆ๋‹ค.

1. master ์™€ worker์˜ tajo-site.xml ํŒŒ์ผ์„ ์•„๋ž˜์™€ ๊ฐ™์ด ์ˆ˜์ •ํ•˜์˜€์Šต๋‹ˆ๋‹ค.

๋ณธ๋ฌธ ์ด๋ฏธ์ง€ 3

2. ํƒ€์กฐ ์„œ๋น„์Šค๋ฅผ ์žฌ์‹œ์ž‘ํ•˜๊ณ , ๋‹ค์‹œ ์ฟผ๋ฆฌ๋ฅผ ์‹คํ–‰ํ•ด ๋ณด์•˜์Šต๋‹ˆ๋‹ค.
--> ๊ฒฐ๊ณผ : ์„ฑ๊ณต

๋ณธ๋ฌธ ์ด๋ฏธ์ง€ 1


3. Web UI ์—์„œ ์ฟผ๋ฆฌ๊ฐ€ ์ •์ƒ์ ์œผ๋กœ์™„๋ฃŒ๋œ๊ฒƒ๋„ ํ™•์ธ ํ• ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. 4์ดˆ๊ฐ€ ๊ฑธ๋ ค์Šต๋‹ˆ๋‹ค.
๋ณธ๋ฌธ ์ด๋ฏธ์ง€ 2

์ •ํ˜œ์ข… ๋“œ๋ฆผ

2015๋…„ 8์›” 28์ผ ์˜ค์ „ 11:16, Jinho Kim <jin...@gmail.com>๋‹˜์ด ์ž‘์„ฑ:

Mellisa

unread,
Aug 28, 2015, 3:44:03โ€ฏAM8/28/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
2๊ฐ€์ง€ ์งˆ๋ฌธ์— ๋Œ€ํ•œ ๋‹ต๋ณ€ ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค.ย 

tajo-site.xmlย 
tajo-env.sh๋ฅผ ์ฒจ๋ถ€ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.ย 
๊ฐ€๋Šฅํ•˜์‹œ๋‹ค๋ฉด ์‚ดํŽด๋ด์ฃผ์‹œ๋ฉด ๊ฐ์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.ย 

worker๋Š” ๋”ฐ๋กœ ์ฒจ๋ถ€ํ•˜์ง€ ์•Š์•˜๋Š”๋ฐย 

ํŒŒ์ผ์„ ์—ด๋ฉด
165.xxx.xxx.xxx
165.xxx.xxx.xxx
165.xxx.xxx.xxx

์œ„์™€ ๊ฐ™์ด ๋งˆ์Šคํ„ฐ๋ฅผ ์ œ์™ธํ•œ 3๊ฐœ์˜ ip์ฃผ์†Œ๋ฅผ ์ ์–ด๋†“์•˜์Šต๋‹ˆ๋‹ค.ย 


host1 : master
host2,3,4 : worker

ํ˜„์žฌ ์ƒํƒœ๋Š”ย 
ํƒ€์กฐ๋ฅผ ์‹คํ–‰ํ•˜๋ฉด
์„œ๋ฒ„์—์„œ master์™€ worker ํ”„๋กœ์„ธ์Šค๊ฐ€ ย ๋ชจ๋‘ ๋œฌ ๊ฒƒ์ด ํ™•์ธ๋ฉ๋‹ˆ๋‹ค.ย 

๊ทธ๋Ÿฐ๋ฐ ๋ฌธ์ œ๊ฐ€ ๋˜๋Š” ๋ถ€๋ถ„์ด master๋ฅผ ํฌํ•จํ•˜์—ฌ ์ด 4๊ฐœ์˜ ๋…ธ๋“œ์ด๋ฏ€๋กœย 
๋งˆ์Šคํ„ฐ1๊ฐœ ์›Œ์ปค 2๊ฐœ ์ฟผ๋ฆฌ๋งˆ์Šคํ„ฐ 1๊ฐœ ์ด๋ ‡๊ฒŒ ์›น ui์— ํ‘œ์‹œ๋˜์–ด์•ผ ํ•˜๋Š”๋ฐ
์›Œ์ปค๊ฐ€ 1๊ฐœ๋งŒ ์ธ์‹๋˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.ย 
๋˜ํ•œ ๋…ธ๋“œ์˜ ์ฃผ์†Œ๊ฐ€ localhost๋กœ ์ธ์‹๋ฉ๋‹ˆ๋‹ค.
์‚ฌ์ง„ ์ฒจ๋ถ€ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.ย 



ย 
์—ญ์‹œ๋‚˜ ์„ค์ •๋ฌธ์ œ์ธ ๊ฒƒ ๊ฐ™์€๋ฐ....
๊ด€๋ จํ•ด์„œ ์กฐ์–ธ ๋ฐ›์„ ์ˆ˜ ์žˆ์„๊นŒ์š”?ย 
ย 



2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ ์˜ค์ „ 10์‹œ 52๋ถ„ 7์ดˆ UTC+9, Jinho Kim ๋‹˜์˜ ๋ง:
tajo-site.xml
tajo-env.sh

Mellisa

unread,
Aug 28, 2015, 3:47:53โ€ฏAM8/28/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์šฐ์„  ๋…ธ๋“œ๋ฅผ ์ œ๋Œ€๋กœ ์ •์ƒ๋™์ž‘ ์‹œํ‚จ ํ›„์—ย 
์œ„์—์„œ ์กฐ์–ธ ์ฃผ์…จ๋˜ ์ฟผ๋ฆฌ ์ˆ˜ํ–‰ ์‹œ ๋ฌดํ•œ ๋Œ€๊ธฐ์ƒํƒœ ๋ฌธ์ œ์— ๋Œ€ํ•ด ๋ณ€๊ฒฝ ์‚ฌํ•ญ์„ ์ ์šฉํ•ด์„œ ๋‹ค์‹œ ์‹คํ—˜ ํ•  ์˜ˆ์ •์ž…๋‹ˆ๋‹ค.ย 
์„ฑ๊ณตํ•˜๋ฉด ๊ฒฐ๊ณผ๋ฅผ ์ •๋ฆฌํ•ด์„œ ๊ณต์œ ํ•˜๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค!ย 
ย 

2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

Jinho Kim

unread,
Aug 28, 2015, 4:17:45โ€ฏAM8/28/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”.

ํ˜น์‹œ /etc/hosts ํŒŒ์ผ๋‚ด์— 165.xxx.xxx.xxx localhost ๊ฐ™์€ ๋ผ์ธ์ด ์žˆ๋Š”๊ฑด ์•„๋‹๊นŒ์š”?
๊ทธ๋ฆฌ๊ณ  IP ๋ฅผ ์ง์ ‘ ์‚ฌ์šฉํ•˜์‹œ๋Š”๊ฒƒ ๋ณด๋‹ค hostname ์„ ์„œ๋ฒ„๋ณ„๋กœ ๋“ฑ๋กํ•˜์‹œ๊ณ  ์‚ฌ์šฉํ•˜๋Š”๊ฒŒ ์ข‹์Šต๋‹ˆ๋‹ค.
tajo ๋Š” HDFS ์—์„œ block location ์„ ๊ฐ€์ ธ์™€ ๊ฐ™์€ ํ˜ธ์ŠคํŠธ์ธ์ง€ ๊ฒ€์‚ฌํ•˜๊ณ  locality ๋ฅผ ๋†’์ด๊ฒŒ ๋˜๋Š”๋ฐ
๋„คํŠธ์›Œํฌ ์ „์†ก์ด ์ค„์–ด์„œ scan ์ด ๋” ๋น ๋ฅด๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ ์˜คํ›„ 4์‹œ 47๋ถ„ 53์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:

Jihoon Son

unread,
Aug 28, 2015, 5:10:54โ€ฏAM8/28/15
to Jinho Kim, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”

workers์— ์›Œ์ปค ๋…ธ๋“œ๋“ค์˜ ip ํ˜น์€ hostname์ด ์ •ํ™•ํ•˜๊ฒŒ ๋“ค์–ด์žˆ๋‹ค๋ฉด ์›น ui์˜ cluster ํŽ˜์ด์ง€์— ์›Œ์ปค ์ฃผ์†Œ๊ฐ€ localhost๋ผ๊ณ  ๋œจ๋ฉด ์•ˆ๋ฉ๋‹ˆ๋‹ค.
์›น ui์— ์›Œ์ปค๊ฐ€ localhost๋ผ๊ณ  ๋œจ๋ฉด, ์›Œ์ปค๊ฐ€ ๋งˆ์Šคํ„ฐ ๋…ธ๋“œ์— ์‹คํ–‰๋˜์—ˆ๋‹ค๊ณ  ์ž˜๋ชป ์ธ์‹๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๋งˆ์Šคํ„ฐ์™€ ์›Œ์ปค ๋กœ๊ทธ๋ฅผ ์˜ฌ๋ ค์ฃผ์‹œ๋ฉด ๋ฌธ์ œ ํŒŒ์•…์— ๋„์›€์ด ๋  ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

์†์ง€ํ›ˆ ๋“œ๋ฆผ

2015๋…„ 8์›” 28์ผ (๊ธˆ) ์˜คํ›„ 5:17, Jinho Kim <jin...@gmail.com>๋‹˜์ด ์ž‘์„ฑ:

Mellisa

unread,
Aug 28, 2015, 5:14:45โ€ฏAM8/28/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
๊ฐ ์„œ๋ฒ„์˜ /etc/hosts์—๋Š”ย 
127.0.0.1 ย  localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 ย  ย  ย  ย  localhost localhost.localdomain localhost6 localhost6.localdomain6

๊ฐ€ ์ €์žฅ๋˜์–ด์žˆ๋Š” ์ƒํƒœ์˜€์Šต๋‹ˆ๋‹ค.ย 
hosts ์„ค์ •์„ ๊ฑด๋“œ๋ ค๋ณธ์ ์ด ์—†์–ด์„œ
๋ง์”€ํ•ด์ฃผ์‹  ๋Œ€๋กœ ์ˆ˜์ •ํ•ด๋ณธ ํ›„ ๋‹ค์‹œ ๋Œ๋ ค๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.
๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค!ย 

2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ ์˜คํ›„ 5์‹œ 17๋ถ„ 45์ดˆ UTC+9, Jinho Kim ๋‹˜์˜ ๋ง:

hyejong jeong

unread,
Aug 28, 2015, 6:38:00โ€ฏAM8/28/15
to Mellisa, Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š” ^^

์›น ui๋ฅผ ๋ณด๋ฉด master ์„ค์ •์€ ์ž˜ ๋˜์–ด ์žˆ๋Š”๋ฐ, ์›Œ์ปค์™€ ์ฟผ๋ฆฌ ๋งˆ์Šคํ„ฐ๊ฐ€ ํ•˜๋‚˜์ธ๊ฑฐ๋ฅผ ๋ณด๋ฉด master ์„œ๋ฒ„์˜ workers ํŒŒ์ผ์ด localhost๋กœ ์„ค์ •๋˜์–ด ์žˆ์ง€๋Š” ์•Š๋Š”์ง€ ํ™•์ธ์ด ํ•„์š”ํ• ๊ฒƒ ๊ฐ™๋‹ค๋Š” ์ƒ๊ฐ์„ ์กฐ์‹ฌ์Šค๋Ÿฝ๊ฒŒ ํ•ด๋ด…๋‹ˆ๋‹ค.ย 

Master server /conf ๋ฐ‘์˜ workers ํŒŒ์ผ์„ ๋ณผ์ˆ˜ ์žˆ์„๊นŒ์š”?

์ฒจ๋ถ€ํ•œ ์ด๋ฏธ์ง€์™€ ๊ฐ™์ด ์›Œ์ปค๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ย ๋“ฑ๋ก์ด ๋˜๋ฉดย 
QueryMasters 2๊ฐœ, node 2๊ฐœ๋กœ ๋‚˜ํƒ€๋‚ฉ๋‹ˆ๋‹ค.ย 

์ €๊ฐ™์€ ๊ฒฝ์šฐ๋Š” ์„œ๋ฒ„๋Š” 2๋Œ€์ธ๋ฐ, workers์— ๋งˆ์Šคํ„ฐ ip๋„ ์ถ”๊ฐ€ํ•˜์—ฌ ์ž์„ธํžˆ ๋ณด์‹œ๋ฉด ์›Œ์ปค์™€ ๋งˆ์Šคํ„ฐ ์•„์ดํ”ผ๊ฐ€ ๊ฒน์น˜๋Š” ๊ฒƒ์„ ๋ณด๊ธธ์ˆ˜ ์žˆ์„๊ฒ๋‹ˆ๋‹ค.ย 

์ •ํ˜œ์ข… ๋“œ๋ฆผ

2015๋…„ 8์›” 28์ผ ๊ธˆ์š”์ผ, Mellisa<goodga...@gmail.com>๋‹˜์ด ์ž‘์„ฑํ•œ ๋ฉ”์‹œ์ง€:
2๊ฐ€์ง€ ์งˆ๋ฌธ์— ๋Œ€ํ•œ ๋‹ต๋ณ€ ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค.ย 
IMG_2656.jpg

Mellisa

unread,
Aug 29, 2015, 1:32:31โ€ฏAM8/29/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”. ์œ„์˜ ๋งŽ์€ ๋‹ต๋ณ€๋“ค์„ ์ฐธ๊ณ ํ•˜์—ฌ ๋‹ค์‹œ ์ˆ˜ํ–‰ํ•ด๋ณธ ๊ฒฐ๊ณผ ย ๋“œ๋””์–ด ! ์ •์ƒ ๋™์ž‘์— ์„ฑ๊ณตํ•˜์˜€์Šต๋‹ˆ๋‹ค.ย 

์ด 4๋Œ€์˜ ์„œ๋ฒ„๋กœ ๊ตฌ๋™์‹œํ‚จ ๊ฒฐ๊ณผ ๋งˆ์Šคํ„ฐ์™€ ์›Œ์ปค ์ •์ƒ์ ์œผ๋กœ ์ธ์‹๋˜๋Š” ๊ฒƒ์„ ํ™•์ธํ•˜์˜€์Šต๋‹ˆ๋‹ค.ย 


๋‹ค์Œ์€ ์ฟผ๋ฆฌ ์ˆ˜ํ–‰ ์‹œ ๋ฌดํ•œ ๋Œ€๊ธฐ ์ƒํƒœ ์˜€๋˜ ๋ถ€๋ถ„์„ ๋‹ค์‹œ ํ…Œ์ŠคํŠธ ํ•œ ๊ฒฐ๊ณผ์ž…๋‹ˆ๋‹ค.ย 

์ •์ƒ๋™์ž‘ํ•ฉ๋‹ˆ๋‹ค ^^ !




๋ฌธ์ œ๊ฐ€ ๋˜์—ˆ๋˜ ๋ถ€๋ถ„๊ณผ ํ•ด๊ฒฐ๋ฒ•์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.ย 
์šฐ์„  ์ œ๊ฐ€ ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ๊ตฌ์ถ•ํ•˜๋‹ค๋ณด๋‹ˆ, ์•„์ง ๋ฏธ์ˆ™ํ•œ ๋ถ€๋ถ„์ด ๋งŽ์•˜์Šต๋‹ˆ๋‹ค. ์•„์ฃผ ๊ธฐ์ดˆ์ ์ธ ๋ถ€๋ถ„์—์„œ ๋ฌธ์ œ๊ฐ€ ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค. ใ… ใ… 
๊ทธ๋ž˜๋„ ์ €์ฒ˜๋Ÿผ ์ดˆ๋ณด์ž๋ฅผ ์œ„ํ•ด ๋งŒ์•ฝ tajo-env.sh์™€ tajo-site.xml์˜ ๋ฌธ์ œ๊ฐ€ ์—†๋‹ค๊ณ  ํŒ๋‹จ๋˜์‹œ๋ฉด ์•„๋ž˜์˜ ๋‚ด์šฉ์„ ํ™•์ธํ•ด๋ณด์‹œ๊ธฐ ๋ฐ”๋ž๋‹ˆ๋‹ค !!ย 

1. worker ์„œ๋ฒ„์— tajoworker ํ”„๋กœ์„ธ์Šค๋Š” ๋– ์žˆ๋Š”๋ฐ ์›น ui ๋…ธ๋“œ์— localhost ๋ผ๊ณ  ์ถœ๋ ฅ๋˜๋˜ ๋ฌธ์ œ์˜ ์›์ธ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค. (์ €์ฒ˜๋Ÿผ ์„œ๋ฒ„ ๊ตฌ์ถ•์— ๋ฏธ์ˆ™ํ•˜์‹  ๋ถ„๋“ค ์ฐธ๊ณ ํ•˜์„ธ์š”) ย 

ย  ย => ์ด 4๋Œ€์˜ ์„œ๋ฒ„์—์„œ ๊ฐ๊ฐ hostname ๋ช…๋ น์–ด๋ฅผ ํƒ€์ดํ•‘ ํ•ด๋ณด๋‹ˆ, ๋ณ„์นญ์ด ๋ชจ๋‘ย localhost.localdomainย ๋ผ๊ณ  ๋“ฑ๋ก์ด ๋˜์–ด ์žˆ๋˜ ๋ฌธ์ œ์˜€์Šต๋‹ˆ๋‹ค.ย 
ย  ย  ย  ย centos ์‚ฌ์šฉ์ค‘์ธ๋ฐ hostname ์ดˆ๊ธฐ ๊ฐ’์ดย ย localhost.localdomain ์œผ๋กœ ๋“ฑ๋ก๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. ย 
ย  ย  ย  ย ์ฆ‰ย ์›Œ์ปค๊ฐ€ ์ •์ƒ ๋“ฑ๋ก ๋˜์–ด ์žˆ์—ˆ๋Š”๋ฐ ์ถœ๋ ฅ์ด localhost๋กœ ๋˜์—ˆ๋˜ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค
ย  => ์ด ๋ถ€๋ถ„์„ย vi /etc/hostname ๋ฅผ ํ†ตํ•ด ๊ฐ ์„œ๋ฒ„์— ๋ณ„์นญ์ธ master1, worker1, worker2,worker3 ๋“ฑ์œผ๋กœ ์ €์žฅ ํ›„ ์‹คํ–‰ํ•ด๋ณด๋‹ˆ ์›น ui์— ์ •์ƒ์ ์œผ๋กœ NODE ์ •๋ณด๊ฐ€ ์ถœ๋ ฅ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.ย 
ย  ย  ย  ย 
2. workers์— ์›๋ž˜ ip๋ฅผ ๊ธฐ์ž…ํ–ˆ์—ˆ๋Š”๋ฐ, ์œ„์˜ ์กฐ์–ธ๋“ค์„ ๋ณด๋‹ˆ host๋ช…์œผ๋กœ ํ•˜๋Š”๊ฒŒ ์ข‹์„ ๊ฒƒ ๊ฐ™์•„์„œ ๋‹ค์‹œ /etc/hosts๋ฅผ ์ˆ˜์ • ํ•œ ํ›„, workers์— host๋ช…์„ ๊ธฐ์ž…ํ–ˆ์Šต๋‹ˆ๋‹ค. ย ย 
ย  ย  host๋ช…์„ ์ ๋Š” ๊ฒƒ์„ ์ถ”์ฒœํ•ฉ๋‹ˆ๋‹ค.ย 

3. tsql ์—์„œ ์ฟผ๋ฆฌ ์ˆ˜ํ–‰ํ–ˆ๋Š”๋ฐ ๋ฌดํ•œ ๋Œ€๊ธฐ ํ˜„์ƒ => ์œ„์˜ ์กฐ์–ธ๋“ค์ฒ˜๋Ÿผ resource ๋ฌธ์ œ์ผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ €๋Š” master 1๋Œ€์™€ worker 3๋Œ€๋ฅผ ์‚ฌ์šฉํ•˜์˜€์ง€๋งŒ,ย 
ย  ย ์ž์› ๋ถ€์กฑ ์‹œ tajo-site.xml์˜ worker ์…‹ํŒ… ์ค‘ cpu-core๋ฅผ 2๋กœ ๋Š˜๋ฆฌ๋Š” ๋ฐฉ๋ฒ•์„ ์‚ฌ์šฉํ•ด๋„ ์ข‹์„ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. !ย 

4. ๋งˆ์ง€๋ง‰์œผ๋กœ ๋งค๋ฒˆ ์„ค์น˜ํ•  ๋•Œ๋งˆ๋‹ค.. ๋А๋ผ๋Š” ๊ฒƒ์ด์ง€๋งŒ master์˜ tajo-env.sh์™€ tajo-site.xml์€ ์œ„์˜ ์กฐ์–ธ๋Œ€๋กœ master์— ์žˆ๋Š” ๊ฒƒ์„ ์›Œ์ปค๋กœ ๋ณต์‚ฌํ•ด์„œ ์‚ฌ์šฉํ•˜์‹œ๋Š” ๊ฒƒ์„ ๊ฐ•์ถ”ํ•ฉ๋‹ˆ๋‹ค!ย 
ย  ย  ๊ฐ ์„œ๋ฒ„์—์„œ ์ผ์ผ์ด ์ž‘์„ฑ์‹œ ํƒ€์ดํ•‘ ์‹ค์ˆ˜ ํ•  ์—ฌ์ง€๊ฐ€ ์ƒ๋‹นํžˆ ํฝ๋‹ˆ๋‹ค!ย 

์„ค์น˜์— ์–ด๋ ค์›€์ด ์žˆ์—ˆ๋Š”๋ฐ ๊ฐ„๋‹จํ•œ ์‹ค์ˆ˜ ์˜€์ง€๋งŒ, ๊ทธ๋ž˜๋„ ์œ„์˜ ์กฐ์–ธ๋“ค ๋ณด๋ฉด์„œ ํ•˜๋‚˜์”ฉ ํ™•์ธํ•ด๋ณผ ์ˆ˜ ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค.
์ •๋ง ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค!!

2015๋…„ 8์›” 26์ผ ์ˆ˜์š”์ผ ์˜คํ›„ 6์‹œ 8๋ถ„ 25์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
์•ˆ๋…•ํ•˜์„ธ์š”

Jinho Kim

unread,
Aug 29, 2015, 7:36:16โ€ฏAM8/29/15
to Apache Tajo ํ•œ๊ตญ ์‚ฌ์šฉ์ž ๊ทธ๋ฃน
์•ˆ๋…•ํ•˜์„ธ์š”.

๋ฌผ๋ฆฌ์  core ๊ฐ€ 1๊ฐœ๊ฐ€ ์•„๋‹ˆ๋ผ๋ฉด ๋”ฐ๋กœ ์„ค์ •ํ•˜์ง€ ์•Š๋Š”๊ฒŒ ์ข‹์Šต๋‹ˆ๋‹ค. ์•„๋ž˜ ๋ฌธ์„œ๋ฅผ ๋ณด์‹œ๋ฉด tajo-env.sh ์— ํž™๋ฉ”๋ชจ๋ฆฌ์™€ ๋ฆฌ์†Œ์Šค ์„ค์ •์ด ์žˆ์œผ๋‹ˆย 
์ฐธ๊ณ ํ•ด์„œ ์‹œ์Šคํ…œ ์ž์›์„ ํ™œ์šฉํ•˜์‹œ๋ฉด ๋น ๋ฅด๊ฒŒ ์“ฐ์‹ค์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

2015๋…„ 8์›” 29์ผ ํ† ์š”์ผ ์˜คํ›„ 2์‹œ 32๋ถ„ 31์ดˆ UTC+9, Mellisa ๋‹˜์˜ ๋ง:
Reply all
Reply to author
Forward
0 new messages