tsql 실행관련

203 views
Skip to first unread message

Seonho Lee

unread,
Oct 16, 2015, 12:36:47 AM10/16/15
to Apache Tajo 한국 사용자 그룹
안녕하세요
이번에 tajo설치해서 써보려고하는데요
설치하고 tsql을 실행시키려는데 다음과 같은 에러를 내고 실행이 안되네요
그리고 웹페이지 접속도 안되요

버전은 0.11.0입니다.

Exception in thread "main" org.apache.tajo.exception.TajoRuntimeException: org.apache.tajo.client.v2.exception.ClientConnectionException: Connection refused: namenode/(ip):26002
        at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:137)
        at org.apache.tajo.client.SessionConnection.<init>(SessionConnection.java:110)
        at org.apache.tajo.client.TajoClientImpl.<init>(TajoClientImpl.java:65)
        at org.apache.tajo.cli.tsql.TajoCli.<init>(TajoCli.java:248)
        at org.apache.tajo.cli.tsql.TajoCli.main(TajoCli.java:703)
Caused by: org.apache.tajo.client.v2.exception.ClientConnectionException: Connection refused: namenode/(ip):26002
        ... 5 more
Caused by: io.netty.channel.ConnectTimeoutException: Connection refused: namenode/(ip):26002
        at org.apache.tajo.rpc.NettyClientBase.makeConnectException(NettyClientBase.java:203)
        at org.apache.tajo.rpc.NettyClientBase.doReconnect(NettyClientBase.java:252)
        at org.apache.tajo.rpc.NettyClientBase.connect(NettyClientBase.java:221)
        at org.apache.tajo.rpc.RpcClientManager.newBlockingClient(RpcClientManager.java:164)
        at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:134)
        ... 4 more



Jihoon Son

unread,
Oct 16, 2015, 12:44:56 AM10/16/15
to Seonho Lee, Apache Tajo 한국 사용자 그룹
안녕하세요

Tsql이 HDFS namenode에 접속하지 못하는 것 같습니다.
혹시 hdfs가 잘 실행되고 있는지 확인 부탁드립니다.

손지훈 드림
2015년 10월 16일 (금) 오후 1:36, Seonho Lee <holy...@gmail.com>님이 작성:

Jihoon Son

unread,
Oct 16, 2015, 1:08:09 AM10/16/15
to Seonho Lee, Apache Tajo 한국 사용자 그룹
아 tsql이 아니라 tajo master가 접속 하고 있지 못한 상황입니다.

손지훈 드림
2015년 10월 16일 (금) 오후 1:44, Jihoon Son <ghoo...@gmail.com>님이 작성:

Seonho Lee

unread,
Oct 16, 2015, 1:54:16 AM10/16/15
to Apache Tajo 한국 사용자 그룹, holy...@gmail.com
hdfs는 잘실행되고있습니다. 

tajo master가 접속하려면 어떻게 해야할까요?

그리고 이것저것 보다가 tajo-hadoop-master-namenode.out이 아래와 같이 나왔는데 관련이 있을까요?
org.apache.tajo.exception.TajoInternalError: internal error: null
  at org.apache.tajo.catalog.CatalogServer.serviceInit(CatalogServer.java:141)
  at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
  at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
  at org.apache.tajo.master.TajoMaster.serviceInit(TajoMaster.java:213)
  at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
  at org.apache.tajo.master.TajoMaster.main(TajoMaster.java:603)

Hyunsik Choi

unread,
Oct 16, 2015, 2:40:18 AM10/16/15
to Seonho Lee, Apache Tajo 한국 사용자 그룹
안녕하세요?

네 가장 위에러는 HDFS 접속이 불가능한 경우가 맞아보입니다.

마지막 주신 오류는 또 다른 오류로 보입니다. 카탈로그 서버 시동 중에 발생한 오류 같은데요. 혹시 카탈로그 관련해서
설정하신게 있으신지요? 저 로그 위에 다른 오류는 없으신가요?

최현식 드림

Jinho Kim

unread,
Oct 16, 2015, 4:15:39 AM10/16/15
to Seonho Lee, tajo-u...@googlegroups.com
namenode 라는 이름이 host 명인가요? 
worker 는 nemenode, secondary,datanode1 이라고 하셨는데 workers 파일에 넣은것도 host 명이 맞는지요?
마지막으로 설정은 모든 tajo worker 에 배포하셨나요?

2015. 10. 16., 16:54, Seonho Lee <holy...@gmail.com> 작성:

안녕하세요 

저 문서대로 
tajo-site.xml은

<configuration>
        <property>
                <name>tajo.rootdir</name>
                <value>hdfs://namenode:9000/tajo</value>
        </property>
        <property>
                <name>tajo.master.umbilical-rpc.address</name>
                <value>namenode:26001</value>
        </property>

        <property>
                <name>tajo.master.client-rpc.address</name>
                <value>namenode:26002</value>
        </property>

        <property>
                <name>tajo.resource-tracker.rpc.address</name>
                <value>namenode:26003</value>
        </property>

        <property>
                <name>tajo.catalog.client-rpc.address</name>
                <value>namenode:26005</value>
        </property>
</configuration>

worker는
namenode
secondary
datanode1

로 설정했는데 혹시 잘못된게 있나요?


2015년 10월 16일 오후 3:26, Jinho Kim <jin...@gmail.com>님이 작성:
안녕하세요.

제가 보기엔 cluster 설정이 잘 안된것 같습니다. namenode 라는게 정상적인 hostname 인지 확인해보시고
아래 문서에 따라서 설정후 실행해보시기 바랍니다.
http://tajo.apache.org/docs/current/configuration/cluster_setup.html

--Jinho
Best regards

2015년 10월 16일 오후 2:54, Seonho Lee <holy...@gmail.com>님이 작성:

Seonho Lee

unread,
Oct 16, 2015, 5:42:01 AM10/16/15
to Apache Tajo 한국 사용자 그룹, holy...@gmail.com
jinho kim님이 말하신거는 다되있습니다.

그리고 카탈로그 설정은 하나도 안했습니다.

master log를 보니까 저런 오류가 있어서 링크따라가서 봤는데
전에 설치했던거 전부를 지워버려서 저방법으로는 못할거같아요
혹시 다른 방법이 있을까요?

455 2015-10-16 17:29:37,581 INFO org.apache.tajo.master.TajoMaster: registered UNIX signal handlers for [TERM, HUP, INT]
456 2015-10-16 17:29:38,786 INFO org.apache.tajo.master.TajoMaster: Tajo Root Directory: hdfs://namenode:9000/tajo
457 2015-10-16 17:29:44,194 INFO org.apache.tajo.master.TajoMaster: FileSystem (hdfs://namenode:9000) is initialized.
458 2015-10-16 17:29:44,509 INFO org.apache.tajo.master.TajoMaster: Tajo Warehouse dir: hdfs://namenode:9000/tajo/warehouse
459 2015-10-16 17:29:44,512 INFO org.apache.tajo.master.TajoMaster: Staging dir: hdfs://namenode:9000/tajo/warehouse
460 2015-10-16 17:29:48,962 WARN org.apache.tajo.storage.FileTablespace: does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled')
461 2015-10-16 17:29:48,965 WARN org.apache.tajo.storage.FileTablespace: does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled')
462 2015-10-16 17:29:48,986 WARN org.apache.tajo.storage.FileTablespace: does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled')
463 2015-10-16 17:29:50,764 INFO org.apache.tajo.plan.function.python.PythonScriptEngine: Python base dir is /tmp/tajo-hadoop/python
464 2015-10-16 17:29:50,790 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.rm.NodeEventType for class org.apache.tajo.master.rm.TajoResourceManager$WorkerEventDispatcher
465 2015-10-16 17:29:50,993 INFO org.apache.tajo.master.rm.TajoResourceManager: Loaded resource scheduler : class org.apache.tajo.master.scheduler.SimpleScheduler
466 2015-10-16 17:29:50,994 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.tajo.master.scheduler.event.SchedulerEventType for class org.apache.tajo.master.scheduler.SimpleScheduler
467 2015-10-16 17:29:51,333 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (TajoResourceTrackerProtocol) listens on /147.46.174.147:26003
468 2015-10-16 17:29:51,333 INFO org.apache.tajo.master.rm.TajoResourceTracker: TajoResourceTracker starts up (namenode/147.46.174.147:26003)
469 2015-10-16 17:29:51,345 INFO org.apache.tajo.catalog.CatalogServer: Catalog Store Class: org.apache.tajo.catalog.store.DerbyStore
470 2015-10-16 17:29:51,676 INFO org.apache.tajo.catalog.store.DerbyStore: Loaded the Catalog driver (org.apache.derby.jdbc.EmbeddedDriver)
471 2015-10-16 17:29:51,676 INFO org.apache.tajo.catalog.store.DerbyStore: Trying to connect database (jdbc:derby:/tmp/tajo-catalog-hadoop/db;create=true)
472 2015-10-16 17:29:54,025 INFO org.apache.tajo.catalog.store.DerbyStore: Connected to database (jdbc:derby:/tmp/tajo-catalog-hadoop/db;create=true)
473 2015-10-16 17:29:55,068 INFO org.apache.tajo.catalog.store.DerbyStore: The meta table of CatalogServer already is created.
474 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: Catalog version (2) and current driver version (12) are mismatch to each other
475 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: =========================================================================
476 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: | Catalog Store Migration Is Needed |
477 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: =========================================================================
478 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: | You might downgrade or upgrade Apache Tajo. Downgrading or upgrading |
479 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: | Tajo without migration process is only available in some versions. |
480 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: | In order to learn how to migration Apache Tajo instance, |
481 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: | please refer http://tajo.apache.org/docs/current/backup_and_restore/catalog.html |
482 2015-10-16 17:29:55,076 ERROR org.apache.tajo.catalog.store.DerbyStore: =========================================================================
483 2015-10-16 17:29:55,084 ERROR org.apache.tajo.catalog.CatalogServer: CatalogServer initialization failed
484 java.lang.reflect.InvocationTargetException
485   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
486   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
487   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
488   at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
489   at org.apache.tajo.catalog.CatalogServer.serviceInit(CatalogServer.java:136)
490   at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
491   at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
492   at org.apache.tajo.master.TajoMaster.serviceInit(TajoMaster.java:213)
493   at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
494   at org.apache.tajo.master.TajoMaster.main(TajoMaster.java:603)
495 Caused by: org.apache.tajo.exception.TajoInternalError: internal error: catalog must be upgraded
496   at org.apache.tajo.catalog.store.AbstractDBStore.<init>(AbstractDBStore.java:193)
497   at org.apache.tajo.catalog.store.DerbyStore.<init>(DerbyStore.java:46)
498   ... 10 more

Jinho Kim

unread,
Oct 16, 2015, 9:02:36 AM10/16/15
to Apache Tajo 한국 사용자 그룹, holy...@gmail.com
안녕하세요.

스키마 버전이 2로 나오는걸보면 0.10.0 또는 0.10.1 로 보입니다.
"Catalog version (2) and current driver version (12) .."

아래 링크에서 다운받으신후 backup 하시면 될듯보이네요
http://mirror.apache-kr.org/tajo/tajo-0.10.1/tajo-0.10.1.tar.gz

아래 링크는 사용하시는 derby 의 catalog history 이니 참고하세요.
https://github.com/apache/tajo/blob/master/tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
    Catalog base version history
      * 12 - 2015-09-28: Change the variable name storeType to dataFormat (TAJO-1663)
      * 11 - 2015-09-23: Add contents length and file count for partition directory (TAJO-1493)
      * 10 - 2015-09-22: Well support for self-describing data formats (TAJO-1832)
      * 9 - 2015-09-12: Allow external catalog store for unit testing (TAJO-1813)
      * 8 - 2015-09-02: Wrong table type problem in catalog (TAJO-1808)
      * 7 - 2015-07-30: Add a column and index for partition keys (TAJO-1346)
      * 6 - 2015-07-24: Merge the index branch into the master branch (TAJO-1300)
      * 5 - 2015-06-15: Implement TablespaceManager to load Tablespaces (TAJO-1616)
      * 4 - 2015-03-27: Partition Schema (TAJO-1284)
      * 3 - 2015-03-12: Nested Schema (TAJO-1329)
      * 2 - 2014-06-09: First versioning
      * 1-  Before 2013-03-20


2015년 10월 16일 금요일 오후 6시 42분 1초 UTC+9, Seonho Lee 님의 말:

Seonho Lee

unread,
Oct 17, 2015, 7:17:01 AM10/17/15
to Apache Tajo 한국 사용자 그룹, holy...@gmail.com
알려주신대로 해서 해결했습니다!

감사합니다. 

김 진호

unread,
Oct 17, 2015, 12:01:31 PM10/17/15
to Seonho Lee, Apache Tajo 한국 사용자 그룹
네 쉽게 해결되셔서 좋네요 나중에 후기 부탁드립니다

나의 iPhone에서 보냄

2015. 10. 17. 20:17 Seonho Lee <holy...@gmail.com> 작성:

xiaohui Huang

unread,
May 3, 2017, 1:25:06 AM5/3/17
to Apache Tajo 한국 사용자 그룹
Hello, I met the same problem. When I typed start-tajo.sh, the system is ok. BUT when I executed tsql, the system is failed.

I sincerely want to know the reason that caused this problem, and how to solve this problem.

Thank you.

More details about my problems can be found in https://groups.google.com/forum/#!topic/tajo-user-kr/h6wkhpswcJI

Here are my terminal records.

[hadoop@master ~]$ start-tajo.sh 
Starting single TajoMaster
starting master, logging to /home/hadoop/tajo-0.11.3/logs/tajo-hadoop-master-master.out
slave1: starting worker, logging to /home/hadoop/tajo-0.11.3/logs/tajo-hadoop-worker-slave1.out
slave2: starting worker, logging to /home/hadoop/tajo-0.11.3/logs/tajo-hadoop-worker-slave2.out
Tajo master web UI: http://master:26080
Tajo Client Service: master:26002
[hadoop@master ~]$ tsql
Exception in thread "main" org.apache.tajo.exception.TajoRuntimeException: org.apache.tajo.client.v2.exception.ClientConnectionException: Connection refused: master/59.71.229.189:26002 (Notes: the IP address of the master is 59.71.229.189)

    at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:137)
    at org.apache.tajo.client.SessionConnection.<init>(SessionConnection.java:110)
    at org.apache.tajo.client.TajoClientImpl.<init>(TajoClientImpl.java:64)
    at org.apache.tajo.cli.tsql.TajoCli.<init>(TajoCli.java:255)
    at org.apache.tajo.cli.tsql.TajoCli.main(TajoCli.java:719)
Caused by: org.apache.tajo.client.v2.exception.ClientConnectionException: Connection refused: master/59.71.229.189:26002
    ... 5 more
Caused by: io.netty.channel.ConnectTimeoutException: Connection refused: master/59.71.229.189:26002
    at org.apache.tajo.rpc.NettyClientBase.makeConnectException(NettyClientBase.java:202)
    at org.apache.tajo.rpc.NettyClientBase.doReconnect(NettyClientBase.java:251)
    at org.apache.tajo.rpc.NettyClientBase.connect(NettyClientBase.java:220)

    at org.apache.tajo.rpc.RpcClientManager.newBlockingClient(RpcClientManager.java:164)
    at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:134)
    ... 4 more
[hadoop@master ~]$ 
Reply all
Reply to author
Forward
0 new messages