Waiting for Master to come up ( connect error )

104 views
Skip to first unread message

Felix

unread,
Jun 29, 2016, 12:52:07 AM6/29/16
to Hypertable User

I just installed Hypertable with Hadoop, but when I executed


ht cluster start


FSBroker (hadoop) starts correct, but the Master fails to start.


Waiting for Master to come up ( connect error ) …
Master CRITICAL – connect error
Command failed on hosts: ubuntu


See the Master.log below


1467137540 INFO htMaster : (/root/src/hypertable/src/cc/Common/Config.cc:649) Initializing htMaster (Hypertable 0.9.8.11 (v0.9.8.11-0-g86e850f))...

CPU cores count=1

CephBroker.MonAddr=10.0.1.245:6789

FsBroker.Local.Root=fs/local

HdfsBroker.Hadoop.ConfDir=/usr/local/hadoop/etc/hadoop

Hyperspace.Replica.Host=[localhost]

Hypertable.Cluster.Name="Primary Server"

Hypertable.Master.Reactors=1

Hypertable.RangeServer.LowActivityPeriod=["* 2-4 * * *"]

Hypertable.RangeServer.Monitoring.DataDirectories="/"

Hypertable.Verbose=true

Qfs.MetaServer.Name=localhost

Qfs.MetaServer.Port=20000

hs-host=[localhost]

pidfile=/opt/hypertable/current/run/Master.pid

reactors=1

verbose=true

1467137540 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/Monitoring.cc:76) rangeservers monitoring stats dir /opt/hypertable/0.9.8.11/run/monitoring exists

1467137540 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/Monitoring.cc:76) rangeservers monitoring stats dir /opt/hypertable/0.9.8.11/run/monitoring/tables exists

1467137540 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/Monitoring.cc:76) rangeservers monitoring stats dir /opt/hypertable/0.9.8.11/run/monitoring/rangeservers exists

1467137541 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/HyperspaceMasterFile.h:136) Obtained lock on '/hypertable/master'

1467137541 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/main.cc:155) Cluster id is 15277701053254300597

1467137542 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:57830

1467137542 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-1

1467137542 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:57830

1467137543 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:60147

1467137543 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-2

1467137543 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:60147

1467137545 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:49278

1467137545 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-3

1467137545 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:49278

1467137546 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:55433

1467137547 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-15

1467137547 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:55433

1467137548 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:50247

1467137548 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-16

1467137548 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:50247

1467137550 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:55991

1467137550 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-17

1467137550 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:55991

1467137553 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:60403

1467137553 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-18

1467137553 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:60403

1467137555 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:58801

1467137555 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-19

1467137555 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:58801

1467137556 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationSystemUpgrade.cc:81) Entering SystemUpgrade-21

1467137556 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/IOHandler.h:175) Event: type=CONNECTION_ESTABLISHED from=192.168.0.30:64395

1467137556 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationStatus.cc:38) Status-22

1467137556 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/ConnectionHandler.cc:281) Event: type=DISCONNECT from=192.168.0.30:64395

1467137557 INFO htMaster : (/root/src/hypertable/src/cc/Hypertable/Master/OperationSystemUpgrade.cc:99) Leaving SystemUpgrade-21

1467137557 ERROR htMaster : main (/root/src/hypertable/src/cc/Hypertable/Master/main.cc:314): Hypertable::Exception: sys/RS_METRICS - HYPERTABLE table does not exist

                at void Hypertable::Table::initialize() (/root/src/hypertable/src/cc/Hypertable/Lib/Table.cc:96)

1467137557 WARN htMaster : (/root/src/hypertable/src/cc/Hyperspace/ClientConnectionHandler.cc:72) Event: type=DISCONNECT from=127.0.0.1:15861

1467137557 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/ConnectionManager.cc:378) Received event Event: type=DISCONNECT from=127.0.0.1:15863

1467137557 INFO htMaster : (/root/src/hypertable/src/cc/AsyncComm/ConnectionManager.cc:442) Disconnected: Problem connecting to FS Broker, will retry in 600000 milliseconds..


Does anyone know a solution for this? 


The operating system is Ubuntu 14.04 running under Virtualbox 5.0.22.


Thanks in advance for your help!


Felix

Doug Judd

unread,
Jul 1, 2016, 10:45:21 AM7/1/16
to hypertable-user
Are you sure the FsBroker came up?  Check the FsBroker.hadoop.log file.  If you post your hypertable.cfg file and cluster.def file, that would help me diagnose the problem.

- Doug

--
You received this message because you are subscribed to the Google Groups "Hypertable User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypertable-us...@googlegroups.com.
To post to this group, send email to hyperta...@googlegroups.com.
Visit this group at https://groups.google.com/group/hypertable-user.
For more options, visit https://groups.google.com/d/optout.

Felix

unread,
Jul 1, 2016, 2:25:51 PM7/1/16
to Hypertable User

There is an error in FsBroker.hadoop.log


Jul 01, 2016 8:16:11 PM org.hypertable.FsBroker.Lib.MetricsHandler handle
INFO: Problem publishing Ganglia metrics - ICMP Port Unreachable


Cluster.def


INSTALL_PREFIX=/opt/hypertable

HYPERTABLE_VERSION=0.9.8.11

PACKAGE_FILE=/root/packages/hypertable-0.9.8.11-linux-x86_64.tar.gz

FS=hadoop

HADOOP_DISTRO=apache2

ORIGIN_CONFIG_FILE=/etc/opt/hypertable/hypertable.cfg

PROMPT_DESTROY=true

PROMPT_CLEAN=true

 

role: source ubuntu

role: master ubuntu

role: hyperspace ubuntu

role: slave ubuntu

role: thriftbroker

role: spare

 

include: "core.tasks"

 

 

Hypertable.cfg

 

#

# Hypertable Configuration

#

 

# HDFS Broker specific properties

# The following property should be set to the absolute path to the

# directory containing the Hadoop configuration.  The Hadoop broker

# uses this property to locate core-site.xml and hdfs-site.xml files

HdfsBroker.Hadoop.ConfDir=/usr/local/hadoop/etc/hadoop

 

# Ceph Broker specific properties

CephBroker.MonAddr=10.0.1.245:6789

 

# Local Broker specific properties

# The following property is used by the local broker to determine where

# to write Hypertable state.  If it is a relative path, the path is

# relative to the Hypertable installation.

FsBroker.Local.Root=fs/local

 

# Qfs Broker specific properties

Qfs.MetaServer.Name=localhost

Qfs.MetaServer.Port=20000

 

# The following property should be repeated for each Hyperspace

# replica node

Hyperspace.Replica.Host=localhost

 

# Crontab entry for when aggresive maintenance can take place in the

# RangeServer.  Consult Unix man page crontab(5) for a description of the

# format.  The specification below sets it for 2am to 5am every night.

# To disable, the word "none" can be used in place of a crontab entry.

# Hypertable.RangeServer.LowActivityPeriod=none

Hypertable.RangeServer.LowActivityPeriod="* 2-4 * * *"

 

# Change the following property to a comma-sparated list of mount points

# to the data directories to get accurate disk usage reporting in the

# monitoring UI

Hypertable.RangeServer.Monitoring.DataDirectories="/"

 

# Specifies a descriptive name for the cluster that will show up in

# the monitoring UI

Hypertable.Cluster.Name="Primary Server"

 

Kind regards,


Felix.

 


Op vrijdag 1 juli 2016 16:45:21 UTC+2 schreef Doug Judd:
Reply all
Reply to author
Forward
0 new messages