hypertable can't run with hadoop file system

[gmz]

Hi,

i want to run hypertable with hadoop file system. but i have errors. Please help me :((

cap> /opt/hypertable/current/bin/start-all-servers.sh hadoop

 ** [out :: slave1] DFS broker: available file descriptors: 1024

*** [err :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 147: /opt/hypertable/current/run/Hyperspace.pid: No such file or directory

*** [err :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 147: /opt/hypertable/current/run/DfsBroker.hadoop.pid: Permission denied

 ** [out :: slave1] DfsBroker.hadoop appears to be running (1380):

 ** [out :: slave1] root 1380 1373 0 Jun29 ? 00:03:12 /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -classpath /opt/hypertable/current:/opt/hypertable/current/lib/java/libthrift-0.8.0.jar:/opt/hypertable/current/lib/java/commons-cli-1.2.jar:/opt/hypertable/current/lib/java/commons-httpclient-3.1.jar:/opt/hypertable/current/lib/java/commons-logging-1.0.4.jar:/opt/hypertable/current/lib/java/guava-r09-jarjar.jar:/opt/hypertable/current/lib/java/hadoop-0.20.0-core.jar:/opt/hypertable/current/lib/java/hadoop-0.20.2-cdh3u3-core.jar:/opt/hypertable/current/lib/java/hbase-0.90.4-cdh3u2.jar:/opt/hypertable/current/lib/java/hive-exec-0.7.0-cdh3u0.jar:/opt/hypertable/current/lib/java/hive-metastore-0.7.0-cdh3u0.jar:/opt/hypertable/current/lib/java/hive-serde-0.7.0-cdh3u0.jar:/opt/hypertable/current/lib/java/hypertable-0.9.5.6-examples.jar:/opt/hypertable/current/lib/java/hypertable-0.9.5.6.jar:/opt/hypertable/current/lib/java/junit-4.3.1.jar:/opt/hypertable/current/lib/java/libthrift-0.8.0.jar:/opt/hypertable/current/lib/java/log4j-1.2.13.jar:/opt/hypertable/current/lib/java/slf4j-api-1.5.8.jar:/opt/hypertable/current/lib/java/slf4j-log4j12-1.5.8.jar:/opt/hypertable/current/lib/java/zookeeper-3.3.3-cdh3u2.jar:/opt/hypertable/current/lib/jetty-ext/*.jar org.hypertable.DfsBroker.hadoop.main --verbose

 ** [out :: dfs1] DFS broker: available file descriptors: 1024

 ** [out :: dfs1] Hyperspace appears to be running (22030):

 ** [out :: dfs1] root 22030 22028 0 09:59 pts/2 00:00:00 /opt/hypertable/current/bin/Hyperspace.Master --pidfile /opt/hypertable/current/run/Hyperspace.pid --verbose

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: slave1] Waiting for Hyperspace to come up...

 ** [out :: slave1] ERROR: Hyperspace did not come up

 ** [out :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 165: /opt/hypertable/current/bin/cronolog: cannot execute binary file

*** [err :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 147: /opt/hypertable/current/run/Hypertable.Master.pid: No such file or directory

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: slave1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] ERROR: Hypertable.Master did not come up

 ** [out :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 165: /opt/hypertable/current/bin/cronolog: cannot execute binary file

*** [err :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 147: /opt/hypertable/current/run/Hypertable.RangeServer.pid: No such file or directory

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for DFS Broker (hadoop) to come up...

 ** [out :: dfs1] ERROR: DFS Broker (hadoop) did not come up

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for Hypertable.Master to come up...

 ** [out :: dfs1] ERROR: Hypertable.Master did not come up

 ** [out :: dfs1] Hypertable.RangeServer appears to be running (28222):

 ** [out :: dfs1] root 28222 28221 1 10:34 ? 00:00:02 /opt/hypertable/current/bin/Hypertable.RangeServer --pidfile /opt/hypertable/current/run/Hypertable.RangeServer.pid --verbose

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for Hypertable.RangeServer to come up...

 ** [out :: slave1] ERROR: Hypertable.RangeServer did not come up

 ** [out :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 165: /opt/hypertable/current/bin/cronolog: cannot execute binary file

*** [err :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 147: /opt/hypertable/current/run/ThriftBroker.pid: No such file or directory

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: dfs1] Waiting for ThriftBroker to come up...

 ** [out :: dfs1] ERROR: ThriftBroker did not come up

 ** [out :: dfs1] ThriftBroker appears to be running (29295):

 ** [out :: dfs1] root 29295 29294 0 10:36 ? 00:00:00 /opt/hypertable/current/bin/ThriftBroker --pidfile /opt/hypertable/current/run/ThriftBroker.pid --verbose

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] Waiting for ThriftBroker to come up...

 ** [out :: slave1] ERROR: ThriftBroker did not come up

 ** [out :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 165: /opt/hypertable/current/bin/cronolog: cannot execute binary file

error: failed: "sh -c '/opt/hypertable/current/bin/start-all-servers.sh hadoop'" on dfs1,slave1

******************my config file:************

****hypertable.cfg****

# hypertable.cfg

# Global properties

Hypertable.Request.Timeout=180000

# HDFS Broker

HdfsBroker.fs.default.name=hdfs://dfs1:9000

# Ceph Broker

CephBroker.MonAddr=10.0.1.245:6789

# Local Broker

DfsBroker.Local.Root=fs/local

# DFS Broker - for clients

DfsBroker.Port=38030

# Hyperspace

Hyperspace.Replica.Host=dfs1

Hyperspace.Replica.Port=38040

Hyperspace.Replica.Dir=hyperspace

Hyperspace.Replica.Workers=20

Hyperspace.Client.Datagram.SendPort=38049

# Hypertable.Master

Hypertable.Master.Port=38050

# Hypertable.RangeServer

Hypertable.RangeServer.Port=38060

Hyperspace.KeepAlive.Interval=30000

Hyperspace.Lease.Interval=1000000

Hyperspace.GracePeriod=200000

# ThriftBroker

ThriftBroker.Port=38080

Hypertable.RangeServer.Monitoring.DataDirectories="/data/1,/data/2,/data/3,/da$

# Test specific

Hypertable.RangeServer.Range.SplitSize=1GB

Hypertable.RangeServer.QueryCache.MaxMemory=2GB

****config/deploy.rb**** 

set :source_machine, "dfs1"

set :install_dir,  "/opt/hypertable"

set :hypertable_version, "0.9.5.6"

set :default_pkg, "/opt/hypertable-0.9.5.6-linux-x86_64.deb"

set :default_dfs, "hadoop"

set :default_config, "/opt/hypertable/current/conf/hypertable.cfg"

role :source, "dfs1"

role :master, "dfs1"

role :hyperspace, "dfs1"

role :slave,  "dfs1", "slave1"

role :localhost, "dfs1"

role :thriftbroker,"dfs1","slave1"

role :spare

[Christoph Rupp]

Hi,

can you check out this one:

 ** [out :: slave1] DFS broker: available file descriptors: 1024

*** [err :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 147: /opt/hypertable/current/run/Hyperspace.pid: No such file or directory

*** [err :: slave1] /opt/hypertable/current/bin/ht-env.sh: line 147: /opt/hypertable/current/run/DfsBroker.hadoop.pid: Permission denied

the hypertable processes do not have write permission in /opt/hypertable/current/run.

As a consequence, the Dfs broker shuts down and all other processes fail to start.

Bye
Christoph

2012/7/2 gmz <hattice....@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Hypertable Development" group.
To view this discussion on the web visit https://groups.google.com/d/msg/hypertable-dev/-/2SFmTIOgvY0J.
To post to this group, send email to hyperta...@googlegroups.com.
To unsubscribe from this group, send email to hypertable-de...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.

[gmz]

Hi,

Yes, i checked it and corrected it but the same problem is still in logs. 

2 Temmuz 2012 Pazartesi 15:29:59 UTC+3 tarihinde Christoph Rupp yazdı:

2012/7/2 gmz <hattice....@gmail.com>

To post to this group, send email to hypertable-dev@googlegroups.com.
To unsubscribe from this group, send email to hypertable-dev+unsubscribe@googlegroups.com.