multi machine implementation

[Binish Xavier]

How to implement multi machine in hyper table.
if its possible?.
Where i get support for multi machine implementation

[Christoph Rupp]

Hi Binish,

what is "multi machine"? Do you want to know how to set up a cluster with several hypertable machines?

Bye
Christoph

2012/5/5 Binish Xavier <binish...@nesote.com>

How to implement multi machine in hyper table.
if its possible?.
Where i get support for multi machine implementation

--
You received this message because you are subscribed to the Google Groups "Hypertable User" group.
To post to this group, send email to hyperta...@googlegroups.com.
To unsubscribe from this group, send email to hypertable-us...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/hypertable-user?hl=en.

[Binish Xavier]

1)       i have installed hadoop, hypertable and Capistrano in my local machine.   can you help me to configure  Capistrano with the hddop. 

          And also i came across the variable "source_machine" in conf/Capfile.cluster, it was set to value "admin", could you tell me whats the actual meaning of this variable. Is this system name or does it stand for any other purpose.

2) Do you want to know how to set up a cluster with several hypertable machines?

Thanks and Regards,
Binish Xavier

[Christoph Rupp]

Hi Binish,

1) you do not need to use capistrano if you use hypertable on a single machine. You can run "start-all-servers.sh hadoop" instead.

2) we already know :)
http://hypertable.com/documentation/installation/quick_start_cluster_installation/

bye
Christoph

2012/10/26 Binish Xavier <binish...@nesote.com>

--

[Binish Xavier]

not in single machine.hypertable installation in multimachine

--

[Christoph Rupp]

Hi Binish,

for a single-machine hadoop "pseudo"-cluster use "start-all-servers.sh hadoop"
for a multi-machine hadoop-cluster please see the link that i sent. It describes how to configure hdfs, hypertable and capistrano.

[Rojin Mani]

Hello HyperTable-Expert,

I am writing this mail to bring an expert attention over my hypertable multi machine implementation. I would be much grateful to you people if you could provide proper guidance.

Current situation is that, I could install every modules successfully master and slave area.

Master run successfully. No Error logs in log folder located in opt/hypertable/version/log/ But the slaves not started. Waiting for DFS Broker (hadoop) (localhost:38030) to come up...

Here is the DfsBroker.hadoop.log  in slave machine

Num CPUs=8

HdfsBroker.Port=38030

HdfsBroker.Reactors=8

HdfsBroker.Workers=20

HdfsBroker.Hadoop.ConfDir=/etc/hadoop-0.20/conf

Adding hadoop configuration file /etc/hadoop-0.20/conf/hdfs-site.xml

Adding hadoop configuration file /etc/hadoop-0.20/conf/core-site.xml

HdfsBroker.dfs.client.read.shortcircuit=false

HdfsBroker.dfs.replication=3

HdfsBroker.Server.fs.default.name=hdfs://localhost:10818

13/09/30 12:31:14 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 0 time(s).

13/09/30 12:31:15 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 1 time(s).

13/09/30 12:31:16 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 2 time(s).

13/09/30 12:31:17 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 3 time(s).

13/09/30 12:31:18 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 4 time(s).

13/09/30 12:31:19 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 5 time(s).

13/09/30 12:31:20 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 6 time(s).

13/09/30 12:31:21 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 7 time(s).

13/09/30 12:31:22 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 8 time(s).

13/09/30 12:31:23 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:10818. Already tried 9 time(s).

Sep 30, 2013 12:31:23 PM org.hypertable.DfsBroker.hadoop.HadoopBroker <init>

SEVERE: ERROR: Unable to establish connection to HDFS.

ShutdownHook called

Exception in thread "Thread-1" java.lang.NullPointerException

        at org.hypertable.DfsBroker.hadoop.main$ShutdownHook.run(main.java:73)

Here is the configuration in core-site.xml

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>

     <name>fs.default.name</name>

     <value>hdfs://localhost:10818</value>

   </property>

</configuration>

Here is the configuration in Mapred-site.xml

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>

        <name>mapred.job.tracker</name>

        <value>localhost:10814</value>

    </property>

    <property>

        <name>mapred.local.dir</name>

        <value>/var/opt/cdh3/cluster/mapred/local</value>

    </property>

 <property>

       <name>mapred.system.dir</name>

       <value>/tmp/mapred/system</value>

    </property>

</configuration>

Here is the configuration in Hdfs-site.xml

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>

 <property>

        <name>dfs.replication</name>

        <value>3</value>

    </property>

    <property>

        <name>dfs.name.dir</name>

        <value>/var/opt/cdh3/cluster/dfs/nn</value>

    </property>

    <property>

        <name>dfs.data.dir</name>

        <value>/var/opt/cdh3/cluster/dfs/dn</value>

    </property>

</configuration>

Here is the configuration in Capfile 

set :source_machine, "master public ip"

set :install_dir,  "/opt/hypertable"

set :hypertable_version, "0.9.7.10"

set :default_pkg, "/hypertable-0.9.7.10-linux-x86_64.rpm"

set :default_dfs, "hadoop"

set :default_distro, "cdh3"

set :default_config, "/opt/hypertable/0.9.7.10/conf/hypertable.cfg"

role :source, "master public ip"

role :master, "master public ip"

role :hyperspace, "master public ip"

role :slave,  "Slave public ip", "slave public ip"

role :localhost, "master public ip"

role :thriftbroker_additional, ""

role :spare, ""

Here is the configuration in hypertable.cfg

HdfsBroker.fs.default.name=hdfs://localhost:10818

# hypertable.cfg

#

Hypertable.Network.Interface=eth0

# HDFS Broker

HdfsBroker.Hadoop.ConfDir=/etc/hadoop-0.20/conf

# Ceph Broker

CephBroker.MonAddr=10.0.1.245:6789

# Local Broker

DfsBroker.Local.Root=fs/localhost

# Qfs Broker

Qfs.MetaServer.Name=master public ip

Qfs.MetaServer.Port=20000

# DFS Broker - for clients

DfsBroker.Port=38030

# Hyperspace

#Hyperspace.Master.Host=master public ip

Hyperspace.Replica.Host=master public ip

Hyperspace.Replica.Port=38040

Hyperspace.Replica.Dir=hyperspace

# Hypertable.Master

Hypertable.Master.Host=master public ip

Hypertable.Master.Port=38050

Hypertable.Master.Workers=20

# Hypertable.RangeServer

Hypertable.RangeServer.Port=38060

Hyperspace.KeepAlive.Interval=30000

Hyperspace.Lease.Interval=1000000

Hyperspace.GracePeriod=200000

"hypertable.cfg" 40L, 859C

..................................................................................................................

Could you please make a glance over my system and guide me where i make a mistake. Please help me.

With Thanks And Regards

Roji