I'm having version-itis problems when trying to start Hypertable on a
Hadoop dfs:
% ~/hypertable/bin/start-master.sh hadoop
[...]
HdfsBroker.Server.fs.default.name=hdfs://motherlode001:9000
org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol
org.apache.hadoop.dfs.ClientProtocol version mismatch. (client = 14,
server = 19)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:253)
at
org.apache.hadoop.dfs.DFSClient.createNamenode(DFSClient.java:141)
at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:153)
at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSyste m.java:
66)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:
118)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:90)
at
org.hypertable.DfsBroker.hadoop.HdfsBroker.<init>(HdfsBroker.java:71)
at org.hypertable.DfsBroker.hadoop.main.main(main.java:136)
Problem statring DfsBroker (hadoop)
That I'm sure would be fixed if I could get a matched set... but not
clear how to determine
what parts of what version are what.
You're probably using the 0.9.0.4, which has an older hadoop-*core.jar
bundled. I'd suggest that you pull the latest version from the
repository, which will have a later version of hadoop-0.16.2-core.jar.
I've seen report that simply replace the hadoop*-core.jar in the
<install_dir>/java/lib would work as well.
On Apr 30, 7:20 pm, slowfood <pb...@plasm.com> wrote:
> You're probably using the 0.9.0.4, which has an older hadoop-*core.jar
> bundled. I'd suggest that you pull the latest version from the
> repository, which will have a later version of hadoop-0.16.2-core.jar.
> I've seen report that simply replace the hadoop*-core.jar in the
> <install_dir>/java/lib would work as well.
> On Apr 30, 7:20 pm, slowfood <pb...@plasm.com> wrote:
> > I'm having version-itis problems when trying to start Hypertable on a
> > Hadoop dfs:
> > % ~/hypertable/bin/start-master.sh hadoop
> > [...]
> > HdfsBroker.Server.fs.default.name=hdfs://motherlode001:9000
> > org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol
> > org.apache.hadoop.dfs.ClientProtocol version mismatch. (client = 14,
> > server = 19)
> > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:253)
> > at
> > org.apache.hadoop.dfs.DFSClient.createNamenode(DFSClient.java:141)
> > at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:153)
> > at
> > org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSyste m.java:
> > 66)
> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:159)
> > at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:
> > 118)
> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:90)
> > at
> > org.hypertable.DfsBroker.hadoop.HdfsBroker.<init>(HdfsBroker.java:71)
> > at org.hypertable.DfsBroker.hadoop.main.main(main.java:136)
> > Problem statring DfsBroker (hadoop)
> > That I'm sure would be fixed if I could get a matched set... but not
> > clear how to determine
> > what parts of what version are what.
To build I had to add:
#include <iostream>
to hypertable/src/cc/Common/Logger.h
Without it I got an error from make:
/home/admin/src/hypertable/src/cc/Common/Logger.h:31: error:
'cout' is not a member of 'std'
With this fix things seemed to build.
Still doesn't run though, and not much to go on as to why:
# bin/start-master.sh hadoop
DfsBroker (hadoop) hasn't come up yet, trying again in 5 seconds ...
Num CPUs=4
HdfsBroker.Port=38030
HdfsBroker.Reactors=4
HdfsBroker.Workers=20
HdfsBroker.Server.fs.default.name=hdfs://motherlode001:9000
Problem statring DfsBroker (hadoop)
#
(changed the name of the HdfsBroker.Server.fs.default.name from what I
really use, how to
determine what should really be there?)
;;peter
On Apr 30, 9:07 pm, slowfood <pb...@plasm.com> wrote:
> I'll give it a try (tomorrow), maybe gen up a git-enabled ebuild
> unless you already have one?
> ;;peter
> On Apr 30, 8:25 pm, Luke <vic...@gmail.com> wrote:
> > Hi Peter,
> > You're probably using the 0.9.0.4, which has an older hadoop-*core.jar
> > bundled. I'd suggest that you pull the latest version from the
> > repository, which will have a later version of hadoop-0.16.2-core.jar.
> > I've seen report that simply replace the hadoop*-core.jar in the
> > <install_dir>/java/lib would work as well.
> > On Apr 30, 7:20 pm, slowfood <pb...@plasm.com> wrote:
> > > I'm having version-itis problems when trying to start Hypertable on a
> > > Hadoop dfs:
> > > % ~/hypertable/bin/start-master.sh hadoop
> > > [...]
> > > HdfsBroker.Server.fs.default.name=hdfs://motherlode001:9000
> > > org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol
> > > org.apache.hadoop.dfs.ClientProtocol version mismatch. (client = 14,
> > > server = 19)
> > > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:253)
> > > at
> > > org.apache.hadoop.dfs.DFSClient.createNamenode(DFSClient.java:141)
> > > at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:153)
> > > at
> > > org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSyste m.java:
> > > 66)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:159)
> > > at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:
> > > 118)
> > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:90)
> > > at
> > > org.hypertable.DfsBroker.hadoop.HdfsBroker.<init>(HdfsBroker.java:71)
> > > at org.hypertable.DfsBroker.hadoop.main.main(main.java:136)
> > > Problem statring DfsBroker (hadoop)
> > > That I'm sure would be fixed if I could get a matched set... but not
> > > clear how to determine
> > > what parts of what version are what.
On May 1, 3:45 pm, slowfood <pb...@plasm.com> wrote:
> To build I had to add:
> #include <iostream>
> to hypertable/src/cc/Common/Logger.h
> Without it I got an error from make:
> /home/admin/src/hypertable/src/cc/Common/Logger.h:31: error:
> 'cout' is not a member of 'std'
Interesting, I wonder why I didn't get an error. Maybe different
compiler version. What does your gcc -v say?
> With this fix things seemed to build.
> Still doesn't run though, and not much to go on as to why:
> # bin/start-master.sh hadoop
> DfsBroker (hadoop) hasn't come up yet, trying again in 5 seconds ...
> Num CPUs=4
> HdfsBroker.Port=38030
> HdfsBroker.Reactors=4
> HdfsBroker.Workers=20
> HdfsBroker.Server.fs.default.name=hdfs://motherlode001:9000
> Problem statring DfsBroker (hadoop)
> #
> (changed the name of the HdfsBroker.Server.fs.default.name from what I
> really use, how to
> determine what should really be there?)
Does hadoop fs ls show anything? Make sure he ...default.name matches
your conf/hadoop-site.xml. Can you send me your hypertable.cfg?
> > I'll give it a try (tomorrow), maybe gen up a git-enabled ebuild
> > unless you already have one?
No, unfortunately we're not using gentoo in production here, so no
ebuild yet. We probably have rpm and deb by the time we're in beta.
But it'll be awesome to have one. Thanks.
My conf/hadoop-site.xml is "empty" (only boiler plate comments).
What parts need to match up between Hypertable and Hadoop config?
(the hadoop-default.xml may have been edited since it seems we are
on non-standard ports from what I can tell, but the site file is
"empty"...)
> On May 1, 3:45 pm, slowfood <pb...@plasm.com> wrote:
> > To build I had to add:
> > #include <iostream>
> > to hypertable/src/cc/Common/Logger.h
> > Without it I got an error from make:
> > /home/admin/src/hypertable/src/cc/Common/Logger.h:31: error:
> > 'cout' is not a member of 'std'
> Interesting, I wonder why I didn't get an error. Maybe different
> compiler version. What does your gcc -v say?
> > With this fix things seemed to build.
> > Still doesn't run though, and not much to go on as to why:
> > # bin/start-master.sh hadoop
> > DfsBroker (hadoop) hasn't come up yet, trying again in 5 seconds ...
> > Num CPUs=4
> > HdfsBroker.Port=38030
> > HdfsBroker.Reactors=4
> > HdfsBroker.Workers=20
> > HdfsBroker.Server.fs.default.name=hdfs://motherlode001:9000
> > Problem statring DfsBroker (hadoop)
> > #
> > (changed the name of the HdfsBroker.Server.fs.default.name from what I
> > really use, how to
> > determine what should really be there?)
> Does hadoop fs ls show anything? Make sure he ...default.name matches
> your conf/hadoop-site.xml. Can you send me your hypertable.cfg?
> > > I'll give it a try (tomorrow), maybe gen up a git-enabled ebuild
> > > unless you already have one?
> No, unfortunately we're not using gentoo in production here, so no
> ebuild yet. We probably have rpm and deb by the time we're in beta.
> But it'll be awesome to have one. Thanks.
Interesting, looks like you're doing something non-standard. All user
customizations of hadoop should really be in hadoop-site.xml, and you
need to make sure the fs.default.name in it exactly matches the
HdfsBroker...default.name in hypertable config.
> My conf/hadoop-site.xml is "empty" (only boiler plate comments).
> What parts need to match up between Hypertable and Hadoop config?
> (the hadoop-default.xml may have been edited since it seems we are
> on non-standard ports from what I can tell, but the site file is
> "empty"...)
> Was not able to do the asked for chmod on it, my hadoop has no such fs
> command.
> Might be the problem?
> ;;peter
> On May 1, 10:43 pm, Luke <vic...@gmail.com> wrote:
> > Thanks for the note, Peter...
> > On May 1, 3:45 pm, slowfood <pb...@plasm.com> wrote:
> > > To build I had to add:
> > > #include <iostream>
> > > to hypertable/src/cc/Common/Logger.h
> > > Without it I got an error from make:
> > > /home/admin/src/hypertable/src/cc/Common/Logger.h:31: error:
> > > 'cout' is not a member of 'std'
> > Interesting, I wonder why I didn't get an error. Maybe different
> > compiler version. What does your gcc -v say?
> > > With this fix things seemed to build.
> > > Still doesn't run though, and not much to go on as to why:
> > > # bin/start-master.sh hadoop
> > > DfsBroker (hadoop) hasn't come up yet, trying again in 5 seconds ...
> > > Num CPUs=4
> > > HdfsBroker.Port=38030
> > > HdfsBroker.Reactors=4
> > > HdfsBroker.Workers=20
> > > HdfsBroker.Server.fs.default.name=hdfs://motherlode001:9000
> > > Problem statring DfsBroker (hadoop)
> > > #
> > > (changed the name of the HdfsBroker.Server.fs.default.name from what I
> > > really use, how to
> > > determine what should really be there?)
> > Does hadoop fs ls show anything? Make sure he ...default.name matches
> > your conf/hadoop-site.xml. Can you send me your hypertable.cfg?
> > > > I'll give it a try (tomorrow), maybe gen up a git-enabled ebuild
> > > > unless you already have one?
> > No, unfortunately we're not using gentoo in production here, so no
> > ebuild yet. We probably have rpm and deb by the time we're in beta.
> > But it'll be awesome to have one. Thanks.
