Serengeti 0.9, unable to create a cluster

[Nicolas Punzo]

Hello,

I have deployed Serengeti 0.9 (via the Big Data Extensions Appliance), and when I try to create a cluster, it fails leaving the following error message in the /opt/serengeti/logs/ironfan.log-<date> file:

[Thu, 27 Jun 2013 12:09:09 +0000] FATAL: RuntimeError: Can't find any nodes which provide hadoop-0.20-jobtracker. Did any node provide hadoop-0.20-jobtracker? Or is the Chef Solr Server down?

Has someone already met this issue?

Thanks in advance,

Nicolas

[Jesse Hu]

Sorry for the late reply.  Could you please attach the file /opt/serengeti/logs/ironfan.log-<date> ? Which hadoop distro are you using to create the cluster ? The default Apache hadoop distro?

-Jesse @Serengeti

在 2013年6月27日星期四UTC-7上午7时33分12秒，Nicolas Punzo写道：

[JunChi ZHANG]

Hi Jesse, 

I got the same error when deploying a GPHD 1.2 cluster with v0.9. I could deploy a native apache hadoop cluster succesfully. I'm attaching the logs for you. 

Thanks,

Junchi

[Jesse Hu]

HI Junchi,

Your log shows 10.110.187.226 [Thu, 04 Jul 2013 13:24:48 +0000] FATAL: RuntimeError: Can't find any nodes which provide hadoop-0.20-namenode. Did any node provide hadoop-0.20-namenode? Or is the Chef Solr Server down?

The root cause is the namenode service is not started successfully due to this problem :

10.110.187.225 [Thu, 04 Jul 2013 12:55:05 +0000] FATAL: Chef::Exceptions::ShellCommandFailed: execute[format namenode] (hadoop_cluster::bootstrap_format_namenode line 26) had an error: Chef::Exceptions::ShellCommandFailed: Expected process to exit with [0], but received '255'
10.110.187.225 ---- Begin output of
10.110.187.225     yes 'Y' | hadoop namenode -format
10.110.187.225
10.110.187.225     exit_status=$?
10.110.187.225     if [ $exit_status -eq 0 ]; then touch /mnt/hadoop/.namenode_formatted.log ; fi
10.110.187.225     exit $exit_status
10.110.187.225    ----
10.110.187.225 STDOUT:
10.110.187.225 STDERR: 13/07/04 12:55:03 INFO namenode.NameNode: STARTUP_MSG:
10.110.187.225 /************************************************************
10.110.187.225 STARTUP_MSG: Starting NameNode
10.110.187.225 STARTUP_MSG:   host = 10.110.187.225/10.110.187.225
10.110.187.225 STARTUP_MSG:   args = [-format]
10.110.187.225 STARTUP_MSG:   version = 1.0.3-gphd-1.2.0.0
10.110.187.225 STARTUP_MSG:   build =  -r ; compiled by 'hadoop' on Fri Oct 12 10:41:08 CST 2012
10.110.187.225 ************************************************************/
10.110.187.225 13/07/04 12:55:04 INFO util.GSet: VM type       = 64-bit
10.110.187.225 13/07/04 12:55:04 INFO util.GSet: 2% max memory = 17.77875 MB
10.110.187.225 13/07/04 12:55:04 INFO util.GSet: capacity      = 2^21 = 2097152 entries
10.110.187.225 13/07/04 12:55:04 INFO util.GSet: recommended=2097152, actual=2097152
10.110.187.225 13/07/04 12:55:04 INFO namenode.FSNamesystem: fsOwner=hdfs
10.110.187.225 13/07/04 12:55:04 INFO namenode.FSNamesystem: supergroup=supergroup
10.110.187.225 13/07/04 12:55:04 INFO namenode.FSNamesystem: isPermissionEnabled=false
10.110.187.225 13/07/04 12:55:04 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
10.110.187.225 13/07/04 12:55:04 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
10.110.187.225 13/07/04 12:55:04 INFO namenode.NameNode: Caching file names occuring more than 10 times
10.110.187.225 13/07/04 12:55:05 ERROR namenode.NameNode: java.io.IOException: Cannot create directory file:/mnt/sdc1/hadoop/hdfs/name/current
10.110.187.225     at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:297)
10.110.187.225     at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1320)
10.110.187.225     at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1339)
10.110.187.225     at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1187)
10.110.187.225     at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1294)
10.110.187.225     at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1311)

Let me check which GPHD version we has tested. Yours is 1.0.3-gphd-1.2.0.0.

Nicolas's problem is the jobtracker service is not started successfully.

-Jesse

在 2013年7月3日星期三UTC-7下午11时10分22秒，JunChi ZHANG写道：