Problem viewing the Hadoop ports on the web browser

[Prakashan Korambath]

I was going through the exercises today and the word count example
worked, but I cannot see the port from the web page. It was ok on last
Wednesday though. Is there anything I am doing wrong? Thanks

http://i51r.idp.iu.futuregrid.org:50030


[train003@i51 ~]$ jps
22717 NameNode
23141 JobTracker
24728 Jps
23287 TaskTracker
22860 DataNode

[train003@i51 ~]$ netstat -plten | grep java
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 0.0.0.0:12500
0.0.0.0:* LISTEN 10070 34565 9012/
java
tcp 0 0 :::
58721 :::* LISTEN
10070 45617 10077/java
tcp 0 0 :::
50020 :::* LISTEN
10070 583782 22860/java
tcp 0 0 :::
49892 :::* LISTEN
10070 583288 22717/java
tcp 0 0 ::ffff:
127.0.0.1:9000 :::* LISTEN
10070 583294 22717/java
tcp 0 0 ::ffff:
127.0.0.1:9001 :::* LISTEN
10070 585920 23141/java
tcp 0 0 :::
53002 :::* LISTEN
10070 583578 22860/java
tcp 0 0 :::
50090 :::* LISTEN
10070 45670 10077/java
tcp 0 0 :::
50060 :::* LISTEN
10070 586230 23287/java
tcp 0 0 :::
48492 :::* LISTEN
10070 585918 23141/java
tcp 0 0 :::
50030 :::* LISTEN
10070 586019 23141/java
tcp 0 0 :::
50070 :::* LISTEN
10070 583400 22717/java
tcp 0 0 ::ffff:
127.0.0.1:48825 :::* LISTEN
10070 586277 23287/java
tcp 0 0 :::
50010 :::* LISTEN
10070 583703 22860/java
tcp 0 0 :::
50075 :::* LISTEN
10070 583708 22860/java

[Stephen TAK-LON WU]

Hi Prakashan,

 

I will contact the IT support team. (Actually, I have the same issue.)

 

They are off during the weekend, so, hopefully, it will works on Monday.

 

Best,

Stephen

--
Stephen Wu
Pervasive Technology Institute
Indiana University, Bloomington

[Prakashan Korambath]

Thanks Stephen.  When I did a ps auxw  | grep java on that node I saw java threads by other accounts also.  May be that is what is causing the problem I don't know.  If they reboot all the nodes before the class and clean up /tmp directory also then we should be ok.  I was also thinking after they do "qsub -I" students may want to clean up the localhost entry in $HOME/.ssh/known_hosts file because ssh gives that annoying warning messages.  I think this is because every node has its own ssh keys. 

Prakashan

[Carolyn]

I assume that is is a reference to Hands-on Exercise 4.

I am trying out this code on the bigdata.india.futuregrid.org site

So I am looking at
http://bigdata.india.futuregrid.org:50070/


What is the meaning of this information?
How do I know when the file system is operational?

I have done

cd ~/hadoop-0.20.2/bin
./hadoop fs -ls /

and I get nothing.

On Jul 24, 11:20 pm, Prakashan Korambath <koramb...@gmail.com> wrote:
> Thanks Stephen.  When I did a ps auxw  | grep java on that node I saw java
> threads by other accounts also.  May be that is what is causing the problem
> I don't know.  If they reboot all the nodes before the class and clean up
> /tmp directory also then we should be ok.  I was also thinking after they do
> "qsub -I" students may want to clean up the localhost entry in
> $HOME/.ssh/known_hosts file because ssh gives that annoying warning
> messages.  I think this is because every node has its own ssh keys.
>
> Prakashan
>
> On Sat, Jul 24, 2010 at 7:58 PM, Stephen TAK-LON WU <tak...@indiana.edu>wrote:
>
>
>
> > Hi Prakashan,
>
> > I will contact the IT support team. (Actually, I have the same issue.)
>
> > They are off during the weekend, so, hopefully, it will works on Monday.
>
> > Best,
> > Stephen
>

[Stephen TAK-LON WU]

Hi Carolyn,

 

Replies in lines:

On Mon, Jul 26, 2010 at 4:55 PM, Carolyn <carolynl...@gmail.com> wrote:

I assume that is is a reference to Hands-on Exercise 4.

I am trying out this code on the bigdata.india.futuregrid.org site

So I am looking at
http://bigdata.india.futuregrid.org:50070/

 

You are running the HDFS in the headnode of bigdata.india.futuregrid.org, please stop all the HDFS and MapReduce Daemon on the node you are working with now.

 

Then, follow the instruction here : http://salsahpc.indiana.edu/tutorial/futuregrid_access.html to get a FutureGrid machine.

 

Again, it's very important that HDFS, MapReduce Daemon, NaradaBroking and Twister need to run on FutureGrid work node, not HeadNode. Or, it may cause system failure or network connection overhead.

What is the meaning of this information?
How do I know when the file system is operational?

I have done

cd ~/hadoop-0.20.2/bin
./hadoop fs -ls /

 

 

For more information about how to use "-ls", please check here http://hadoop.apache.org/common/docs/current/hdfs_shell.html

 

You should put sth on the HDFS assoicated with a directory name, e.g. ./hadoop fs -put ~/sth_local_dir HDFS_remote_DIR_NAME. then use ./hadoop fs -ls HDFS_remote_DIR_NAME