GPS Example Not Working

[Simone Centellegher]

Hi!

I'm trying to run the gps example I found in the documentation

https://sites.google.com/site/gpsdocumentation/home/how-to-program-and-run-gps

I have followed all the documentation instructions but I can't get gps work on a small amazon ec2 cluster (1 master, 2 slaves)

when I run the example i get this!

ubuntu@ip-172-31-20-7:~/trunk/master-scripts$ ./start_gps_nodes.sh 2 PROVA1 \

> "-ifs /data/soc-Epinions1-d-n.txt \

> -hcf /usr/local/hadoop-1.2.1/conf/core-site.xml \

> -jc gps.examples.pagerank.PageRankVertex###JobConfiguration \

> -mcfg /machine-configs/test_machine_config.cfg \

> -log4jconfig ~/trunk/conf/log4j.config \

> -other -max###40"

master GPS_DIR=/home/ubuntu/trunk

starting gps master locally

1: /home/ubuntu/trunk/scripts

2: -1

3: 2

4: PROVA1

ubuntu

starting gps worker /home/ubuntu/trunk/scripts

-Xincgc -Xms50M -Xmx50M

1 172.31.20.9 2

ssh -i ~/hadoopkey.pem -o StrictHostKeyChecking=no 172.31.20.9 "/home/ubuntu/trunk/scripts/start_gps_node.sh  2 PROVA1 \"-ifs /data/soc-Epinions1-d-n.txt -hcf /usr/local/hadoop-1.2.1/conf/core-site.xml -jc gps.examples.pagerank.PageRankVertex###JobConfiguration -mcfg /machine-configs/test_machine_config.cfg -log4jconfig ~/trunk/conf/log4j.config -other -max###40\""running command: ssh -i ~/hadoopkey.pem -o StrictHostKeyChecking=no 172.31.20.9 /home/ubuntu/trunk/scripts/start_gps_node.sh /home/ubuntu/trunk/scripts 0 2 PROVA1 "-ifs /data/soc-Epinions1-d-n.txt -hcf /usr/local/hadoop-1.2.1/conf/core-site.xml -jc gps.examples.pagerank.PageRankVertex###JobConfiguration -mcfg /machine-configs/test_machine_config.cfg -log4jconfig /home/ubuntu/trunk/conf/log4j.config -other -max###40"

./start_gps_node.sh: line 30: /home/ubuntu/var/tmp/PROVA1-machine-1-output.txt: No such file or directory

1: /home/ubuntu/trunk/scripts

2: 0

3: 2

4: PROVA1

ubuntu

starting gps worker /home/ubuntu/trunk/scripts

-Xincgc -Xms256M -Xmx256M

2 172.31.20.8 2

ssh -i ~/hadoopkey.pem -o StrictHostKeyChecking=no 172.31.20.8 "/home/ubuntu/trunk/scripts/start_gps_node.sh  2 PROVA1 \"-ifs /data/soc-Epinions1-d-n.txt -hcf /usr/local/hadoop-1.2.1/conf/core-site.xml -jc gps.examples.pagerank.PageRankVertex###JobConfiguration -mcfg /machine-configs/test_machine_config.cfg -log4jconfig ~/trunk/conf/log4j.config -other -max###40\""running command: ssh -i ~/hadoopkey.pem -o StrictHostKeyChecking=no 172.31.20.8 /home/ubuntu/trunk/scripts/start_gps_node.sh /home/ubuntu/trunk/scripts 1 2 PROVA1 "-ifs /data/soc-Epinions1-d-n.txt -hcf /usr/local/hadoop-1.2.1/conf/core-site.xml -jc gps.examples.pagerank.PageRankVertex###JobConfiguration -mcfg /machine-configs/test_machine_config.cfg -log4jconfig /home/ubuntu/trunk/conf/log4j.config -other -max###40"

1: /home/ubuntu/trunk/scripts

2: 1

3: 2

4: PROVA1

ubuntu

starting gps worker /home/ubuntu/trunk/scripts

-Xincgc -Xms256M -Xmx256M

The documentation says that the results should be in the hadoop fs in /user/$USER/gps/output/ but I cannot find it! So i'm assuming that there are some error in my gps configuration.. 

can somebody halp me?

Thanks,

Simone

[Semih Salihoglu]

This output looks fine but is not very helpful for finding the problem I think. Can you send the output from the workers and the master? By default the log files are under /var/tmp/{execution-name}-machine*-output.txt.

Best,

semih

--
You received this message because you are subscribed to the Google Groups "stanfordgpsusers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stanfordgpsuse...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Simone Centellegher]

In the attachments there are the log files of the workers! 

I cannot find any output file neither in the master nor the workers..

it seems that there is the file ~/trunk/conf/log4j.config missing, but I have checked in both the workers and the file is exactly there!

And there are some connection problem..

This is the content of the configuration files I have edited:

in the master, the slaves file ~/trunk/master-scripts/slaves contains the ip addresses of the workers:

172.31.20.9

172.31.20.8

and the machine config file ~/trunk/test-data/test_machine_config.cfg contains

-1 172.31.20.7 1111

0 172.31.20.9 2222

1 172.31.20.8 3333