Fwd: Question about Sparrow

[Kay Ousterhout]

Forwarding this to the Sparrow users list, which we're trying to use for questions so that others can benefit from the answer!

---------- Forwarded message ----------
From: mrhaoji <mrh...@gmail.com>
Date: Tue, Nov 12, 2013 at 2:15 PM
Subject: Question about Sparrow
To: kayous...@gmail.com

Hi, Kay

I'm trying to run and benchmark Sparrow to do some comparison. However, I met some question that confused me.

1. According to your paper, it says:

The client accepts a list of schedulers from the application and connects to the first scheduler in the list. 

How can I give the scheduler list when I submitting the job? Add some parameters while running the Frontend commond? 

java -XX:+UseConcMarkSweepGC -cp target/sparrow-1.0-SNAPSHOT.jar edu.berkeley.sparrow.daemon.SparrowDaemon -c sparrow.conf

java -cp target/sparrow-1.0-SNAPSHOT.jar edu.berkeley.sparrow.examples.SimpleBackend

java -cp target/sparrow-1.0-SNAPSHOT.jar edu.berkeley.sparrow.examples.SimpleFrontend

2. Does the schedulers list influence the whole throughput?

Thank a lot!

MrHaoJi.COM         Twitter.com/mrhaoji
Designed by MyParents©. Assembled in China.
Model No.:80s Serial No.:88CN06HN10YY

[Kay Ousterhout]

Hi,

The master branch right now just allows the user to specify a single scheduler (failover is in an experimental branch that we haven't yet merged into master).  You do this when running the frontend command, as you predicted, using the scheduler_host and scheduler_port config variables.  So, you would write a conf file (e.g., frontend.conf) that would look like:

scheduler_host = 1.2.3.4

scheduler_port = 12345

and then start the frontend with:

java -cp target/sparrow-1.0-SNAPSHOT.jar edu.berkeley.sparrow.examples.SimpleFrontend -c frontend.conf

However, the simplest way to run Sparrow is not to specify the host/port, and run the frontend on the same machine as the scheduler; in this case, the default parameters should make everything work correctly. 

If you use a list of schedulers, it doesn't affect the throughput in the normal case, because the list is only used on failover.  However, as I mentioned, that feature is not currently in the master branch.

-Kay