Master terminated with errors - help

[sk]

Hi
I am trying to execute SampleWebApp workload from Faban.
I am getting following error.
Any suggestions what is going wrong here? It shows "Master terminated
with errors."

Thank you
Shikha

Time Host Level Message
13:30:52 INFO Starting sampleweb run using /opt/faban/output/
sampleweb.2G/
13:30:52 INFO START TIME : 3/20/12 1:30 PM
13:30:52 INFO Starting Registry.
13:30:53 webserver INFO Registry started.
13:31:23 INFO CmdService: Configured File...@143.182.13.202 on
server webserver
13:31:23 INFO CmdService: Configured File...@10.2.4.104 on server
webserver2
13:31:23 INFO Time difference to host webserver2 is 4,481 ms.
Attempting to set clock.
13:31:25 INFO Setting time succeeded for webserver2 after 1 retries.
Time difference is -1 ms.
13:31:25 INFO Gathering system configuration
13:31:25 INFO Setting up tools on machine 143.182.13.202
13:31:30 INFO Starting SampleWebAgent[0] on host 143.182.13.202.
13:31:35 webserver SEVERE

exception Master terminated with errors.

13:31:36 webserver WARNING [/usr/java/jdk1.7.0_03/bin/java, -
Dfaban.home=/opt/faban/, -Djava.security.policy=/opt/faban/config/
faban.policy, -Djava.util.logging.config.file=/opt/faban/config/
logging.webserver.properties, -Dfaban.registry.port=9998, -
Dfaban.logging.port=9999, -Xmx256m, -Xms64m, -XX:+DisableExplicitGC, -
cp, /opt/faban/benchmarks/sampleweb/lib/sampleweb.jar:/opt/faban/lib/
commons-httpclient-3.1.jar:/opt/faban/lib/commons-codec-1.2.jar:/opt/
faban/lib/fabanagents.jar:/opt/faban/lib/fabandriver.jar:/opt/faban/
lib/commons-logging.jar:/opt/faban/lib/fabancommon.jar,
com.sun.faban.driver.engine.PingMaster, RAMPUP]
stderr:
Exception in thread "main" java.rmi.ConnectException: Connection
refused to host: 143.182.13.202; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:
216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:128)
at com.sun.faban.driver.engine.MasterImpl_Stub.waitForState(Unknown
Source)
at com.sun.faban.driver.engine.PingMaster.waitForState(PingMaster.java:
88)
at com.sun.faban.driver.engine.PingMaster.main(PingMaster.java:104)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:
339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:
200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:
182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at
sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:
40)
at
sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:
146)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 6 more
13:31:36 INFO Ramp up started
13:31:36 SEVERE Master terminated with exit value 1
13:31:36 SEVERE
exception Run end failed!
13:31:48 webserver INFO [/opt/faban/bin/fenxi, process, /opt/faban/
output/sampleweb.2G/, /opt/faban/output/sampleweb.2G//post/, sampleweb.
2G]
stdout:
Creating database ... 9.14s
Parsing raw files ... 0.01s
Processing profile default_profile ...OK
Opening Database ... 1.17s
Executing queries ... 0.03s
Generating html pages ... 0.11s
13:31:48 INFO Killing benchmark
13:31:48 webserver INFO Killing tools
13:31:48 INFO killing CmdA...@143.182.13.202
Top PgUp PgDn Bottom

[Shanti Subramanyam (gmail)]

You can see the error message clearly enough - 'Connection refused'.

Make sure the master can talk to all the other machines via ssh without using a password.

See http://faban.org/1.0/docs/install.html - especially the Prerequisites section.

Shanti

[sk]

Hi Shanti,
Thanks.
The master terminates even before error this shows up.
The machines are able to ssh each other without password.
As, it also shows message (Time difference to host webserver2 is 4,481
ms. ), which indicates that ssh works fine initially.
And after that the master terminates.

Time Host Level Message
13:30:52 INFO Starting sampleweb run using /opt/
faban/output/
sampleweb.2G/
13:30:52 INFO START TIME : 3/20/12 1:30 PM
13:30:52 INFO Starting Registry.
13:30:53 webserver INFO Registry started.
13:31:23 INFO CmdService: Configured

FileAg...@143.182.13.202 on

server webserver
13:31:23 INFO CmdService: Configured

FileAg...@10.2.4.104 on server

webserver2
13:31:23 INFO Time difference to host webserver2 is
4,481 ms.
Attempting to set clock.
13:31:25 INFO Setting time succeeded for webserver2
after 1 retries.
Time difference is -1 ms.
13:31:25 INFO Gathering system configuration
13:31:25 INFO Setting up tools on machine
143.182.13.202
13:31:30 INFO Starting SampleWebAgent[0] on host
143.182.13.202.
13:31:35 webserver SEVERE
exception Master terminated with errors.

So, is there any thing else that might be happening and triggering the
master to terminate, which then causes the communication error.

Thanks
Shikha

On Apr 2, 9:17 pm, "Shanti Subramanyam (gmail)"

<shanti.subraman...@gmail.com> wrote:
> You can see the error message clearly enough - 'Connection refused'.
> Make sure the master can talk to all the other machines via ssh without
> using a password.

> Seehttp://faban.org/1.0/docs/install.html- especially the Prerequisites

> section.
>
> Shanti
>
>
>
>
>
>
>
> On Mon, Mar 26, 2012 at 12:04 AM, sk <shikha.khe...@gmail.com> wrote:
> > Hi
> > I am trying to execute SampleWebApp workload from Faban.
> > I am getting following error.
> > Any suggestions what is going wrong here? It shows "Master terminated
> > with errors."
>
> > Thank you
> > Shikha
>
> > Time                        Host        Level   Message
> > 13:30:52                INFO    Starting sampleweb run using
> > /opt/faban/output/
> > sampleweb.2G/
> > 13:30:52                INFO    START TIME : 3/20/12 1:30 PM
> > 13:30:52                INFO    Starting Registry.
> > 13:30:53        webserver       INFO    Registry started.
> > 13:31:23                INFO    CmdService: Configured

> > FileAg...@143.182.13.202 on

> > server webserver
> > 13:31:23                INFO    CmdService: Configured

> > FileAg...@10.2.4.104 on server

> > 13:31:48                INFO    killing CmdAg...@143.182.13.202
> > Top PgUp PgDn Bottom

[Shanti Subramanyam (gmail)]

Is the host named 'webserver' your driver machine or server machine?

To debug the failure, edit config/logging.properties and add the following line:

com.sun.faban.harness.level=FINEST

You will then see lots more messages in the log which will give us some idea of where the failure is occurring.

Shanti

[sk]

'webserver' - is the faban master +geocoder + mysql load
'webserver2' - faban agent + apache + olio + mysql

I will try with your suggestion.

Thanks

On Apr 3, 10:14 am, "Shanti Subramanyam (gmail)"

> > > Seehttp://faban.org/1.0/docs/install.html-especially the Prerequisites

[sk]

Hi
I tried it with the fine tuning. I see lot more messages with this.
But, nothing specific to this error message.
I got the same sequence of things there and then the "master
termination message"

:
:
:
15:10:09 INFO Starting UIDriverAgent[0] on host 143.182.13.202.
15:10:14 webserver SEVERE

exception Master terminated with errors.

On expanding the exception on master error message, it shows :

Exception:

Message:
java.lang.annotation.AnnotationTypeMismatchException: Incorrectly
typed data found for annotation element public abstract float
com.sun.faban.driver.BenchmarkDriver.threadPerScale() (Found data of
type class java.lang.Integer[1])

Stack Trace:
Class Method Line
sun.reflect.annotation.AnnotationTypeMismatchExceptionProxy
generateException 57
sun.reflect.annotation.AnnotationInvocationHandler invoke 75
$Proxy5 threadPerScale
com.sun.faban.driver.engine.BenchmarkDefinition read 173
com.sun.faban.driver.engine.MasterImpl runBenchmark 183
com.sun.faban.driver.engine.MasterImpl main 1608

Any idea what is the problem here.
Please suggest.

Thanks
Shikha

> > > > Seehttp://faban.org/1.0/docs/install.html-especiallythe Prerequisites

[Shanti Subramanyam (gmail)]

As the error indicates, threadsPerScale data type has been changed. Just change the value you have to a float type (e.g. threadPerScale = 1.0) in the driver source code and it should work.

Shanti