test listener
Manav Brar
/**
* The underlying {@link Set<String>} instance which contains the
* {@link WebDriver} session id. This listener only expects
* {@link AbstractSessionIds} implemented in all the tests.
*
* @author mbrar
*/
public class SauceMobileSiteListener extends TestListenerAdapter {
private static final Logger LOGGER = LoggerFactory.getLogger(SauceMobileSiteListener.class);
private static final String SELENIUM_IS_LOCAL = "SELENIUM_IS_LOCAL";
private CounterMobileSite count;
/**
* The instance of the Sauce OnDemand Java REST API client.
*/
private SauceREST sauceREST;
/**
* Treat this test as a local test or run in SauceLabs.
*/
private boolean isLocal = false;
/**
*
* @param testContext
*/
@Override
public void onStart(final ITestContext testContext) {
super.onStart(testContext);
final String local = System.getenv(SELENIUM_IS_LOCAL);
if (local != null && local.isEmpty()) {
isLocal = true;
}
}
/**
* @param result
*/
@Override
public void onTestStart(final ITestResult result) {
super.onTestStart(result);
if (isLocal) {
return;
}
this.sauceREST = SelectorsInputs.identity(SauceREST.class, "sauceRest");
}
private void initlize(final ITestResult result) {
if (result.getInstance() instanceof AbstractSessionIds) {
final AbstractSessionIds sessionIds = (AbstractSessionIds) result.getInstance();
if (count == null) {
this.count = new CounterMobileSite(7, null, 0);
}
final AbstractMobileSite local = new MobileImmutable(null, "", null);
final Set<String> localSet = sessionIds.getImmutableList();
final Boolean validateSize = count.getCount().compareTo(count.getTop()) <= 0 && !localSet.isEmpty();
if (validateSize) {
count = count.withImmutable(local.withSessionId(localSet.iterator().next()));
if (!localSet.isEmpty()) {
localSet.remove(count.getImmutable().getSessionId());
count.increment();
}
}
LOGGER.info("SauceOnDemandSessionID={} job-name={}", count.getImmutable().getSessionId(), result
.getMethod().getMethodName());
}
}
/**
* @param testResult
*/
@Override
public void onTestFailure(final ITestResult testResult) {
super.onTestFailure(testResult);
if (isLocal) {
return;
}
validateNextSuit();
initlize(testResult);
if (this.sauceREST != null
&& count.getImmutable().getSessionId() != null) {
markJobAsFailed();
printPublicJobLink();
}
}
private void validateNextSuit() {
if (count != null && count.score()) {
count.resetCount();
}
}
private void markJobAsFailed() {
final Map<String, Object> updates = new HashMap<String, Object>();
updates.put("passed", false);
Utils.addBuildNumberToUpdate(updates);
sauceREST.updateJobInfo(count.getImmutable().getSessionId(), updates);
}
private void printPublicJobLink() {
final String authLink = this.sauceREST.getPublicJobLink(count
.getImmutable().getSessionId());
LOGGER.info("Test failure public job link: {}", authLink);
}
/**
* @param testResult
*/
@Override
public void onTestSuccess(final ITestResult testResult) {
super.onTestSuccess(testResult);
if (isLocal) {
return;
}
validateNextSuit();
initlize(testResult);
markJobAsPassed();
}
private void markJobAsPassed() {
if (this.sauceREST != null && count.getImmutable().getSessionId() != null) {
final Map<String, Object> updates = new HashMap<String, Object>();
updates.put("passed", true);
Utils.addBuildNumberToUpdate(updates);
sauceREST.updateJobInfo(count.getImmutable().getSessionId(), updates);
}
}
}
Esko Luontola
> need to replace testng with jumi simpletest runner but a few
> prerequisite required.
>
> I need to replace testng listener I currently use in my current site of
> tests, what would I use in jumi?
interfaces it implements) is Jumi's SuiteListener interface [1].
Currently Jumi exposes those events only to the JVM process which
invokes Jumi (i.e. a build tool or IDE) through the
JumiLauncher.getEventStream() method [2].
I understand that those events might also be useful to test code running
inside the test runner JVM process. Would you please tell more about
your use case, so that we could think of a good solution to it? Jumi is
still early access software and it is missing many features which more
mature test runners have.
Also, I'm curious to hear that why you are replacing TestNG with Jumi's
SimpleUnit, which is meant to be just a minimal example for testing
framework developers instead of a full testing framework. A more general
purpose solution would be to create an adapter for TestNG so that you
can run TestNG tests inside Jumi [3].
[1]
https://github.com/orfjackal/jumi/blob/v0.5.437/jumi-core/src/main/java/fi/jumi/core/api/SuiteListener.java
[2]
https://github.com/orfjackal/jumi/blob/v0.5.437/jumi-launcher/src/main/java/fi/jumi/launcher/JumiBootstrap.java#L71
https://github.com/orfjackal/jumi/blob/v0.5.437/jumi-launcher/src/main/java/fi/jumi/launcher/JumiLauncher.java#L31
[3] Jumi already has such an adapter for JUnit. It converts events from
JUnit's RunListener to Jumi's SuiteNotifier, making it possible to run
JUnit tests inside Jumi. See
https://github.com/orfjackal/jumi/blob/v0.5.437/jumi-core/src/main/java/fi/jumi/core/junit/JUnitRunListenerAdapter.java
--
Esko Luontola
www.orfjackal.net
Manav Brar
- light weight in terms on it's reliance on actors.
- tests running in parallel still show results for each individual separately
- If each test is running in a light weight jumi actor then maybe will have an ability to restart on failure an UI test based on Webdriver
- Q: ON adapter for JUnit. Does the use of this adapter would let me run the tests in parallel across multiple classes and not use JUnit native threads, (regular java thread is basically a OS thread on a VM have limited resources).
Esko Luontola
> At work run all the tests in parallel which was 16 limit when I started
> writing them, testNg and JUnit try to use native threads they get very
> heavy when things have to run for a long time
>
> Like the fact that jumi is
>
There is nothing special about Jumi's threads that would magically make
them faster. It's a normal thread pool, the same kind as for example
TestNG uses.
Modern operating systems are good at scheduling threads, so you would
typically need to have thousands of threads to have noticable overhead
due to thread switching or the thread's stack size (typically 512KB).
What matters is the work that the threads do. If the tests do lots of IO
or use lots of memory, then that may drain those resources of the
computer, causing it to slow down or run out of memory. If the tests
just use the CPU, then it won't really matter much even if there are ten
times more threads than CPU cores.
> 2. tests running in parallel still show results for each individual
> separately
Yes, that's something that Jumi does well.
> 3. If each test is running in a light weight jumi actor then maybe will
restarting. Also the Jumi test runner doesn't have any built-in support
for restarting failing tests.
But individual testing frameworks may support it regardless. It can even
be done with a simple JUnit rule: https://gist.github.com/abyx/897229
> 4. Q: ON adapter for JUnit. Does the use of this adapter would let me
> native threads, (regular java thread is basically a OS thread on a
> VM have limited resources).
thread pool for running them, which defaults to as many threads are
there are CPU cores (it's configurable with
`JumiBootstrap.daemon.setTestThreadsCount(int)`). There are only one
kinds of threads in Java and there's nothing special about Jumi's use of
them.
--
Esko Luontola
www.orfjackal.net
Manav Brar
Let say I need to run 46(regular threads) test concurrently, each test in turn spawn 40 to 50 actors that would (poll the DOM)[this is already encapsulated] for WebElements, test need to fail if WebElement not found.
as for restart https://gist.github.com/abyx/897229 might be fine, thank you