Speed up testing while developing Guava

[milos.g...@gmail.com]

Hi All,

We developed a tool, called Ekstazi (www.ekstazi.org), that speeds up
the execution of tests by only running the tests that are affected by
recent code changes.  Ekstazi has already been integrated in
repositories of several (Apache) open source projects.

We tried Ekstazi with Guava and obtained significant savings in test
execution time (on average over several commits).  We created a patch
for Guava (*only* a couple of lines are added in pom.xml), which adds
a profile for running tests with Ekstazi.

  wget http://mir.cs.illinois.edu/gliga/projects/ekstazi/guava.patch
  git apply guava.patch
  # run the tests with Ekstazi
  mvn test -Pekstazi
  # the same command would execute 0 tests (as no changes are made)
  mvn test -Pekstazi

We would be happy to answer any question.

Thanks,
Milos and Lamyaa

p.s. Demo video (< 5min): https://www.youtube.com/watch?v=jE8K5_UCP28

[Christian Gruber]

Thanks - we'll take a peek at this. 

--
guava-...@googlegroups.com
Project site: https://github.com/google/guava
This group: http://groups.google.com/group/guava-discuss
 
This list is for general discussion.
To report an issue: https://github.com/google/guava/issues/new
To get help: http://stackoverflow.com/questions/ask?tags=guava
---
You received this message because you are subscribed to the Google Groups "guava-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guava-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/guava-discuss/dd77281f-827e-4c03-b6c4-5d13d1b3ee42%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joachim Durchholz]

Am 22.12.2014 um 22:01 schrieb milos.g...@gmail.com:
> We developed a tool, called Ekstazi (www.ekstazi.org), that speeds up
> the execution of tests by only running the tests that are affected by
> recent code changes.

Do you know how your change detection algorithm compares to Infinitest's?

[milos.g...@gmail.com]

Below is a short reply to your question.  As this group may not be the
best place to discuss Ekstazi, I would suggest that we continue the
discussion via emails or on the Ekstazi Google group
(https://groups.google.com/forum/#!forum/ekstazi).

Infinitest and Ekstazi differ in several aspects.  The two most
notable differences are 1) Infinitest collects dependencies
statically, while Ekstazi collects dependencies dynamically, and 2)
the (current) Infinitest implementation is oriented towards continuous
testing (and integration with IDEs), while the Ekstazi implementation
is oriented towards regression test selection (and integration with
build systems).

Infinitest is one of the best implementations of the test selection
algorithm known as class firewall
(http://www.engr.sjsu.edu/gaojerry/course/287/Class-Firewall.pdf).
Although this algorithm can provide benefits (i.e., reduce test
execution time) in some cases, several research projects (e.g.,
http://www.cc.gatech.edu/home/orso/papers/orso.shi.harrold.FSE04.pdf)
showed that this algorithm selects (on average) many tests for
execution.  In addition, static approaches to finding dependencies
have some limitations: supporting reflection, library changes, and
similar.  Also, Infinitest uses timestamps to detect modified
dependencies, which has not been the best solution for several build
systems.

Ekstazi implements a new algorithm that collects dependencies
dynamically.  This algorithm selects fewer tests than the algorithm
based on static dependencies.  Additionally, this algorithm supports
reflection, library changes, etc.  Finally, Ekstazi detects modified
dependencies based on the semantics of changes rather than on the
timestamps.

Milos

[Joachim Durchholz]

Am 14.01.2015 um 02:45 schrieb milos.g...@gmail.com:
> Below is a short reply to your question. As this group may not be the
> best place to discuss Ekstazi, I would suggest that we continue the
> discussion via emails or on the Ekstazi Google group
> (https://groups.google.com/forum/#!forum/ekstazi).

Thanks for the answer, it already provides the insights I was looking for.

Regards,
Jo