[wiki.randoop] push by - Edited wiki page through web user interface. on 2010-04-29 18:04 GMT

[ran...@googlecode.com]

Revision: c6e2760786
Author: carlos.pche
Date: Thu Apr 29 11:04:53 2010
Log: Edited wiki page through web user interface.
http://code.google.com/p/randoop/source/detail?repo=wiki&r=c6e2760786

Modified:
/UserManual.wiki

=======================================
--- /UserManual.wiki Thu Apr 29 10:50:00 2010
+++ /UserManual.wiki Thu Apr 29 11:04:53 2010
@@ -1,58 +1,97 @@
-Running Randoop
-The easiest way to run Randoop is by adding to your classpath the file
randoop.jar, provided with the Randoop distribution. Run Randoop by
invoking its main class randoop.main.Main. Randoop's interface is
command-based: it expects a specific command as the first argument,
followed by command arguments. (Currently, Randoop includes only two
commands, help and gentests.) For example, to run the help command, do:
-
+#summary Randoop user manual
+
+Contents:
+<wiki:toc max_depth="2" />
+
+(Under construction... Migrating from CSAIL...)
+
+= Running Randoop =
+
+The easiest way to run Randoop is by adding to your classpath the file
`randoop.jar` provided with the Randoop distribution. Run Randoop by
invoking its main class `randoop.main.Main`. Randoop's interface is
command-based: it expects a specific command as the first argument,
followed by command arguments. (Currently, Randoop includes only two
commands, `help` and `gentests`.) For example, to run the help command, do:
+
+{{{
java -classpath randoop.jar randoop.main.Main help
+}}}
+
Randoop will print out something like this:
+
+
+{{{
Randoop is a command-line tool that creates unit tests for Java.
It accepts one of the commands listed below. For the user manual,
-please visit http://people.csail.mit.edu/cpacheco/randoop/
-
-Type `help' followed by a command name to see documentation.
+please visit http://code.google.com/p/randoop/wiki/UserManual
+
+Type 'help' followed by a command name to see documentation.

Commands:

gentests -- Generates unit tests for a set of classes.
help -- Displays a help message for a given command.
-As the above message states, to get help on a specific command, run
Randoop with `help command-name' as arguments. For example:
-
+}}}
+
+As the above message states, to get help on a specific command, run
Randoop with `help` _command-name_ as arguments. For example:
+
+{{{
java -classpath randoop.jar randoop.main.Main help gentests
+}}}
+
Randoop will output the syntax, description and options related to the
gentests command.
-Generating JUnit tests
-This section shows an example use of Randoop with the goal of generating
JUnit test cases. Imagine we want to generate tests for the class
java.util.Collections, a utility class that defines several methods for
manipulating collections.
+
+= Generating JUnit tests =
+
+This section shows an example use of Randoop with the goal of generating
[http://www.junit.org JUnit] test cases. Imagine we want to generate tests
for the class `java.util.Collections`, a utility class that defines several
methods for manipulating collections.

The first important thing to keep in mind is that Randoop will only
generate tests using the classes you specify. In order to effectively test
Collections, You should probably also specify some helper classes,
including a class that generates collections. For this example, we will add
java.util.TreeSet to the mix.

Invoke Randoop as follows (all in a single line):

+{{{
java -classpath randoop.jar randoop.main.Main gentests
--testclass=java.util.TreeSet
--testclass=java.util.Collections
--timelimit=10
-Alternatively, you can create a file that lists the names of the classes
under test, and use the --classlist option:
-
+}}}
+
+Alternatively, you can create a file that lists the names of the classes
under test, and use the `--classlist` option:
+
+{{{
java -classpath randoop.jar randoop.main.Main gentests
--classlist=myclasses.txt
--timelimit=10
-Where the contents of myclasses.txt are as follows.
-myclasses.txt:
-
+}}}
+
+Where the contents of `myclasses.txt` are as follows.
+
+{{{
java.util.Collections
java.util.TreeSet
-After 10 seconds, Randoop stops generating tests (if you omit the
--timelimit option, Randoop's default behavior is to generate tests for 1
minute). The last thing Randoop prints out is the name of the JUnit files
containing the tests it generated. You should see a message similar to the
following:
-
+}}}
+
+After 10 seconds, Randoop stops generating tests (if you omit the
`--timelimit` option, Randoop's default behavior is to generate tests for 1
minute). The last thing Randoop prints out is the name of the JUnit files
containing the tests it generated. You should see a message similar to the
following:
+
+{{{
Created file: my/home/directory/RandoopTest0.java
Created file: my/home/directory/RandoopTest.java
done.
-The main test driver is in class RandoopTest. You can now compile and run
the tests. Don't forget to include randoop.jar in the classpath when you
compile and run the tests. In this example, you should also add the current
directory (".") when you run the tests.
-
+}}}
+The main test driver is in class `RandoopTest`. You can now compile and
run the tests. Don't forget to include randoop.jar in the classpath when
you compile and run the tests. In this example, you should also add the
current directory (".") when you run the tests.
+
+{{{
javac -classpath randoop.jar RandoopTest*.java
java -classpath .:randoop.jar junit.textui.TestRunner RandoopTest
+}}}
+
JUnit will execute the generated tests. Most will pass, and some will
fail. The next section describes the kinds of tests that Randoop generates,
and the reason why they may fail.
-Interpreting the Results
+
+= Interpreting the Results =
+
Randoop generates two kinds of unit tests, regression tests and
contract-violating tests.

-Regression tests
+== Regression tests ==
+
Most of the tests that Randoop generates will probably pass. They are
regression tests: they record the current behavior of the classes under
test. Regression tests are useful because they can alert you in the future
if you make a change to your code that changes the external behavior of the
classes. Here is an example of a regression test for the TreeSet class we
generated tests for in the previous section.
+
+{{{
// This test passes when executed
public void test10() throws Throwable {

@@ -63,13 +102,20 @@
// Regression assertion (captures the current behavior of the code)
assertTrue(var2 == false);
}
+}}}
+
This test will pass when you run it right after executing it. But notice
that it captures an important behavior of the method TreeSet.contains: it
returns false if the set is empty. If later, as the developers of this
class, we introduced an error that caused contains to return true on an
empty set, the test would fail and thus alert us to the error.
-Contract-violating tests
+
+== Contract-violating tests ==
+
Some of the tests that Randoop generated may fail when you first run the
tests, indicating that an assertion was violated. These are
contract-violating tests. The failure suggests a potential error in one or
more classes under test. A contract-violating test shows a specific use of
a class under test that leads to a violation of an API contract. A contract
is a property that should hold (e.g. an object invariant, or a method
postcondition), and a contract violation suggests an error. Currently,
Randoop checks that the classes under test exhibit the following properties:
-Given a non-null object o, o.equals(o) should always return true, and
o.equals(null) should always return false.
-The hashCode() and toString() methods should not throw an exception.
-Take a look at the generated unit tests. Open the file RandoopTest0.java.
Each unit test consists of a snippet of code that use the classes under
test, along with assertions stating properties that fail to hold. For
example, one of the tests might look as follows (actually, it will probably
be longer, but we show a short test for simplicity).
-
+
+* Given a non-null object `o`, `o.equals(o)` should always return `true`,
and `o.equals(null)` should always return `false`.
+* The `hashCode()` and `toString()` methods should not throw an exception.
+
+Take a look at the generated unit tests. Open the file
`RandoopTest0.java`. Each unit test consists of a snippet of code that use
the classes under test, along with assertions stating properties that fail
to hold. For example, one of the tests might look as follows (actually, it
will probably be longer, but we show a short test for simplicity).
+
+{{{
// This test fails when executed
public static void test1() {
LinkedList var0 = new LinkedList();
@@ -80,82 +126,14 @@
// Checks the contract: var3.equals(var3)
Assert.assertTrue(var3.equals(var3) == true);
}
-This test shows a scenario that makes it possible to create a set that
does not preserve reflexivity of equality, a property specified in the API
for java.lang.Object. If you spent some time with the debugger, you would
discover that the erroneous behavior arises because the constructor call
new TreeSet(var0) fails to throw a ClassCastException, which it should
because the element in var0 is not Comparable (see the API for
java.util.TreeSet). This is an error in the TreeSet constructor. Later on,
somewhere deep inside the call to var3.equals(var3), a ClassCastException
is thrown, but it is caught, and the equals method ends up returning true.
This tests reveals at least one error, and possibly two, that could be
fixed as follows:
-
-The constructor to TreeSet should throw an exception when given a
single-element list with a non-comparable element.
-The first statement in the equals method in SynchronizedSet should be
- if (o == this) return true;
-In general, all implementations of equals should include this check.
-Note that not all failing tests that Randoop generates may reveal errors.
Some tests may exhibit behavior that represents normal operation of the
classes under test. For example, consider the output of Randoop on the
class java.util.Formatter.
-
-Randoop Commands
-This section describes Randoop's commands. The information presented here
is the same as that which you can get via Randoop's help command.
-gentests
-Usage: gentests OPTIONS
-Where: At least one class is specified via `--testclass' or `--classlist'.
-Summary. Attempts to generate JUnit tests that capture the behavior of the
classes under test and/or find contract violations. Randoop generates tests
using feedback-directed random test generation.
-Input: One or more names of classes to test. A class to test can be
specified via the `--testclass=' or `--classlist=' options.
-
-Output: A JUnit test suite (as one or more Java source files). The tests
in the suite will pass when executed using the classes under test.
-Example use:
-java randoop.main.Main gentests --testclass=java.util.Collections
--testclass=java.util.TreeSet
-Notes.
-
-Randoop executes the code under test, with no mechanisms to protect your
system from harm resulting from arbitrary code execution. If random
execution of your code could have undesirable effects (e.g. deletion of
files, opening network connections, etc.) make sure you execute Randoop in
a sandbox machine.
-Randoop will only use methods from the classes that you specify for
testing. If Randoop is not generating tests for a particular method, make
sure that you are including classes for the types that the method requires.
Otherwise, Randoop may fail to generate tests due to missing input
parameters.
-Randoop is designed to be deterministic when the code under test is itself
deterministic. This means that two runs of Randoop will generate the same
tests. To get variation across runs, use the --randomseed option.
-Options
-
---check-object-contracts=boolean
-Default: true
-Check java.lang.Object contracts, e.g. equals(Object) is reflexive,
hashCode() throws no exceptions, etc.
---testclass=string
-Default: []
-Specify the fully-qualified name of a class under test.
---classlist=string
-Default: no default.
-Specify the name of a file that contains a list of classes under test.
Eachclass is specified by its fully qualified name on a separate line.
---methodlist=string
-Default: no default.
-Specify the name of a file that contains a list of methods under test.
Eachmethod is specified on a separate line.
---inputlimit=int
-Default: 1000000
-Used to determine when to stop test generation. Generation stops when
either the time limit (--timelimit=int) OR the input limit
(--inputlimit=int) is reached. Note that the number of tests output may be
smaller than then number of inputs created, because redundant and illegal
inputs may be discarded.
---timelimit=int
-Default: 100
-Used to determine when to stop test generation. Generation stops when
either the time limit (--timelimit=int) OR the input limit
(--inputlimit=int) is reached.
---testsperfile=int
-Default: 500
-Maximum number of tests to write to each JUnit file.
---junit-classname=string
-Default: RandoopTest
-Name of the JUnit file containing Randoop-generated tests.
---junit-package-name=string
-Default:
-Name of the package that the generated JUnit files should have.
---junit-output-dir=string
-Default: no default.
-Name of the directory to which JUnit files should be written.
---randomseed=int
-Default: 0
-The random seed to use in the generation process
---maxsize=int
-Default: 100
-Do not generate tests with more than statements
---forbid-null=boolean
-Default: true
-Forbid Randoop to use null as input to methods. IMPORTANT: even if this
option is set to true, null is only used if there is no non-null values
available.
---null-ratio=double
-Default: 0.0
-Use null with the given frequency. [TODO explain]
---dont-output-tests=boolean
-Default: false
-Run Randoop but do not create JUnit tests (used in research experiments).
---log=filewriter
-Default: no default.
-Log lots of information to a given file name (`filewriter' is the name of
a file).
-help
-Usage: help
-Summary. Displays a help message for a given command.
-Input: None (for the general help message), or the name of a command (for
command-specific help).
-Output: A help message is printed to stdout.
+}}}
+
+This test shows a scenario that makes it possible to create a set that
does not preserve reflexivity of equality, a property specified in the API
for `java.lang.Object`. If you spent some time with the debugger, you would
discover that the erroneous behavior arises because the constructor call
new `TreeSet(var0)` fails to throw a `ClassCastException`, which it should
because the element in `var0` is not `Comparable` (see the API for
`java.util.TreeSet`). This is an error in the `TreeSet` constructor. Later
on, somewhere deep inside the call to `var3.equals(var3)`, a
`ClassCastException` is thrown, but it is caught, and the equals method
ends up returning true. This tests reveals at least one error, and possibly
two, that could be fixed as follows:
+
+* The constructor to TreeSet should throw an exception when given a
single-element list with a non-comparable element.
+* The first statement in the equals method in SynchronizedSet should be
+{{{if (o == this) return true;}}}
+
+In general, all implementations of equals should include this check.
+
+Note that not all failing tests that Randoop generates may reveal errors.
Some tests may exhibit behavior that represents normal operation of the
classes under test. For example, consider the output of Randoop on the
class java.util.Formatter.