Preserving test case output. running ATF test cases outside of kyua?

[Craig Rodrigues]

Hi,

When I run "kyua test", as part of test-case isolation,
kyua creates a temporary directory,

chdir's to that, then runs the test case.  After the test case has finished,
the temporary directory is removed.

How can I preserve the test directory?

For example, if I run a test case, sometimes the test case may

coredump, or write out a text file.  There are two scenarios that I am encountering:

(1)  During an automated test run, I want to preserve the output of failed test cases

      (core files, text files) in a tar ball.  Then I want to give the tar ball to developers for further

     analysis.

(2)  As an individual developer, I may want to do:

    "kyua test testcase1"

     and preserve the test output when trying to reproduce a test failure or analyze the problem.

Is there a way to do this?

Also, for plain test cases, these can simply be run outside of kyua.  However,
for atf test cases, it looks like these cannot be run outside of kyua at all.  Is that right,
or is there a way to do this?

Thanks.

--

Craig

[Julio Merino]

On Sat, Sep 27, 2014 at 1:02 AM, Craig Rodrigues <rod...@freebsd.org> wrote:
> and preserve the test output when trying to reproduce a test failure or
> analyze the problem.

My plan to solve this, which I haven't gotten to yet, was to add a new
"--work-directory" flag to debug so that you'd do:

kyua debug --work-directory=/tmp/mydir test:case1

If not given, then the work directory would be automatically
determined and cleaned up. If given, then kyua would just use the
provided directory and not remove it on exit.

It's currently tricky to implement this due to how the internals of
debug work... but I'm currently working on simplifying everything
related to test case execution to make this and other stuff (say,
parallel execution!) possible.

> Also, for plain test cases, these can simply be run outside of kyua.
> However,
> for atf test cases, it looks like these cannot be run outside of kyua at
> all. Is that right,
> or is there a way to do this?

Yes you can, just as with any other program.
"/path/to/atf/test/program test-case-name-1" will run the specified
test case. You can even pass that to "gdb --args" to use the
debugger.