Testing Distributions

[Peter Li]

Is there a way to run test-distributions so that when one distribution
test fails, the rest keep running?


Peter

[Ben Goodrich]

Add -k to the make test-distributions call

[Daniel Lee]

The issue isn't the -k flag. When you add the parallelization flag with it being greater or equal to 3, make doesn't detect failures properly. The solution is to run it without the -j flag or set it to 2. The other solution is to run the target again after running it once without the -j flag. That will pick up the failure and it won't spend the time compiling. 

I believe the way the target is set up, it will compile everything first, so it won't detect failures until runtime if it compiles. You could run a single test if you know what distribution you are testing. 

Daniel

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Ben Goodrich]

On Saturday, June 22, 2013 12:35:26 PM UTC-4, Daniel Lee wrote:

The issue isn't the -k flag. When you add the parallelization flag with it being greater or equal to 3, make doesn't detect failures properly. The solution is to run it without the -j flag or set it to 2. The other solution is to run the target again after running it once without the -j flag.

The other solution would be to fix the makefile. Is there a reproducible way to trigger the missed failure with >= 3 processes?

Ben

 

[Daniel Lee]

Sorry, this isn't a problem with our makefile. Yes, just build a test that fails and run test-unit.

[Ben Goodrich]

On Saturday, June 22, 2013 12:55:36 PM UTC-4, Daniel Lee wrote:

The other solution would be to fix the makefile. Is there a reproducible way to trigger the missed failure with >= 3 processes?

Sorry, this isn't a problem with our makefile. Yes, just build a test that fails and run test-unit.

Then where does the problem originate? This erroneous patch

diff --git a/src/test/agrad/agrad_test.cpp b/src/test/agrad/agrad_test.cpp
index dc15f91..772ec2a 100644
--- a/src/test/agrad/agrad_test.cpp
+++ b/src/test/agrad/agrad_test.cpp
@@ -1,6 +1,6 @@
 #include <gtest/gtest.h>
 #include <stan/agrad/agrad.hpp>
-#include <test/agrad/util.hpp>
+#include <test/agrad/utils.hpp>

 TEST(AgradRev, multiple_grads) {
   for (int i = 0; i < 100; ++i) {

yields the expected behavior

make clean-all > /dev/null && make CC=clang++ -j9 test-unit > /tmp/clang++.log && echo "success"
...
src/test/agrad/agrad_test.cpp:3:10: fatal error: 'test/agrad/utils.hpp' file not found
#include <test/agrad/utils.hpp>
         ^
1 error generated.
make: *** [test/agrad/agrad.o] Error 1
make: *** Waiting for unfinished jobs....

Ben

[Daniel Lee]

Windows and Mac do not behave that way. I will verify later.

[Daniel Lee]

I take it back, it doesn't behave that way on a Mac for compiler errors. I'm checking for a runtime test failure.

[Daniel Lee]

Ok. Runtime failures operate as expected. I take it all back. Ben, you're right.

And, I misread Peter's first post. Listen to Ben. -k is the correct flag.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.