hmc_refactor ready for merge?

[Michael Betancourt]

I updated command.hpp to mimic the interface currently in master and tested all of the individual algorithms and I think we're ready to consider merging hmc_refactor in master. Are there any tests I should run to validate? Should we hold off on considering a merge until after releasing 1.3?

-Mike

[Daniel Lee]

I'm assuming you ran the unit tests. The next is to run the model tests. There's one in there now that checks the command line options. Want to run:

make test/models/command

If that works out ok:

make test-models

On Wed, Apr 3, 2013 at 12:18 PM, Michael Betancourt <betan...@gmail.com> wrote:

I updated command.hpp to mimic the interface currently in master and tested all of the individual algorithms and I think we're ready to consider merging hmc_refactor in master.  Are there any tests I should run to validate?  Should we hold off on considering a merge until after releasing 1.3?

-Mike

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Michael Betancourt]

Actually I just did a bunch of detailed checks on the algorithmic behavior. How long will the unit/model tests take to run? My computer isn't anywhere near as beefy as everyone else's....

[Marcus Brubaker]

Given how large and core the hmc_refactor changes are, I'd suggest
waiting until after 1.3 to merge it. Then we can have a full release
cycle for us (and anyone else running from git) to shake out any bugs.
If we merge it now, before 1.3, there's a real chance to release a
significant bug,

On Wed, Apr 3, 2013 at 12:34 PM, Michael Betancourt

[Bob Carpenter]

On 4/3/13 12:34 PM, Michael Betancourt wrote:
> Actually I just did a bunch of detailed checks on the algorithmic behavior. How long will the unit/model tests take to run? My computer isn't anywhere near as beefy as everyone else's....

The unit tests shouldn't take that long. What we should be
doing is writing new unit tests for all the new features.
Anything with setters and getters needs to have their
behavior tested. I'm not sure how to test some of the
other interfaces --- there's not much in the way of unit
testing for the mcmc package or command.

The model tests take a while. But if you run them under
clang++ they'll be much faster. How many cores do you have?
You want to run -j2 if you have two cores.

If there's a branch I can check out to run them, I can
run them later tonight after I get home.

- Bob

[Michael Betancourt]

Cool, I'll give them a try. Worst case I'll run the model tests overnight.

[Daniel Lee]

I forgot to tell you how long they run. On the build box, it runs in 1.5 hours.

[Ben Goodrich]

On Wednesday, April 3, 2013 1:16:30 PM UTC-4, Michael Betancourt wrote:

Cool, I'll give them a try.  Worst case I'll run the model tests overnight.

There are two includes that don't exist

goodrich@CYBERPOWERPC:/opt/stan$ git grep -F -n "<stan/mcmc/adaptive_sampler.hpp>" src/ | cat
src/stan/mcmc/nuts.hpp:15:#include <stan/mcmc/adaptive_sampler.hpp>
src/test/file_includes/stan_mcmc_adaptive_sampler_test.cpp:1:#include <stan/mcmc/adaptive_sampler.hpp>

goodrich@CYBERPOWERPC:/opt/stan$ ls src/stan/mcmc
adapter.hpp  chains.hpp  hamiltonian.hpp  hmc_base.hpp  integrator.hpp  mcmc_base.hpp  nuts.hpp  static_hmc.hpp  util.hpp

Did you mean <stan/mcmc/adapter.hpp>?

Ben

[Michael Betancourt]

You have an old pull -- there shouldn't be a "nuts.hpp" file.

[Ben Goodrich]

On Wednesday, April 3, 2013 2:01:49 PM UTC-4, Michael Betancourt wrote:

You have an old pull -- there shouldn't be a "nuts.hpp" file.

OK, now I re-re-pulled and am up to commit 5bdd957 "Reorganized directory structure". Now I am getting

src/test/file_includes/stan_mcmc_adaptive_sampler_test.cpp:1:10: fatal error: 'stan/mcmc/adaptive_sampler.hpp' file not found
#include <stan/mcmc/adaptive_sampler.hpp>

Ben

[Michael Betancourt]

Ah, I see the problem. I haven't realized that the unit tests were that fine-grained. I'll start fixing the tests.

Thanks.

[Ben Goodrich]

On Wednesday, April 3, 2013 2:16:59 PM UTC-4, Michael Betancourt wrote:

Ah, I see the problem.  I haven't realized that the unit tests were that fine-grained.  I'll start fixing the tests.

So far, I've had to comment out these tests due to missing includes

#       modified:   src/test/file_includes/stan_mcmc_adaptive_cdhmc_test.cpp
#       modified:   src/test/file_includes/stan_mcmc_adaptive_sampler_test.cpp
#       modified:   src/test/file_includes/stan_mcmc_dualaverage_test.cpp
#       modified:   src/test/file_includes/stan_mcmc_sampler_test.cpp

And I hit this too

src/stan/mcmc.hpp:4:10: fatal error: 'stan/mcmc/adaptive_cdhmc.hpp' file not found
#include "stan/mcmc/adaptive_cdhmc.hpp"

Ben

[Ben Goodrich]

On Wednesday, April 3, 2013 2:24:11 PM UTC-4, Ben Goodrich wrote:

On Wednesday, April 3, 2013 2:16:59 PM UTC-4, Michael Betancourt wrote:

Ah, I see the problem.  I haven't realized that the unit tests were that fine-grained.  I'll start fixing the tests.

Doing a make test-all had no serious problems and took me about 100 minutes with -j8 and clang++. Most of the errors were just includes

goodrich@CYBERPOWERPC:/opt/stan$ grep -A 2 -F "fatal error" /tmp/hmc_refactor.txt
src/test/file_includes/stan_mcmc_adaptive_sampler_test.cpp:1:10: fatal error: 'stan/mcmc/adaptive_sampler.hpp' file not found
#include <stan/mcmc/adaptive_sampler.hpp>
         ^
--
src/test/file_includes/stan_mcmc_adaptive_cdhmc_test.cpp:1:10: fatal error: 'stan/mcmc/adaptive_cdhmc.hpp' file not found
#include <stan/mcmc/adaptive_cdhmc.hpp>
         ^
--
src/test/file_includes/stan_mcmc_dualaverage_test.cpp:1:10: fatal error: 'stan/mcmc/dualaverage.hpp' file not found
#include <stan/mcmc/dualaverage.hpp>
         ^
--
src/test/file_includes/stan_mcmc_sampler_test.cpp:1:10: fatal error: 'stan/mcmc/sampler.hpp' file not found
#include <stan/mcmc/sampler.hpp>
         ^
--
src/test/file_includes/stan_mcmc_hmc_base_test.cpp:1:10: fatal error: 'stan/mcmc/hmc_base.hpp' file not found
#include <stan/mcmc/hmc_base.hpp>
         ^
--
src/test/file_includes/stan_mcmc_adaptive_hmc_test.cpp:1:10: fatal error: 'stan/mcmc/adaptive_hmc.hpp' file not found
#include <stan/mcmc/adaptive_hmc.hpp>
         ^
--
src/test/file_includes/stan_mcmc_nuts_test.cpp:1:10: fatal error: 'stan/mcmc/nuts.hpp' file not found
#include <stan/mcmc/nuts.hpp>
         ^
--
src/test/file_includes/stan_mcmc_nuts_diag_test.cpp:1:10: fatal error: 'stan/mcmc/nuts_diag.hpp' file not found
#include <stan/mcmc/nuts_diag.hpp>

But I did some minor patching

goodrich@CYBERPOWERPC:/opt/stan$ git diff | cat
diff --git a/src/stan/math/matrix.hpp b/src/stan/math/matrix.hpp
index aee9cc7..88d23cb 100755
--- a/src/stan/math/matrix.hpp
+++ b/src/stan/math/matrix.hpp
@@ -17,7 +17,7 @@
 namespace stan {
   
   namespace math {
-    typedef Eigen::Matrix<double,Eigen::Dynamic,Eigen::Dynamic>::size_type size_type;
+    typedef size_t size_type;
     // from input type F to output type T
 
     // scalar, F != T  (base template)
diff --git a/src/stan/mcmc.hpp b/src/stan/mcmc.hpp
index dc994ad..8dea03b 100644
--- a/src/stan/mcmc.hpp
+++ b/src/stan/mcmc.hpp
@@ -1,12 +1,12 @@
 #ifndef STAN_MCMC_HPP
 #define STAN_MCMC_HPP
 
-#include "stan/mcmc/adaptive_cdhmc.hpp"
-#include "stan/mcmc/adaptive_hmc.hpp"
-#include "stan/mcmc/adaptive_sampler.hpp"
-#include "stan/mcmc/dualaverage.hpp"
-#include "stan/mcmc/nuts.hpp"
-#include "stan/mcmc/sampler.hpp"
+#include "stan/mcmc/base_adapter.hpp"
+#include "stan/mcmc/base_mcmc.hpp"
+#include "stan/mcmc/chains.hpp"
+#include "stan/mcmc/covar_adapter.hpp"
+#include "stan/mcmc/sample.hpp"
+#include "stan/mcmc/stepsize_adapter.hpp"
 #include "stan/mcmc/util.hpp"
-
+#include "stan/mcmc/var_adapter.hpp"
 #endif

Ben

[Michael Betancourt]

Aaaaaaaaaaaaaaaaand we're passing the model tests. I need to add more unit tests and tweak the expected values in command_test, but otherwise we're pretty much there -- we can chat about it tomorrow.

On Apr 3, 2013, at 12:25 PM, Daniel Lee wrote:

[Bob Carpenter]

Great.

Are you changing the way the commands work? If
so, we're going to need to rewrite the doc.

And given that other interfaces are changing, the
bulk of the RStan command will also need to be
changed to conform to the new interfaces.

So it sounds like lots to talk about.

- Bob

[Michael Betancourt]

Nope, the interface is the same.

The next big step will be to redo the interface to make specifying different samplers more easily (as well as cleaning up the sampler output mechanism to allow for diagnostic outputs).

[Bob Carpenter]

On 4/8/13 8:35 PM, Michael Betancourt wrote:
> Nope, the interface is the same.

I was talking about the MCMC/sampler interface, not
the command interface from the outside.

If the interface is the same, you shouldn't have had
to touch command.hpp and RStan (and PyStan and ...)
should still work as is.

- Bob

[Michael Betancourt]

Ah, okay. There were small changes to make more sophisticated sampling approaches (multi-stage warmup, etc) readily implemented, but changes none-the-less.

[Bob Carpenter]

OK, so the merge will probably break RStan, so we'll have
to let Jiqiang know what's changed so he doesn't have
to hunt for it. (And Rob if he's still working on PyStan.)

- Bob

[Michael Betancourt]

I added new unit tests for all of the file includes and mutators in the new hmc_refactor code, as well as a few tests for some of the individual algorithmic elements (namely momentum sampling). Everything (including the existing unit tests as well as the model and command tests) passes, so once RStan has been updated we're ready to merge.

I'll be out of town and without my computer until Tuesday, but if there are any pressing issues I should be checking email occasionally.

[Bob Carpenter]

Great -- I think we're ready to go with 1.3 as soon
as all the tests pass. More on that in the next reply.

- Bob