How to run RStan unit tests?

[Daniel Lee]

I want to add some more tests to verify that refactoring doesnt completely tank RStan.

How can I fire off the unit tests in RStan? It looks like there's been some activity lately, so hopefully someone can guide me.

[Jiqiang Guo]

First, run 'make build' to make sure it can be built.

Second, run 'make check'

Third, there is an r script at folder call test.

All the above can be seen from the script on Jenkins

--
Jiqiang

On Mar 5, 2014 2:49 AM, "Daniel Lee" <bea...@alum.mit.edu> wrote:

I want to add some more tests to verify that refactoring doesnt completely tank RStan.

How can I fire off the unit tests in RStan? It looks like there's been some activity lately, so hopefully someone can guide me.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Daniel Lee]

Thanks! Just in case anyone wants to follow along, you can type this from the rstan directory:

git submodule init
git submodule update
cd stan
git fetch
git checkout origin/develop
cd ..
git submodule status
cd rstan
make clean
make install_pre_rpkg
make check && (cd tests & R -q -f runRunitTests.R --args ../rstan.Rcheck)

I can't actually run make check on my machine. I'm having a problem building the vignette. Hopefully this is a quick fix. So far, I've figured out that the error message reported by R CMD build actually hides the true error message in the vignette.

When I type:

> R CMD Sweave rstan/vignettes/rstan_vignette.Rnw

I get this error:

...
Warning in file(fname, "rt") :
  cannot open file 'schools.stan': No such file or directory
Error in file(fname, "rt") : cannot open the connection

When I run from the rstan/vignettes directory on rstan_vignette.Rnw, it works fine.

Is there an easy way to specify the working directory and possibly hide that output?

Daniel

[Jiqiang Guo]

Not sure what you are asking.  What output do you want to hide and why?  What true error message is hidden?

I think if you go to build that vignette individually after building rstan without vignette option, you have to go to that folder for now.  But the vignette is the first example that really builds a model.  So ultimately, you have to be able to build the vignette.

Jiqiang

[Daniel Lee]

On Thu, Mar 6, 2014 at 12:00 PM, Jiqiang Guo <guo...@gmail.com> wrote:

Not sure what you are asking.

Ok. From the rstan directory, this fails for me:

> R CMD Sweave rstan/vignettes/rstan_vignette.Rnw

I think once that works, I can get RStan to build end to end. That's what I'm trying to fix now.

Does that work on your setup? I'm running of my Mac, R 3.0.2.

If I type:

> cd rstan/vignettes && R CMD Sweave rstan_vignette.Rnw

it works.

 

What output do you want to hide and why?

You can ignore that. I was thinking about using a `setwd()` inside and not show it, but hopefully we don't need to.

 

What true error message is hidden?

Here's what happens when I type make build:
> make build
R --vanilla CMD build rstan --md5   # --no-build-vignettes --no-manual
* checking for file ‘rstan/DESCRIPTION’ ... OK
* preparing ‘rstan’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to process help pages
* saving partial Rd database
* creating vignettes ... ERROR
Loading required package: Rcpp
Loading required package: inline

Attaching package: ‘inline’

The following object is masked from ‘package:Rcpp’:

    registerPlugin

rstan (Version 2.2.0, packaged: , GitRev: 01df8de9d9ae)

Error: processing vignette ‘rstan_vignette.Rnw’ failed with diagnostics:
‘x’ must be an atomic vector
Execution halted
make: *** [build] Error 1

If you notice, the error is non-sensical: "'x' must be an atomic vector."

When I run R CMD sweave on the file, the real error is:
> R CMD Sweave rstan/vignettes/rstan_vignette.Rnw
Writing to file rstan_vignette.tex
Processing code chunks with options ...
 1 : keep.source term verbatim (rstan_vignette.Rnw:76)
 2 : echo keep.source term verbatim (rstan_vignette.Rnw:293)
 3 : echo keep.source term verbatim (rstan_vignette.Rnw:304)
 4 : echo keep.source term hide (label = callstan, rstan_vignette.Rnw:319)
Loading required package: Rcpp
Loading required package: inline

Attaching package: ‘inline’

The following object is masked from ‘package:Rcpp’:

    registerPlugin

rstan (Version 2.2.0, packaged: 2014-02-14 04:29:17 UTC, GitRev: 52d7b230aaa0)

Warning in file(fname, "rt") :
  cannot open file 'schools.stan': No such file or directory
Error in file(fname, "rt") : cannot open the connection

Error:  chunk 4 (label = callstan)
Error in get_model_strcode(file, model_code) :
  cannot open model file "schools.stan"
Error in rle(filenames) : 'x' must be an atomic vector
Calls: <Anonymous> ... do.call -> <Anonymous> -> <Anonymous> -> RweaveTryStop
Execution halted

[Jiqiang Guo]

On Thu, Mar 6, 2014 at 12:07 PM, Daniel Lee <bea...@alum.mit.edu> wrote:

On Thu, Mar 6, 2014 at 12:00 PM, Jiqiang Guo <guo...@gmail.com> wrote:

Not sure what you are asking.

Ok. From the rstan directory, this fails for me:

> R CMD Sweave rstan/vignettes/rstan_vignette.Rnw

I think once that works, I can get RStan to build end to end. That's what I'm trying to fix now.

Does that work on your setup? I'm running of my Mac, R 3.0.2.

I would go to folder rstan/vignettes to do that. 

If I type:

> cd rstan/vignettes && R CMD Sweave rstan_vignette.Rnw

it works.

It works based on the rstan that is installed on your system, not the one you are trying to build.  

 

What output do you want to hide and why?

You can ignore that. I was thinking about using a `setwd()` inside and not show it, but hopefully we don't need to.

 

What true error message is hidden?

Here's what happens when I type make build:
> make build
R --vanilla CMD build rstan --md5   # --no-build-vignettes --no-manual

You should remove above # so that vignettes would not be built.  Then after you install the rstan you want to test, you can try to build the vignette or run other model to see what is the problem.  It is a painful process, but I do not see a good way to avoid that.  One thing you should do if you have not is to use ccache. 

[Daniel Lee]

On Thu, Mar 6, 2014 at 12:13 PM, Jiqiang Guo <guo...@gmail.com> wrote:

On Thu, Mar 6, 2014 at 12:07 PM, Daniel Lee <bea...@alum.mit.edu> wrote:

On Thu, Mar 6, 2014 at 12:00 PM, Jiqiang Guo <guo...@gmail.com> wrote:

Not sure what you are asking.

Ok. From the rstan directory, this fails for me:

> R CMD Sweave rstan/vignettes/rstan_vignette.Rnw

I think once that works, I can get RStan to build end to end. That's what I'm trying to fix now.

Does that work on your setup? I'm running of my Mac, R 3.0.2.

I would go to folder rstan/vignettes to do that.

How does Jenkins work then? It runs make check, which essentially runs make build. Is NOTBUILD set to TRUE?

Right now, I can't get make check to work.

 

If I type:

> cd rstan/vignettes && R CMD Sweave rstan_vignette.Rnw

it works.

It works based on the rstan that is installed on your system, not the one you are trying to build.  

Yeah, that's fine for now. I have one built and installed. The problem I'm having is that make check fails based on the makefile inside rstan.

 

 

What output do you want to hide and why?

You can ignore that. I was thinking about using a `setwd()` inside and not show it, but hopefully we don't need to.

 

What true error message is hidden?

Here's what happens when I type make build:
> make build
R --vanilla CMD build rstan --md5   # --no-build-vignettes --no-manual

You should remove above # so that vignettes would not be built.  Then after you install the rstan you want to test, you can try to build the vignette or run other model to see what is the problem.  It is a painful process, but I do not see a good way to avoid that.  One thing you should do if you have not is to use ccache. 

I see. So is there any reason why that comment isn't removed permanently?

[Jiqiang Guo]

I think Jenkins always uses the develop branch against Stan's develop branch, so that is why it does not fail for now.

It works based on the rstan that is installed on your system, not the one you are trying to build. 

Yeah, that's fine for now. I have one built and installed. The problem I'm having is that make check fails based on the makefile inside rstan.

I am not sure what is exactly failing now. If make build succeeds and make check fails, then it is not because of the vignette. But you also showed that make build failed.

You should remove above # so that vignettes would not be built. Then after you install the rstan you want to test, you can try to build the vignette or run other model to see what is the problem. It is a painful process, but I do not see a good way to avoid that. One thing you should do if you have not is to use ccache.

I see. So is there any reason why that comment isn't removed permanently?

Sometimes, I need it to be uncommented when building vignette fails so that I can try to build the vignette to see more error message.

Jiqiang

[Daniel Lee]

On Thu, Mar 6, 2014 at 12:38 PM, Jiqiang Guo <guo...@gmail.com> wrote:

I think Jenkins always uses the develop branch against Stan's develop branch, so that is why it does not fail for now.

It works based on the rstan that is installed on your system, not the one you are trying to build. 

Yeah, that's fine for now. I have one built and installed. The problem I'm having is that make check fails based on the makefile inside rstan.

I am not sure what is exactly failing now. If make build succeeds and make check fails, then it is not because of the vignette. But you also showed that make build failed.

Sorry -- to be clear:

- make build fails

- make check fails

- R --vanilla CMD build rstan --md5 fails
- R --vanilla CMD build rstan --md5 --no-build-vignettes works!!! (after appropriate setup of stansrc and stanlib, etc.)

So even after I build the package, I can't run: make build. That still fails after I build the package.

 

You should remove above # so that vignettes would not be built. Then after you install the rstan you want to test, you can try to build the vignette or run other model to see what is the problem. It is a painful process, but I do not see a good way to avoid that. One thing you should do if you have not is to use ccache.

I see. So is there any reason why that comment isn't removed permanently?

Sometimes, I need it to be uncommented when building vignette fails so that I can try to build the vignette to see more error message.

Jiqiang

--

[Jiqiang Guo]

If make build fails, there is no package built.  Then you should not even try make check since there is no package to check.

--
Jiqiang

[Daniel Lee]

Ok. Does `make build` based on rstan's develop branch work for you?

I'm guessing there's something going on with my setup if you can get it to work.


Daniel

[Daniel Lee]

Odd. I just combed through the Jenkins build of RStan. It doesn't look like the vignette is being built there.

[Jiqiang Guo]

I can still build by make build.  Also I triggered travis-ci though it is still working on it.  https://travis-ci.org/stan-dev/rstan/jobs/20229169

You need a finer comb.  

http://d1m1s1b1.stat.columbia.edu:8080/job/rstan%20build/ws/rstan/rstan.Rcheck/rstan/doc/rstan_vignette.pdf

Jiqiang

[Daniel Lee]

On Thu, Mar 6, 2014 at 2:05 PM, Jiqiang Guo <guo...@gmail.com> wrote:

I can still build by make build.  Also I triggered travis-ci though it is still working on it.  https://travis-ci.org/stan-dev/rstan/jobs/20229169

Ok, maybe it's something wrong with my setup. I don't know how to fix it, though.

 

You need a finer comb.  

http://d1m1s1b1.stat.columbia.edu:8080/job/rstan%20build/ws/rstan/rstan.Rcheck/rstan/doc/rstan_vignette.pdf

Thanks! I didn't realize that it was sitting in rstan.Rcheck. That definitely wasn't the place I was looking for it.

[Jiqiang Guo]

Also 

$ tar ztf rstan_2.2.0.tar.gz|grep vignette.pdf

rstan/inst/doc/rstan_vignette.pdf

The only thing I can think of are what is the branch of stan and what is the branch of rstan. 

-

Jiqiang 

[Jiqiang Guo]

By the way, I had problems with the make you newly installed and could not figured out why if you are working on Windows.

--
Jiqiang