errors during model compilation

[atzap]

Hi all

I'm trying to fit my first STAN model and have hit a rather indecipherable wall.  The model fails to run and throws up the following error (more attached):

TRANSLATING MODEL 'anon_model' FROM Stan CODE TO C++ CODE NOW.

COMPILING THE C++ CODE FOR MODEL 'anon_model' NOW.

cygwin warning:

  MS-DOS style path detected: C:/PROGRA~1/R/R-215~1.1/etc/i386/Makeconf

  Preferred POSIX equivalent is: /cygdrive/c/PROGRA~1/R/R-215~1.1/etc/i386/Makeconf

  CYGWIN environment variable option "nodosfilewarning" turns off this warning.

  Consult the user's guide for more details about POSIX paths:

    http://cygwin.com/cygwin-ug-net/using.html#using-pathnames

file23c28003c5a.cpp: In member function 'void anon_model_namespace::anon_model::set_param_ranges()':

file23c28003c5a.cpp:254:40: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

file23c28003c5a.cpp:255:62: error: 'nil' was not declared in this scope

file23c28003c5a.cpp:255:62: note: suggested alternative:

C:/.../R/win-library/2.15/rstan/include//stanlib/boost_1.51.0/boost/fusion/container/list/cons.hpp:37:12: note:   'boost::fusion::nil'

file23c28003c5a.cpp:258:40: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

file23c28003c5a.cpp:259:62: error: 'nil' was not declared in this scope

file23c28003c5a.cpp:259:62: note: suggested alternative:

C:/.../R/win-library/2.15/rstan/include//stanlib/boost_1.51.0/boost/fusion/container/list/cons.hpp:37:12: note:   'boost::fusion::nil'

C:/.../R/win-library/2.15/rstan/include//stansrc/stan/agrad/agrad.hpp: At global scope:

C:.../R/win-library/2.15/rstan/include//stansrc/stan/agrad/agrad.hpp:2215:17: warning: 'void stan::agrad::free_memory()' defined but not used [-Wunused-function]

C:/.../R/win-library/2.15/rstan/include//stansrc/stan/math/special_functions.hpp:591:19: warning: 'double stan::math::log_sum_exp(const std::vector<double>&)' defined but not used [-Wunused-function]

make: *** [file23c28003c5a.o] Error 1

ERROR(s) during compilation: source code errors or compiler configuration errors!

Does anyone have any idea what this might be trying to tell me?  I am using R 2.15.1 with rstan_1.0.0, RcppEigen_0.3.1, inline_0.3.8, and Rcpp_0.9.13 (among other packages loaded).  The "schools" example runs fine.  Presumably, there are several problems with my model (hence the warnings), but I can't figure out what the error is trying to tell me...  I'm happy to email my model and data structure off-list.

Many thanks and sorry I chose such a complicated example to begin my work with STAN!

Andrew

[Daniel Lee]

Before we dig into this error, could you update RStan to 1.0.2?



Daniel

[Bob Carpenter]

There are a couple of problems. I've inserted
comments below.

Users -- if you want help debugging compilation, please
please include the model as a plain text file attachment
(not embedded in R or with line numbers, etc.).

You can send to st...@mc-stan.org if you want, but I have
to warn you that we're going to prioritize questions on
our open-access lists over proprietary communications.
One reason is that we're already drowning in complex user
models and if we take them on one-by-one off the list,
we have to deal with them one-by-one rather than hopefully
building up some group expertise so users can help each
other in the future.

More below inline.

On 10/1/12 5:55 PM, atzap wrote:
> Hi all
>
> I'm trying to fit my first STAN model and have hit a rather indecipherable wall.
> The model fails to run and throws up
> the following error (more attached):
>
>
> TRANSLATING MODEL 'anon_model' FROM Stan CODE TO C++ CODE NOW.
>
> COMPILING THE C++ CODE FOR MODEL 'anon_model' NOW.
>
> cygwin warning:
>
> MS-DOS style path detected: C:/PROGRA~1/R/R-215~1.1/etc/i386/Makeconf
>
> Preferred POSIX equivalent is: /cygdrive/c/PROGRA~1/R/R-215~1.1/etc/i386/Makeconf

If you use cygwin, you should use Unix-style paths, not
DOS-style paths. This isn't Stan-related per se and isn't fatal,
but it's why you get these warnings.

>
> file23c28003c5a.cpp: In member function 'void anon_model_namespace::anon_model::set_param_ranges()':
>
> file23c28003c5a.cpp:254:40: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

This is an issue we need to fix, but it's
not fatal. Actually, it won't come up in
1.0.2 because we trap and throw errors if users
try to define discrete parameters.

>
> file23c28003c5a.cpp:255:62: error: 'nil' was not declared in this scope

This is the killer. You use a variable named 'nil' without
defining it.

My guess without seeing your model is that you used an int
parameter -- that gets flagged in 1.0.2 with an error message.

Stan doesn't support int params.

> file23c28003c5a.cpp:255:62: note: suggested alternative:
>
> C:/.../R/win-library/2.15/rstan/include//stanlib/boost_1.51.0/boost/fusion/container/list/cons.hpp:37:12: note:
> 'boost::fusion::nil'
>
> file23c28003c5a.cpp:258:40: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
>
> file23c28003c5a.cpp:259:62: error: 'nil' was not declared in this scope
>
> file23c28003c5a.cpp:259:62: note: suggested alternative:
>
> C:/.../R/win-library/2.15/rstan/include//stanlib/boost_1.51.0/boost/fusion/container/list/cons.hpp:37:12: note:
> 'boost::fusion::nil'
>
> C:/.../R/win-library/2.15/rstan/include//stansrc/stan/agrad/agrad.hpp: At global scope:
>
> C:.../R/win-library/2.15/rstan/include//stansrc/stan/agrad/agrad.hpp:2215:17: warning: 'void
> stan::agrad::free_memory()' defined but not used [-Wunused-function]
>
> C:/.../R/win-library/2.15/rstan/include//stansrc/stan/math/special_functions.hpp:591:19: warning: 'double
> stan::math::log_sum_exp(const std::vector<double>&)' defined but not used [-Wunused-function]
>
> make: *** [file23c28003c5a.o] Error 1
>
>
> ERROR(s) during compilation: source code errors or compiler configuration errors!
>
>
>
> Does anyone have any idea what this might be trying to tell me? I am using R 2.15.1 with rstan_1.0.0, RcppEigen_0.3.1,
> inline_0.3.8, and Rcpp_0.9.13 (among other packages loaded). The "schools" example runs fine. Presumably, there are
> several problems with my model (hence the warnings), but I can't figure out what the error is trying to tell me... I'm
> happy to email my model and data structure off-list.
>
> Many thanks and sorry I chose such a complicated example to begin my work with STAN!

Our advice is to always start with simpler models and build up.

This isn't just a principle for beginners using Stan, but for
any level user with any kind of software.

The alternative is to start bigger, and then trim down
until you isolate the problem, but that's relatively painful
than going the other way around.

- Bob

[atzap]

Hi Daniel and Bob

Thanks for the prompt replies.  I've upgraded RStan to 1.0.2 but am now getting equally vague errors during compiling with the "schools" example from the STAN wiki.  I've attached the output from this (sorry about the line numbers).

Obviously, something with my setup then?...

Thanks,

Andrew

PS - Bob, I agree entirely that building the knowledge base on the list will save time in the future.  I can attach the model, but it looks like that might not be the issue here, at least for now...  For the record, I fiddled with simpler versions of this particular model, but was still hitting the same error.  The model runs fine in JAGS, so I thought, perhaps naively, that I could just "translate" it.

[Daniel Lee]

On Mon, Oct 1, 2012 at 6:50 PM, atzap <ata...@gmail.com> wrote:
> Hi Daniel and Bob
>
> Thanks for the prompt replies. I've upgraded RStan to 1.0.2 but am now
> getting equally vague errors during compiling with the "schools" example
> from the STAN wiki. I've attached the output from this (sorry about the
> line numbers).
>
> Obviously, something with my setup then?...

I was able to compile the schools example on the Stan wiki from the
command line in v1.0.2. Could you attach the model you are trying to
compile?

[atzap]

I was just copying and pasting from the box here: http://code.google.com/p/stan/wiki/RStanGettingStarted#Example_1:_Eight_Schools

So the model is:

schools_code <- '

  data {

    int<lower=0> J; // number of schools 

    real y[J]; // estimated treatment effects

    real<lower=0> sigma[J]; // s.e. of effect estimates 

  }

  parameters {

    real mu; 

    real<lower=0> tau;

    real eta[J];

  }

  transformed parameters {

    real theta[J];

    for (j in 1:J)

      theta[j] <- mu + tau * eta[j];

  }

  model {

    eta ~ normal(0, 1);

    y ~ normal(theta, sigma);

  }

- Andrew

[Jiqiang Guo]

See 
https://groups.google.com/forum/m/?fromgroups#!topic/stan-users/JnUNTBiYMgU
Sent from my iPad

[Daniel Lee]

Thanks, Jiqiang!

Would you mind updating the wiki with update directions for R?



Daniel

[atzap]

Thanks Jiqiang.  I saw that earlier today.  I'll try to remove and recompile then?...

  

I've provided the error dump in my attachment, though unsure how to pull out the g++ dump...

[atzap]

Hi all

Sorry this took a few hours to try.  Had a nasty fall cycling home this evening.  The good news is that I wanted to work with STAN when I finally made it in, which probably dismisses the potential for a concussion.  How's that for a selling point for your work?

Anyway, the "schools" example works now that I've removed and recompiled.  

As for my original model, it still isn't compiling, but I'm getting a much more informative error.  It seems that I've been trying to set an integer parameter, which I know STAN won't handle.  

Specifically, I have some missing values that I'm modelling as:

parameters{

    ...

    int<lower = 0>phyto_miss[Nmiss_phy];

}

model{

  ...

 for(i in 1:Nmiss_phy){

       phyto_miss[i] ~ poisson(phyto_miss_lambda[i]);

     }

}"

Is there any recommended work around in this case?  I saw the following: https://groups.google.com/forum/?fromgroups=#!searchin/stan-users/integer$20parameter/stan-users/MJf_SwNmGFo/AvVS1qPVGP0J, but couldn't readily see how it might be applied in my case.

Thanks again,

Andrew

[Ben Goodrich]

On Monday, October 1, 2012 10:46:05 PM UTC-4, atzap wrote:

As for my original model, it still isn't compiling, but I'm getting a much more informative error.  It seems that I've been trying to set an integer parameter, which I know STAN won't handle.  

Specifically, I have some missing values that I'm modelling as:

parameters{

    ...

    int<lower = 0>phyto_miss[Nmiss_phy];

}

model{

  ...

 for(i in 1:Nmiss_phy){

       phyto_miss[i] ~ poisson(phyto_miss_lambda[i]);

     }

}"

Is there any recommended work around in this case?  I saw the following: https://groups.google.com/forum/?fromgroups=#!searchin/stan-users/integer$20parameter/stan-users/MJf_SwNmGFo/AvVS1qPVGP0J, but couldn't readily see how it might be applied in my case.

It seems that my integer hack doesn't really work in practice. But poisson() can take a real (nonnegative) parameter for its expectation, so you could use that.

Ben

[Bob Carpenter]

On 10/1/12 10:46 PM, atzap wrote:
> Hi all
>

> Sorry this took a few hours to try. Had a nasty fall cycling home this evening. The good news is that I wanted to work
> with STAN when I finally made it in, which probably dismisses the potential for a concussion. How's that for a selling
> point for your work?

I hate to follow up your good news/bad news with more
of the same, but...

> Anyway, the "schools" example works now that I've removed and recompiled.

That's good news.

> As for my original model, it still isn't compiling, but I'm getting a much more informative error. It seems that I've
> been trying to set an integer parameter, which I know STAN won't handle.

That's bad news and what I deduced from your
first error report.

I'm afraid there's no workaround to integer parameters
other than to marginalize them out of the models.
And I doubt that'll be possible for Poisson missing
data "parameters".

We're still brainstorming on a clever way to add
discrete samplers, but we don't see how to make it
both easy to specify the models and efficient to
sample. We're interested in exactly this case,
because Poisson-like missing data is the only scenario
we've dreamed up where you have unbounded integer
parameters.

I'm guessing you'll need the missing data somewhere
else in the model specification. If not, we are
going to add random number generators soon in the
generated quantities block to generate simulations.

- Bob

> Specifically, I have some missing values that I'm modelling as:
>
> parameters{
>
> ...
>
> int<lower = 0>phyto_miss[Nmiss_phy];
>
> }
>
> model{
>
> ...
>
> for(i in 1:Nmiss_phy){
>
> phyto_miss[i] ~ poisson(phyto_miss_lambda[i]);
>
> }
>
> }"
>
>
> Is there any recommended work around in this case? I saw the f

> <https://groups.google.com/forum/?fromgroups=#!forum/stan-users>ollowing:

> https://groups.google.com/forum/?fromgroups=#!searchin/stan-users/integer$20parameter/stan-users/MJf_SwNmGFo/AvVS1qPVGP0J,
> but couldn't readily see how it might be applied in my case.

It may be possible to use some kind of continuous
approximation for some -- I haven't thought about it
enough to say.

- Bob

[Andrew Tanentzap]

Really??  I thought the expected value of a Poisson has to be discrete (http://en.wikipedia.org/wiki/Poisson_distribution)...

The STAN reference manual (v1.0.1) also says on pg 122 that the expectation "n" has to be an integer.  It threw up an error that suggested this when I tried switching my missing parameter to a "real".

Perhaps, I'm misunderstanding the point you're trying to make?

[Ben Goodrich]

On Tuesday, October 2, 2012 7:49:20 AM UTC-4, atzap wrote:

Really??  I thought the expected value of a Poisson has to be discrete (http://en.wikipedia.org/wiki/Poisson_distribution)...

The expected value (lambda) can be any positive number.
 

The STAN reference manual (v1.0.1) also says on pg 122 that the expectation "n" has to be an integer.  It threw up an error that suggested this when I tried switching my missing parameter to a "real".

But this is a problem with the current implementation of Stan. The poisson() function in Stan expects the "data" to be nonnegative integers, which is a problem when the "data" are missing and the missing value is a parameter.

There are lots of hits on Google about continuous generalizations of the Poisson distribution. This guy

http://cmscience.blogspot.com/2008/04/mt-6-poisson-and-gamma-distribution.html

has a simple one, which just replaces the factorial with a gamma function. The comments ask him to show that the continuous distribution integrates to something finite, which he doesn't do, but I'll presume that is always so, in which case you could take the log and do something like this in Stan

parameters {
  real<lower=0> phyto_miss[Nmiss_phy];
}
model {
  for(i in 1:Nmiss_phy) {
    lp__ <- lp__ + phyto_miss[i] * log(phyto_miss_lambda[i]) -
                   phyto_miss_lambda[i] - lgamma(phyto_miss[i] + 1.0);
  }
}

Or, as the link above points out, you could use a gamma distribution for the missing values.

Ben

[Bob Carpenter]

On 10/2/12 7:56 AM, Andrew Tanentzap wrote:
> Hi Bob

Just as a matter of form, this is an open list, so
posts shouldn't really be addressed to me personally!

> So just to clarify, even if I know the rate of my Poisson distribution, and simply want to sample from it randomly, I
> can't do this at present because it requires specifying the expectations as "integers" and "parameters"?... And I can't
> sample in any of the other "non-parameter" blocks...

Let me first try to clarify terminology:

poisson(y|lambda) = ...

y: non-negative integer outcome (aka the variate)
lambda: non-negative real value

For the Poisson, the value of parameter lambda just
happens to be the mean of the distribution (and
variance in this case). And also the expectation of
any random variable distributed as Poisson(lambda).

Following BUGS, we call any unknown variable a parameter.
And y is restricted to integers. But y is not an
expectation, so I wonder if we're talking
about the same thing.

Stan is fine sampling lambda as a parameter.

Stan cannot (yet) sample an unknown integer y. Not
in any of the blocks. We're going to add sampling
in the derivated quantities block, but that will only
work to draw random integers from a distribution
like Poisson(lambda). You won't be able to use them
as latent responsibility indicators in mixture models,
for instance, because there's no way to transfer information
from the derived quantities block back to the model.

- Bob

[Bob Carpenter]

On 10/2/12 7:49 AM, Andrew Tanentzap wrote:
> Really?? I thought the expected value of a Poisson has to be discrete
> (http://en.wikipedia.org/wiki/Poisson_distribution)...
>
> The STAN reference manual (v1.0.1) also says on pg 122 that the expectation "n" has to be an integer.

What's going on with Poisson? Did we botch something in
the manual? It looks OK to me. It's just that n is
not an expectation in any sense of the word I understand.

Let me try to clarify. Here's the density function:

Poisson(y|lambda) = (exp(-lambda) / lambda!) * lambda^y

y: the variate, an integer >= 0

lambda: the location parameter, a real > 0

If Y is an integer random variable and Y has
a Poisson(lambda) distribution, i.e., Y ~ Poisson(lambda),
then the expectation of Y is lambda.

Sometimes people conflate means (properties of
densities), expectations (properties of random variables),
and location parameters (parameters of distributions).

> It threw up an
> error that suggested this when I tried switching my missing parameter to a "real".

As it should. y has to be an integer as declared in
the manual.

What people were suggesting is that you could write
your own probability density and fudge the result
by treating y as real in that density. See the
citation in an earlier e-mail, for example. I don't
know enough about this strategy to tell you if or
how well it'd work.

> Perhaps, I'm misunderstanding the point you're trying to make?

If the above didn't help, maybe you should take some time
to read up on expectations in general and the Poisson
distribution in particular.

- Bbo

[Andrew Tanentzap]

Thanks this all makes sense.  I was simply getting mixed up with the meaning of "expectation".  

The suggested, custom PDF looks like a very nice workaround!