CRAN, MSVC, long long, C++11, size_t, and other headaches

[Bob Carpenter]

Can we just drop MSVC support now that we have g++
working? Then we could presumably get rid of this
file.

Does MSVC use different sizes than g++?

Which compiler and bit size are we using on Jenkins?

What's required for CRAN?

If I just leave everything alone with "long long", what
are the consequences?

Does someone else understand Boost fixed-size types or
the built-in fixed size macros well enough to help me out?

What I need is the following constructors for agrad::var:

var()
explicit var(vari*);
var(double);

What I want is to have every arithmetic type use the
var(double) constructor, but there are conflicts with
unsigned long and vari*, despite var(vari*) being explicit.

Yes, I can get enable_if working on my Mac to pick out
arithmetic types, but I can't get it to work on Windows, so
tests fail on Jenkins.

Previously, we just used all of the following
constructors:

var()
explicit var(vari* vi)
var(float), var(double), var(long double)
var(bool), var(short), var(int), var(long), *** var(long long) ***
var(unsigned short), var(unsigned int), var(unsigned long), *** var(unsigned long long) ***

The built-in types don't run into overload conflicts with each
other, but as soon as you get into the typedefs like size_t, uint64_t,
etc., the shit hits the fan when trying to overload.

The problem is that "long long" missed the boat for C++, so is only
in C++11.

Does using long long run afoul of CRAN policy.

The memory models in play are these:

ILP32 LP64 LLP64
char 8 8 8
short 16 16 16
int 32 32 32
long 32 64 32
long long 64 64 64
size_t 32 64 64
pointer 32 64 64

ILP32: 32-bit Windows
LP64: 64-bit linux & mac
LLP64: 64-bit windows

I can't for the life of me figure out how to get the right set of
constructors to cover all inputs and not run into conflicts without
using long long. And it looks like we can't get 64-bit integers
on ILP32 wihtout long long.

We could just punt altogether on long support in autodiff.
I think that'd require some serious test rewriting, and would be
less than ideal for third parties, but I think it may come to that.
We don't need it for running Stan models.

- Bob

> On Mar 1, 2015, at 3:19 AM, Daniel Lee <notifi...@github.com> wrote:
>
> just poking around in our repo, I see msinttypes.h sitting in
> src/msinttypes.h. It's included in stack_alloc.hpp. Not sure if it helps.
> Looking at it makes me dizzy.
>
> On Sat, Feb 28, 2015 at 11:13 AM, Bob Carpenter <notifi...@github.com>
> wrote:
>
> > We tried and failed last time to fix this, which is why
> > there were long long left in the code.
> >
> > I'm pretty sure the size situation is this:
> >
> > /* uint ulong size_t AMBIGUOUS
> > * Windows 32 32 32 32 ulong / size_t
> > * Windows 64 32 32 64
> > * Linux/Mac 32 32 64 32
> > * Linux/Mac 64 32 64 64 ulong / size_t
> > */
> >
> > What I need is a set of constructors for var() that can handle
> > any size int and any size unsigned int across platform.
> >
> > I tried the Boost built-in fixed size types and the C built-in
> > fixed size types, but I can't get anything to work without causing
> > ambiguities in constructor resolution on either Windows or the Mac.
> > I tried Boost's size ifdefs, and couldn't get that to work.
> >
> > If I try to get away with just
> >
> > var(double)
> >
> > because whatever the arg is it gets cast to double, it creates
> > ambiguities with
> >
> > explicit var(vari*)
> >
> > even though it's explicit when trying to do:
> >
> > var x(3L);
> >
> > If I just use { uint, size_t }, I miss 64-bit longs on 32-bit Linux/Mac.
> >
> > If I use just { uint, ulong }, I miss 64-bit longs on Windows.
> >
> > C++'s messed up sizing just makes me want to weep.
> >
> > - Bob
> >
> >
> > > On Mar 1, 2015, at 2:49 AM, Daniel Lee <notifi...@github.com> wrote:
> > >
> > > Yes, you can. See src/test/unit/agrad/rev/matrix/LDLT_alloc_test.cpp:
> > > #ifndef _WIN32
> > >
> > >
> > >
> > > On Sat, Feb 28, 2015 at 10:42 AM, Bob Carpenter <
> > notifi...@github.com>
> > > wrote:
> > >
> > > > Thanks. It may come down to that, but right now, the
> > > > current failure has me completely stumped.
> > > >
> > > > Unfortunately, the enable-if solution that works just
> > > > fine on the Mac fails utterly on Windows. So I reverted
> > > > to the list of types, which excludes size_t. But that
> > > > is somehow failing on fvar<fvar<var>> determinant calculations.
> > > >
> > > > I have no idea how long long is getting implicated.
> > > >
> > > > I don't actually think there are as many instances of
> > > > the problem as I thought (trying to construct a var with
> > > > a size_t on Windows, where size_t is not equal to unsigned
> > > > long, but rather equal to the illegal until C++11 unsigned
> > > > long long).
> > > >
> > > > - Bob
> > > >
> > > > > On Mar 1, 2015, at 2:27 AM, Daniel Lee <notifi...@github.com>
> > wrote:
> > > > >
> > > > > Bob, how can I help? Do you want me to set up an account on our
> > Windows
> > > > test box so you can run on it directly?
> > > > >
> > > > > —
> > > > > Reply to this email directly or view it on GitHub.
> > > > >
> > > >
> > > > —
> > > > Reply to this email directly or view it on GitHub
> > > > <https://github.com/stan-dev/stan/pull/1318#issuecomment-76530976>.
> >
> > > >
> > > —
> > > Reply to this email directly or view it on GitHub.
> > >
> >
> > —
> > Reply to this email directly or view it on GitHub
> > <https://github.com/stan-dev/stan/pull/1318#issuecomment-76532320>.
> >
> —
> Reply to this email directly or view it on GitHub.
>

[Jiqiang Guo]

Only stanc needs to be compiled on CRAN.  Can we just make stanc CRAN compatible?

Jiqiang

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bob Carpenter]

That's good news. Does that mean I can leave the "long long" in?

- Bob

[Bob Carpenter]

That is, can I leave "long long" in the autodiff library
if it's not in stanc?

- Bob

[Ben Goodrich]

On Saturday, February 28, 2015 at 12:10:17 PM UTC-5, Jiqiang Guo wrote:

Only stanc needs to be compiled on CRAN.  Can we just make stanc CRAN compatible?

Not quite. The vignette compiles models and that is what triggered these warnings.

Ben
 

[Ben Goodrich]

On Saturday, February 28, 2015 at 11:49:31 AM UTC-5, Bob Carpenter wrote:

Can we just drop MSVC support now that we have g++
working?  Then we could presumably get rid of this
file.

Someone on stan-users gor MSVC to work with a more recent Boost, so I think dropping MSVC support would be a last resort.
 

Does MSVC use different sizes than g++?

Probably not for MSVC >= 2012, which is what works with Boost
 

Which compiler and bit size are we using on Jenkins?

Not MSVC

What's required for CRAN?

Not MSVC; see

http://cran.r-project.org/web/checks/check_flavors.html
 

If I just leave everything alone with "long long", what
are the consequences?

Would have to compile rstan in C++11 mode and thus not have a Snow Leopard binary.
 

Does someone else understand Boost fixed-size types or
the built-in fixed size macros well enough to help me out?

No but maybe

Does using long long run afoul of CRAN policy.

Without compiling in C++11 mode, yes.
 

We could just punt altogether on long support in autodiff.
I think that'd require some serious test rewriting, and would be
less than ideal for third parties, but I think it may come to that.
We don't need it for running Stan models.

I could patch the files that use long long for the StanHeaders package. However, that would slightly undermine the API if someone wanted to use StanHeaders in C++ mode. At this time, I think C++11 for rstan is not quite the right choice because we haven't done enough testing with how all the other code gets compiled.

Ben

[Ben Goodrich]

On Saturday, February 28, 2015 at 11:49:31 AM UTC-5, Bob Carpenter wrote:

Previously, we just used all of the following
constructors:

  var()
  explicit var(vari* vi)
  var(float), var(double), var(long double)
  var(bool), var(short), var(int), var(long), *** var(long long) ***
  var(unsigned short), var(unsigned int), var(unsigned long), *** var(unsigned long long) ***

The built-in types don't run into overload conflicts with each
other, but as soon as you get into the typedefs like size_t, uint64_t,
etc., the shit hits the fan when trying to overload.

The problem is that "long long" missed the boat for C++, so is only
in C++11.  

Can you put the long long constructors inside an ifdef that tests for C++11?

Ben

[Jiqiang Guo]

Can we replace the vignette with a plain latex file?

Jiqiang

--

[Ben Goodrich]

On Saturday, February 28, 2015 at 1:39:45 PM UTC-5, Jiqiang Guo wrote:

Can we replace the vignette with a plain latex file?

I think we need a real fix, even if it is a hackish fix. The bug / warning is technically from StanHeaders, so Joshua or anyone else who wants to build something that links to StanHeaders and uses the var.hpp will face this issue.

Ben

[Jiqiang Guo]

I think in general it is not necessary that Stan needs to be CRAN compatible especially if it is hard.  Joshua or anyone else might only want to use C++11 mode....

But for this particular long long issue, I am afraid it is from compiling a file in rstan that uses matrix.hpp, which unnecessarily includes var.hpp (see below). 

Jiqiang  

ccache g++ -I/usr/share/R/include -DNDEBUG -I"../inst/include"  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -ftemplate-depth-256  -Wno-unused-function -Wno-uninitialized  -I"/home/jq/R/x86_64-pc-linux-gnu-library/3.1/Rcpp/include" -I"/usr/lib/R/site-library/RcppEigen/include" -I"/home/jq/R/x86_64-pc-linux-gnu-library/3.1/BH/include" -I"/home/jq/R/x86_64-pc-linux-gnu-library/3.1/StanHeaders/include"   -fpic  -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2           -pedantic  -c chains.cpp -o chains.o

In file included from /home/jq/R/x86_64-pc-linux-gnu-library/3.1/StanHeaders/include/stan/meta/traits.hpp:11:0,

                 from /home/jq/R/x86_64-pc-linux-gnu-library/3.1/StanHeaders/include/stan/math/matrix/append_row.hpp:5,

                 from /home/jq/R/x86_64-pc-linux-gnu-library/3.1/StanHeaders/include/stan/math/matrix.hpp:7,

                 from chains.cpp:2:

/home/jq/R/x86_64-pc-linux-gnu-library/3.1/StanHeaders/include/stan/agrad/rev/var.hpp:160:25: warning: ISO C++ 1998 does not support ‘long long’ [-Wlong-long]

       var(unsigned long long n) :

                         ^

/home/jq/R/x86_64-pc-linux-gnu-library/3.1/StanHeaders/include/stan/agrad/rev/var.hpp:170:16: warning: ISO C++ 1998 does not support ‘long long’ [-Wlong-long]

       var(long long n) :

--

[Bob Carpenter]

I'm going to continue to try to fix this after a partial
night's sleep. Debugging is such an obsessive business.

- Bob