[Solved] difficulties installing v2.5

[Sean O'Riordain]

Good day!

FYI, I finally managed to get v2.5 installed with some difficulty.  I'm running Ubuntu 14.04LTS on my laptop which has 6Gb RAM and *no* swap - which was the default when I installed this version of Ubuntu - I didn't notice until after I finished installing and it's a bit late now unfortunately!

Using  g++ (Ubuntu 4.8.2-19ubuntu1) 4.8.2, I ran into difficulties with memory, and I succeeded in the end by running with

Sys.setenv(MAKEFLAGS= "-j1")

and having "nothing" running - okay still under X :-)  except a pair of shell windows - one running R and the install and the other running 'top'... so I tested with 8schools and it seems fine now...

In previous attempts g++ just couldn't allocate the memory to run !!!

--
Kind regards,

Sean O'Riordain

[Ben Goodrich]

For g++-4.9 at least, you can make it use a lot less memory by adding -flto -ffat-lto-objects to the CFLAGS under ~/.R/Makevars. Although at the moment, the install.R script ignores your ~/.R/Makevars file anyway, but it would help when you compile models. I'm not sure how much of a difference it makes with g++-4.8.

Ben

[Bob Carpenter]

The C++ compiler's a memory hog. It was a pretty
big effort just getting the requirement down to 8 GB.
I didn't even know Stan could be built in 6GB RAM.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Sean O'Riordain]

I'm not sure how you might put this on the "RStan Getting Started" page - but not everybody has as much as 6gb so that might be a common enough cause of install failure...

As a nice to have feature... it'd be nice to have a faster compile of a model - currently on this not too slow Dell XPS-laptop it takes about 28 seconds to compile a model... which is a bit tedious when experimenting with models and importantly it is not enough time to make a coffee! :-)

seor...@tcd.ie

[Bob Carpenter]

Thanks for the feedback --- we really appreciate it.
More below.

On Oct 23, 2014, at 12:09 PM, Sean O'Riordain <seor...@tcd.ie> wrote:

> I'm not sure how you might put this on the "RStan Getting Started" page - but not everybody has as much as 6gb so that might be a common enough cause of install failure...

I'm not sure how it varies by platform, either, but you're right,
we should have memory requirements on the install page.

It's hard to remember that not everyone has 8 GB. I've been running on
computers with at least 8GB for over a decade! One book I love had some
advice I hate: give developers the slowest computers rather than the
fastest ones.

I would strongly recommend that anyone who can afford it upgrade their
memory to at least 8GB if not 16GB.

> As a nice to have feature... it'd be nice to have a faster compile of a model - currently on this not too slow Dell XPS-laptop it takes about 28 seconds to compile a model... which is a bit tedious when experimenting with models and importantly it is not enough time to make a coffee! :-)

I take it getting a faster computer and coffee maker is out
of the question.

How are you compiling? On my three-year old Macbook Pro (quad-core 2.3GHz i7 CPU,
16GB 1600MHz DDR3 memory), compile times at -O3 are closer to 10 seconds with clang++.
I'm guessing you must be using g++ or some other compiler than clang++.

It is possible to compile models with less optimization. They will
take an order of magnitude (or more) more time to run at -O0 than at -O3,
but this may be a win for a model that takes less than a second to fit.
And they'll compile much faster at -O0.

We're unlikely to drastically speed up compile time. The
rock and hard place we're caught between are too many instantiations
of our templated function to sensibly precompile (e.g.., 10s of thousands
just for Studnent-t) and the slowness of compiling templated functions.
Not to mention lack of good C++ tools on Windows.

- Bob

[Sean O'Riordain]

Hi Bob,

Well 30 seconds is not really long enough to make a *good* coffee... :-)

While the "desktop" machines I use have 16-128 Gb of memory, this three year old Dell XPS laptop - "high performance" for its' day - all we could afford under the budget... well the CPU is...
Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz (fam: 06, model: 2a, stepping: 07)

It came with 6Gb memory and it can take at most 8Gb.

Running on Ubuntu Linux, I've deliberately run things at the defaults... I've not installed clang... so g++ it is...

Similarly I haven't changed the defaults for optimization in Stan... I might look into trying that...

I seem to recollect that it that Bill Gates insisted that his developers used modest machines... but that was about 1996 ? maybe?

As an aside, I noticed that the log-likelihood for my model has sufficient statistics that can be pre-calculated in the transformed data block which *dramatically* speeds up the time per iteration... since looping is no longer required :-)

Sean

[Ben Goodrich]

Installing and using clang++ is the easiest way (as long as we are not talking about Windows) to reduce memory and wall time during compilation.

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Kind regards,
>
> Sean O'Riordain
> seor...@tcd.ie
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Sean O'Riordain]

Hi Ben,

Finally figured out how to do this by modifying .R/Makevars - as per Bob's email a short while ago.

FYI - for my simple model which only takes about a second to run, the total compile time and runtime for clang++ is 25 seconds, as against 32 seconds for g++ on this laptop running ubuntu 14.04 LTS.

cheers,

Sean

To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Bob Carpenter]

Thanks for the report. We need to look into Stan 2.5
compile times.

I'm also finding compile times have really gone up in
2.5 (or at least it feels like it --- as Jiqiang says, it
may just be my imagination). I wonder if we're
doing something like bringing in higher-order autodiff or
have somehow made other parts of the compilation more
complex.

The problem would be on the Stan C++ side, not on the R side.
I don't think R does anything here to get in the way.

- Bob