automating creation of .R/Makevars file from R

[Bob Carpenter]

I don't know what the conventions are in R. It seems like
different projects just kill the Makevars file, but that
seems like very bad form.

Andrew's pretty insistent that having users create a file
in .R is too difficult. Problem 1: Andrew didn't know what
"~/" was and the finder doesn't show ".R" because it starts
with a ".". I tried finding it using Spotlight, but that's
apparently also a no-go for files in folders starting with a ".".

(Apparently, Andrew never creates a file within
emacs, but rather takes the roundabout approach of creating
a file on the system using the Finder and then clicking it to open
in Aquamacs. I concur with Andrew that if this gave him problems
it's probably going to be an issue for others.)

Andrew's proposed approach looks safe, but it should at
the very least report if the file exists and that other steps should
be taken.

Is this something that only works for Mac users? Do others need
to modify the Makevars? What do other packages do for this?

Andrew's keen to solve this problem, so we should do something about
it rather than recommend that users find a friend to help them.
What would we write back to someone with Andrew's concern on the Mac
that they can't find that file in the Finder to open it?

- Bob

On 1/14/14, 11:29 PM, Andrew Gelman wrote:
> Here's what I wrote:
>
> From the R console:
>
>> if (!file.exists ("~/.R/Makevars")) cat ('CXX=g++ -arch x86_64 -ftemplate-depth-256 -stdlib=libstdc++\nCXXFLAGS="-mtune=native -O3 -Wall -pedantic -Wconversion"\n', file="~/.R/Makevars")
>
> And here's what Jiqiang replied:
>
> It is a good idea. The issue is that a user might have a file already there, so it is not bulletproof. The file already there might have the correct configuration or not. So we need to deal with that differently. My opinion is to leave it almost as what it is now. If a user does not know how to do it, it should not be hard to find someone who can help with it.
>
>
> On Jan 14, 2014, at 12:09 PM, Bob Carpenter wrote:
>
>> I'll do it.
>>
>> - Bob
>
>
>

[Daniel Lee]

bump.

I think these are the relevant questions:

On Wed, Jan 15, 2014 at 9:58 PM, Bob Carpenter <ca...@alias-i.com> wrote:

Is this something that only works for Mac users?  Do others need
to modify the Makevars?  What do other packages do for this?

What would we write back to someone with Andrew's concern on the Mac
that they can't find that file in the Finder to open it?

 

On 1/14/14, 11:29 PM, Andrew Gelman wrote:

Here's what I wrote:

 From the R console:

if (!file.exists ("~/.R/Makevars")) cat ('CXX=g++ -arch x86_64 -ftemplate-depth-256 -stdlib=libstdc++\nCXXFLAGS="-mtune=native  -O3 -Wall -pedantic -Wconversion"\n', file="~/.R/Makevars")

And here's what Jiqiang replied:

It is a good idea.  The issue is that a user might have a file already there, so it is not bulletproof.  The file already there might have the correct configuration or not.  So we need to deal with that differently.  My opinion is to leave it almost as what it is now.  If a user does not know how to do it, it should not be hard to find someone who can help with it.


On Jan 14, 2014, at 12:09 PM, Bob Carpenter wrote:

I'll do it.

- Bob

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Andrew Gelman]

Bob:

1.  I suspect this will work for windows and linux users too, assuming the file always has that name.  To put it another way, I'd guess that if these new instructions _don't_ work with someone's install, then the existing instructions (which specify to find this file in that certain place) won't work either.  So if there might be a problem, in any case we would need a sentence in the instructions saying something like:  If these instructions don't work, you'll have to search for Makevars, which for some reason might not be in the usual place ~/.R/Makevars

I think Jiqiang or Ben will have to answer whether this could ever occur.

My guess is that R has some direct way of accessing the location of its Makevars file (e.g., a function called getMakevars()) or something like that.  This would seem to me to be more robust than hard-coding "~/.R/Makevars".  But, again, the current instructions hard-code it already, so I can't see my new instructions causing problems that weren't already there.

2.  In answer to your second question, we can just send them this R code:

if (!file.exists ("~/.R/Makevars")) cat ('CXX=g++ -arch x86_64 -ftemplate-depth-256 -stdlib=libstdc++\nCXXFLAGS="-mtune=native  -O3 -Wall -pedantic -Wconversion"\n', file="~/.R/Makevars")

Of course, that just raises the question of why not put this R code on the website in the first place?  Or, maybe better still, 2 lines of R code, where the first line searches for the Makevars file and, if it exists, saves it with some name such as Makevars_old_current_date, and the second line is what I have above.  Then we add a note in the website saying that, if there was anything important in the old Makevars file, you can find it at ~/.R/Makevars_old_current_date

This will satisfy the needs of Andrew-syle users while also following a no-destroy policy and making everything accessible to the sophisticates.

All this of course will be superseded by our eventual one-click install…

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Daniel Lee]

Does Ben's response to "Rstan compilation optimization?" address everything?



On Thu, Jan 16, 2014 at 9:59 AM, Ben Goodrich <goodri...@gmail.com> wrote:

On Wednesday, January 15, 2014 4:20:22 PM UTC-5, Bob Carpenter wrote:

Following on from the message from Andrew and the help request
from a user, I'd like to try to automate as much as possible:

   * removing -g option from compilation

   * making sure parser and code generator compile with -O0

   * making sure lib compiles with -O3

   * making sure we don't have bogus architecture flags (e.g., -core2 for
     non core2 machines)

   * making sure models are compiled with -O3

I'm afraid if we don't, Stan installations are going to be slow and
thus hurt the user experience and tarnish our reputation.

Mostly reasons why we need binary installs. But in the meantime, I think we just need to have the installation directions say to do

Sys.setenv(R_MAKEVARS_USER = "foobar")
options(repos = c(getOption("repos"), rstan = "http://wiki.rstan-repo.googlecode.com/git/"))
install.packages('rstan', type = 'source')
Sys.unsetenv("R_MAKEVARS_USER")

so that it will read the non-existent configuration file foobar instead of whatever the hell the user has when installing. Then,

library(rstan)
set_cppo("fast") # or set_cppo("debug")

should achieve the appropriate behavior for Stan models.

It is already frowned upon for an R package to be messing with the Makevars file. We are probably breaking a few packages that work at -O2 but not at -O3. But just take a bodyguard with you to conferences and we should be all right.

Ben

[Bob Carpenter]

On 1/20/14, 12:52 AM, Daniel Lee wrote:
> Does Ben's response to "Rstan compilation optimization?" address everything?

I added Ben's instructions to the R Getting Started guide.

I have no idea. This is all an R issue and is tied up in
what their defaults are. I don't even know if they vary
by platform, or what.

I'm also worried that the install instructions are getting more
and more complex. I think it would make sense to have separate
Wikis for the install on each platform, with a checklist a user
can go through to make sure they get everything. Anyone willing
to take a shot at that?

- Bob

> All this of course will be superseded by our eventual one-click install�

>
>
> On Jan 20, 2014, at 12:35 AM, Daniel Lee wrote:
>
>> bump.
>>
>> I think these are the relevant questions:
>>

>> On Wed, Jan 15, 2014 at 9:58 PM, Bob Carpenter <ca...@alias-i.com <mailto:ca...@alias-i.com>> wrote:
>>
>> Is this something that only works for Mac users? Do others need
>> to modify the Makevars? What do other packages do for this?
>>
>> What would we write back to someone with Andrew's concern on the Mac
>> that they can't find that file in the Finder to open it?
>>
>>
>>
>>
>>
>>
>> On 1/14/14, 11:29 PM, Andrew Gelman wrote:
>>
>> Here's what I wrote:
>>
>> From the R console:
>>
>> if (!file.exists ("~/.R/Makevars")) cat ('CXX=g++ -arch x86_64 -ftemplate-depth-256

>> -stdlib=libstdc++\nCXXFLAGS="-__mtune=native -O3 -Wall -pedantic -Wconversion"\n', file="~/.R/Makevars")

>>
>>
>> And here's what Jiqiang replied:
>>
>> It is a good idea. The issue is that a user might have a file already there, so it is not bulletproof.
>> The file already there might have the correct configuration or not. So we need to deal with that
>> differently. My opinion is to leave it almost as what it is now. If a user does not know how to do it,
>> it should not be hard to find someone who can help with it.
>>
>>
>> On Jan 14, 2014, at 12:09 PM, Bob Carpenter wrote:
>>
>> I'll do it.
>>
>> - Bob
>>
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to

>> stan-dev+unsubscribe@__googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.
>> For more options, visit https://groups.google.com/__groups/opt_out <https://groups.google.com/groups/opt_out>.

>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to

>> stan-dev+u...@googlegroups.com <mailto:stan-dev+u...@googlegroups.com>.

>> For more options, visit https://groups.google.com/groups/opt_out.
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+u...@googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.

[Bob Carpenter]

What we want to do is find the users home directory.

Your

On 1/20/14, 12:47 AM, Andrew Gelman wrote:
> Bob:
>
> 1. I suspect this will work for windows and linux users too, assuming the file always has that name.

Unfortnateky, Windows is a whole different beast.

There is no "~/." on Windows --- those instructions are
for Mac OS X Mavericks only! Mac's Xcode set the max template
depth too low, so we've had to code around it.

> To put it another
> way, I'd guess that if these new instructions _don't_ work with someone's install, then the existing instructions (which
> specify to find this file in that certain place) won't work either. So if there might be a problem, in any case we
> would need a sentence in the instructions saying something like: If these instructions don't work, you'll have to
> search for Makevars, which for some reason might not be in the usual place ~/.R/Makevars

We could do that. As I said in a previous post, our instructions
are already so qualified that I'm reluctant to make them even
deeper without (a) breaking the instructions apart per platform,
and (b) creating a checklist.

> I think Jiqiang or Ben will have to answer whether this could ever occur.
> My guess is that R has some direct way of accessing the location of its Makevars file (e.g., a function called
> getMakevars()) or something like that.

No that I could find, but it has a way of accessing a home environment, which
I mentioned before, and the ".R" directory should be under the
user's home directory. That Makevars file might not even exist, so searching
for it's not going to help. The instructions are of the variety "if you do
have such a Makevars and it has the wrong thing in it, you need to fix it".

> This would seem to me to be more robust than hard-coding "~/.R/Makevars". But,
> again, the current instructions hard-code it already, so I can't see my new instructions causing problems that weren't
> already there.

I'm with you there.

> 2. In answer to your second question, we can just send them this R code:
>
> if (!file.exists ("~/.R/Makevars")) cat ('CXX=g++ -arch x86_64 -ftemplate-depth-256
> -stdlib=libstdc++\nCXXFLAGS="-mtune=native -O3 -Wall -pedantic -Wconversion"\n', file="~/.R/Makevars")
>
> Of course, that just raises the question of why not put this R code on the website in the first place? Or, maybe better
> still, 2 lines of R code, where the first line searches for the Makevars file and, if it exists, saves it with some name
> such as Makevars_old_current_date, and the second line is what I have above. Then we add a note in the website saying
> that, if there was anything important in the old Makevars file, you can find it at ~/.R/Makevars_old_current_date
>
> This will satisfy the needs of Andrew-syle users while also following a no-destroy policy and making everything
> accessible to the sophisticates.

As Ben pointed out, it may break other packages that won't compile
at -O3. But we can let the other packages sort that out --- Ben just
said we should bring bodyguards to conferences :-)

So, I did as you asked and added this to the instructions:

One way you can achieve this is to execute this from within R:

if (!file.exists("~/.R/Makevars")) {
cat('CXX=g++ -arch x86_64 -ftemplate-depth-256 -stdlib=libstdc++\n

CXXFLAGS="-mtune=native -O3 -Wall -pedantic -Wconversion"\n',

file="~/.R/Makevars");
} else {
file.show("~/.R/Makevars");
}

If the Makevars file already exists, its content will be displayed and you
can decide whether to run the cat part of the above command and destroy the
existing Makevars file.

What a huge pain!

> All this of course will be superseded by our eventual one-click install�

Feel free to go edit the Wiki instructions to your liking --- you should
have permission if you're logged into GitHub:

https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started

- Bob