Issues with CmdStan build setup

[Marcus Brubaker]

Hey everyone (well, Daniel mostly I guess),

So I'm finally getting up to speed with the new CmdStan structure and have hit two snags so far. 

First, it seems like the dependencies are broken with the new CmdStan setup.  In particular, if you change a source file in stan (proper) the unit tests in CmdStan don't rebuild.

Second,  stan and CmdStan build their object files in different places.  This means that when running unit tests in both it's necessary to, e.g., build the parser twice.  Not a major issue, but annoying all the same.

Anyway, just wanted to point these issues out.  I'd take a crack at fixing them, but my make-foo is weak.  Glad to see the refactoring is happening!

Cheers,

Marcus

[Daniel Lee]

On Fri, Apr 11, 2014 at 2:06 PM, Marcus Brubaker <marcus.a...@gmail.com> wrote:

Hey everyone (well, Daniel mostly I guess),

So I'm finally getting up to speed with the new CmdStan structure and have hit two snags so far.
 

First, it seems like the dependencies are broken with the new CmdStan setup.  In particular, if you change a source file in stan (proper) the unit tests in CmdStan don't rebuild.

Yeah -- I noticed that recently. Sorry I haven't gotten it fixed yet, but I'll try to get it up and working soon. I think it also happens in Stan proper. Damn make! But really, it's my fault. I must have done something and not checked hard enough.

 

Second,  stan and CmdStan build their object files in different places.  This means that when running unit tests in both it's necessary to, e.g., build the parser twice.  Not a major issue, but annoying all the same.

Good point. I figured if you were using CmdStan, you'll only build the parser from there. If you're working in Stan, you'll build the parser there. I don't think there's much to do about this one unless we find some way to copy over the one built in Stan.

 

Anyway, just wanted to point these issues out.  I'd take a crack at fixing them, but my make-foo is weak.  Glad to see the refactoring is happening!

If you find anything else, please let me know. Like I said, I only stumbled on the broken make recently (this week) and if I had known earlier, I would have spent some time earlier fixing it. There isn't a testing framework for make itself, is there?

 

Cheers,

Marcus

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marcus Brubaker]

On Fri, Apr 11, 2014 at 2:13 PM, Daniel Lee <bea...@alum.mit.edu> wrote:

On Fri, Apr 11, 2014 at 2:06 PM, Marcus Brubaker <marcus.a...@gmail.com> wrote:

Hey everyone (well, Daniel mostly I guess),

So I'm finally getting up to speed with the new CmdStan structure and have hit two snags so far.
 

First, it seems like the dependencies are broken with the new CmdStan setup.  In particular, if you change a source file in stan (proper) the unit tests in CmdStan don't rebuild.

Yeah -- I noticed that recently. Sorry I haven't gotten it fixed yet, but I'll try to get it up and working soon. I think it also happens in Stan proper. Damn make! But really, it's my fault. I must have done something and not checked hard enough.

No problem, just wanted to mention it in case no one had stumbled on it yet.

 

 

Second,  stan and CmdStan build their object files in different places.  This means that when running unit tests in both it's necessary to, e.g., build the parser twice.  Not a major issue, but annoying all the same.

Good point. I figured if you were using CmdStan, you'll only build the parser from there. If you're working in Stan, you'll build the parser there. I don't think there's much to do about this one unless we find some way to copy over the one built in Stan.

What if (for CmdStan) we just build all the object files in stan/bin/... instead of bin/stan/...?  Assuming Stan uses the same structure for it's object files that should cover it right?

 

 

Anyway, just wanted to point these issues out.  I'd take a crack at fixing them, but my make-foo is weak.  Glad to see the refactoring is happening!

If you find anything else, please let me know. Like I said, I only stumbled on the broken make recently (this week) and if I had known earlier, I would have spent some time earlier fixing it. There isn't a testing framework for make itself, is there?

Personally I've been wondering if we need to start looking at something like CMake.  My experience has been it automatically handles dependencies really well and it has a number of extra features which could be quite handy.  For instance, things like subprojects and integrated testing support.  Plus, added features like building against installed boost/Eigen libraries for broader distribution (particular if we start using more than just the header only parts of Boost), ability to ensure consistency of compilers and compiler flags, support for Visual Studio/MSVC, etc.

Given how structured our repository is, setting up a CMake based build probably wouldn't be a huge amount of work.  If other people think it's worth trying I might take a crack at it.

Cheers,

Marcus

[Bob Carpenter]

On Apr 11, 2014, at 11:14 PM, Marcus Brubaker <marcus.a...@gmail.com> wrote:

> ...

> Given how structured our repository is, setting up a CMake based build probably wouldn't be a huge amount of work. If other people think it's worth trying I might take a crack at it.

Other people have suggested CMake. I'm OK with whatever you
guys decide, especially if you're going to handle the conversion.
The current make is well beyond my comprehension as is, which makes
me uncomfortable, so if CMake's easier to read/modify and easy to get
on every platform, I'm all for it.

- Bob

[Michael Betancourt]

I’ve never had good experiences with CMake on either side, but
then make isn’t exactly milk and cookies.

[Ben Goodrich]

I've always thought that CMake was more appropriate for new projects but didn't say anything because I didn't want to commit to doing any of the work to convert from make. Really, it should be Daniel's decision, I think.

Ben

[Marcus Brubaker]

Ok, I was inspired today to take a whack at creating a CMake based build system for Stan.  A partial effort in this direction is now in the feature/cmake branch.  Currently it supports:

- Building libstan and libstanc

- (Always) Disabling optimization for libstanc and unit tests

- Unit tests, both generated and non-generated

- Using either included or system installed versions of Eigen, Boost and GTest

- CTest integration (although I've not played with this much)

Not yet implemented or to be fixed later:

- Building stanc (or any of CmdStan)

- Building models or any tests which require compiled models

- Building the manual or doxygen docs

- Currently there are only a few CMakeLists.txt, we should split it out more but that can come later

- I haven't set our custom compiler flags yet so things may not behave exactly the same way

- Probably a few other things...

This took me about two hours to put together and, while I've used CMake before, I'm far from a CMake expert which suggests that this might be easier to update and maintain going forward.  If anyone is interested in testing it out, you can do the following:

(from the top-level stan directory)

mkdir build ; cd build ; cmake -D CMAKE_CXX_COMPILER:file=/opt/local/bin/clang++ -D CMAKE_C_COMPILER:file=/opt/local/bin/clang -D CMAKE_BUILD_TYPE:string=Release .. ; make -j12 ; 

I'd ask that people take a quick look and let me know what you think.  I'm willing to finish the conversion if we want to go this way.  Personally I think it could be worth it but this is pretty big shift.

Cheers,

Marcus

[Ido Rosen]

Hi, just chiming in to say that I'm glad you took this upon yourself!
I'll be involved again in May, so if this is still ongoing then I can
help.

[Ben Goodrich]

I got past the first part and then it wouldn't link anything. See attached log. I'll dig into it more tomorrow.

Ben

[Marcus Brubaker]

I haven't tested on Linux and I think we need to explicitly link to pthreads there.  That looks to be the issue.  If you don't beat me to it, I will fix it tomorrow or Monday.

Cheers,
Marcus

[Marcus Brubaker]

Ok, I think I've fixed the pthreads issue.  I've also started a little on building Stan models, but haven't had too much time to work on it today so the tests which rely on compiled models still don't build yet.  I've also added some of the custom compile flags and split things out a bit better.

Cheers,

Marcus

[Ben Goodrich]

After a git pull, I got

CMake Error at src/test/CMakeLists.txt:13 (add_subdirectory):
  The source directory

    /opt/stan/src/test/test-models

  does not contain a CMakeLists.txt file.


-- Configuring incomplete, errors occurred!
See also "/opt/stan/build/CMakeFiles/CMakeOutput.log".
See also "/opt/stan/build/CMakeFiles/CMakeError.log".

but once I commented out that line, it ran until

Linking CXX executable unit_agrad_dot_product_test
1 error generated.
make[2]: *** [src/test/CMakeFiles/unit_common_command_init_test.dir/unit/common/command_init_test.cpp.o] Error 1
make[1]: *** [src/test/CMakeFiles/unit_common_command_init_test.dir/all] Error 2
[ 14%] Built target unit_agrad_dot_product_test
make: *** [all] Error 2

This is a lot more progress than I would have expected in 2 days.

One question: We have a functioning cluster at Columbia with thousands of cores so the optimal way to use it is to call make individually on every single micro target, rather than something like make test-all. That is possible with static makefiles, although it is a bit tedious to write a shell script to do it. Do you have an idea about how we could do something like that with cmake?

Ben

[Marcus Brubaker]

On Sun, Apr 13, 2014 at 4:36 PM, Ben Goodrich <goodri...@gmail.com> wrote:

After a git pull, I got

CMake Error at src/test/CMakeLists.txt:13 (add_subdirectory):
  The source directory

    /opt/stan/src/test/test-models

  does not contain a CMakeLists.txt file.


-- Configuring incomplete, errors occurred!
See also "/opt/stan/build/CMakeFiles/CMakeOutput.log".
See also "/opt/stan/build/CMakeFiles/CMakeError.log".

but once I commented out that line, it ran until

Sorry about that, the gitignore file caught that CMakeLists.txt so I didn't realize it was missing.  This has now been fixed.

 

Linking CXX executable unit_agrad_dot_product_test
1 error generated.
make[2]: *** [src/test/CMakeFiles/unit_common_command_init_test.dir/unit/common/command_init_test.cpp.o] Error 1
make[1]: *** [src/test/CMakeFiles/unit_common_command_init_test.dir/all] Error 2
[ 14%] Built target unit_agrad_dot_product_test
make: *** [all] Error 2

This is a lot more progress than I would have expected in 2 days.

command_init_test is one of the tests that uses a compiled model which I haven't finished yet so this is more or less expected.

 

One question: We have a functioning cluster at Columbia with thousands of cores so the optimal way to use it is to call make individually on every single micro target, rather than something like make test-all. That is possible with static makefiles, although it is a bit tedious to write a shell script to do it. Do you have an idea about how we could do something like that with cmake?

In principle in exactly the same way you would with make.  CMake produces a set of make files which have micro-targets for each executable.  You might need to change into a different directory first, but each one can be called individually.

With CMake it may also be possible to automatically generate that script relatively easily.  Basically, every time a target is created you could write a line out to a text file indicating a new target to be built.

Cheers,

Marcus

 

Ben

On Sunday, April 13, 2014 4:15:41 PM UTC-4, Marcus Brubaker wrote:

Ok, I think I've fixed the pthreads issue.  I've also started a little on building Stan models, but haven't had too much time to work on it today so the tests which rely on compiled models still don't build yet.  I've also added some of the custom compile flags and split things out a bit better.

Cheers,

Marcus

 

--

[Daniel Lee]

Just wanted to say I'd fully support a move away from make. I've done what I could learning and supporting it -- don't really have the time to learn how to implement what we have in CMake, but if someone else could do it and support it for a while, that'd be awesome!

Over the past couple years, we've have a handful of volunteers say they would tackle it, but none have actually succeeded. I'm going to try to maintain our current builds until we get it up and working enough to replace it.

Daniel

[Marcus Brubaker]

Part of the hope with CMake is that it should be easier to maintain going forward so that the work of extending. updating and fixing the build system doesn't fall on one person anymore, whether that's Daniel, me or someone else.  With that in mind, once I get at least all the tests building and running (should be soon...), I'd like to ask that people take a closer look at it.  Not just try to build with it but also look at the CMakeLists.txt files and see whether they make sense.

Also, I can get a CMake-based build system about 90% of the way there, but I will need help from people for that remaining 10%.  Specifically, there are a few things I won't be able to test myself:

- Building on Windows

- Building on Linux for Windows using mingw and unit testing with Wine

- Jenkins integration

(As a side note, I also learned today that CMake can generate XCode project files, which might make it easier for people who use that for debugging/testing/etc.)

Cheers,

Marcus

[Ben Goodrich]

On Tuesday, April 15, 2014 4:38:33 PM UTC-4, Marcus Brubaker wrote:

Also, I can get a CMake-based build system about 90% of the way there, but I will need help from people for that remaining 10%.  Specifically, there are a few things I won't be able to test myself:

- Building on Windows

- Building on Linux for Windows using mingw and unit testing with Wine

- Jenkins integration

Trying to cross-compile from Linux to Windows with mingw failed immediately (full log attached) due to

  x86_64-w64-mingw32-gcc: error: unrecognized command line option
  ‘-rdynamic’

I don't know how to prevent that option from being passed. I'm also not sure why a C compiler is needed.

It does get all the way through with clang++ now.

Ben

[Ben Goodrich]

Full log actually attached now.

Ben

[Marcus Brubaker]

I've pushed a quick change which should fix this.  From the base stan source directory run:

mkdir build-mingw32 ; cd build-mingw32 ; cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/Toolchain-mingw32-linux.cmake ..

On my linux box it now fails with an error about "string table overflow" when compiling the parser but I'll look into that later when I have a bit more time.

Cheers,

Marcus

[Ben Goodrich]

On Wednesday, April 16, 2014 12:02:57 PM UTC-4, Marcus Brubaker wrote:

I've pushed a quick change which should fix this.  From the base stan source directory run:

mkdir build-mingw32 ; cd build-mingw32 ; cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/Toolchain-mingw32-linux.cmake ..

On my linux box it now fails with an error about "string table overflow" when compiling the parser but I'll look into that later when I have a bit more time.

This needs O >= 2. Where does that get set?

Ben

[Marcus Brubaker]

Ah, ok, I've now fixed this.  It is less than elegant but I haven't had time to think about how to do it better.  In short, in src/stan/gm/CMakeLists.txt I have code which was turning optimization off to speed up compilation since we generally don't care about how fast the parser is.  (I'm doing the same thing for all the units tests as well.)  I've made it conditional on using mingw such that if we're not using mingw, we always turn optimization off (to speed up compilation) or if mingw is in use, we always turn it on to work around that bug.

We could get rid of all of this and just adjust the compiler options (CMAKE_CXX_FLAGS*) by hand for the entire project or use different build types (e.g., debug vs release) if that is preferred.  It's certainly more elegant.  Thoughts?

Cheers,

Marcus

[Ben Goodrich]

On Wednesday, April 16, 2014 4:23:48 PM UTC-4, Marcus Brubaker wrote:

Ah, ok, I've now fixed this.  It is less than elegant but I haven't had time to think about how to do it better.  In short, in src/stan/gm/CMakeLists.txt I have code which was turning optimization off to speed up compilation since we generally don't care about how fast the parser is.  (I'm doing the same thing for all the units tests as well.)  I've made it conditional on using mingw such that if we're not using mingw, we always turn optimization off (to speed up compilation) or if mingw is in use, we always turn it on to work around that bug.

We could get rid of all of this and just adjust the compiler options (CMAKE_CXX_FLAGS*) by hand for the entire project or use different build types (e.g., debug vs release) if that is preferred.  It's certainly more elegant.  Thoughts?

The only places where compiler optimization is important are building stanc on Windows and the model tests. For the unit tests and the distribution tests, it generally slows down the wall time but is somewhat useful to do occasionally to make sure there are no compiler bugs. I guess it would be prudent to have different levels of optimization set in the various CMakeLists.txt files but have the ability to override that on a global basis somehow.

Ben

[Marcus Brubaker]

Ok, then maybe the best solution is to simply leave the compiler flags alone instead of adjusting them locally in individual CMakeLists.txt files.  Then, if people want a faster build/test cycle they setup a debug build and when building on Windows or with mingw under Linux they should use the release build to prevent the compiler from choking.

Alternatively, I can make all of this configurable with CMake options.  It's not a huge amount of work, it's just a question of whether it's worth it.

Cheers,

Marcus

[Ben Goodrich]

My cross-compiling works on develop with

make CC=x86_64-w64-mingw32-g++ AR=x86_64-w64-mingw32-ar O_STANC=3 src/test/unit

but it fails on the feature/cmake branch with

goodrich@CYBERPOWERPC:/opt/stan$ mkdir build-mingw32 ; cd build-mingw32 ; cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/Toolchain-mingw32-linux.cmake ..
-- The C compiler identification is GNU 4.8.2
-- The CXX compiler identification is GNU 4.8.2
-- Check for working C compiler: /usr/bin/x86_64-w64-mingw32-gcc
-- Check for working C compiler: /usr/bin/x86_64-w64-mingw32-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/x86_64-w64-mingw32-g++
-- Check for working CXX compiler: /usr/bin/x86_64-w64-mingw32-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- Performing Test HAVE_PIPE_FLAG
-- Performing Test HAVE_PIPE_FLAG - Success
-- Performing Test HAVE_SYNTAXONLY_FLAG
-- Performing Test HAVE_SYNTAXONLY_FLAG - Failed
-- Found Wine executable at: /usr/bin/wine
-- Configuring done
-- Generating done
-- Build files have been written to: /opt/stan/build-mingw32
goodrich@CYBERPOWERPC:/opt/stan/build-mingw32$ make -j1
Scanning dependencies of target stan
[  0%] Building CXX object CMakeFiles/stan.dir/src/stan/agrad/rev/var_stack.cpp.obj
Linking CXX static library libstan.a
[  0%] Built target stan
Scanning dependencies of target print
[  0%] Building CXX object CMakeFiles/print.dir/src/stan/command/print.cpp.obj
Linking CXX executable print.exe
[  0%] Built target print
Scanning dependencies of target stanc
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/ast_def.cpp.obj
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/grammars/expression_grammar_inst.cpp.obj
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/grammars/whitespace_grammar_inst.cpp.obj
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/grammars/program_grammar_inst.cpp.obj
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/grammars/statement_grammar_inst.cpp.obj
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/grammars/var_decls_grammar_inst.cpp.obj
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/grammars/term_grammar_inst.cpp.obj
[  0%] Building CXX object src/stan/gm/CMakeFiles/stanc.dir/grammars/statement_2_grammar_inst.cpp.obj
Linking CXX static library libstanc.a
[  0%] Built target stanc
Scanning dependencies of target stanc-bin
[  0%] Building CXX object CMakeFiles/stanc-bin.dir/src/stan/command/stanc.cpp.obj
Linking CXX executable stanc.exe
[  0%] Built target stanc-bin
Scanning dependencies of target gtest
[  0%] Building CXX object lib/gtest_1.7.0/CMakeFiles/gtest.dir/src/gtest-all.cc.obj
[  0%] Building CXX object lib/gtest_1.7.0/CMakeFiles/gtest.dir/src/gtest_main.cc.obj
Linking CXX static library libgtest.a
[  0%] Built target gtest
Scanning dependencies of target Test_SyntaxOnly_Models
[  0%] Generating function_signatures_student_t_4.cpp
wine: '/home/goodrich/.wine' is a 64-bit installation, it cannot be used with a 32-bit wineserver.

The problem seems to be that it is making a 32bit executable with a 64bit compiler on a 64bit OS

goodrich@CYBERPOWERPC:/opt/stan/build-mingw32$ file stanc.exe
stanc.exe: PE32+ executable (console) x86-64, for MS Windows

I also tried to make a 32bit wine prefix, but Stan didn't seem to work with that either. I did not see anything in the Toolchain file about the bitness. Maybe it also needs to specify the archiver?

Ben

[Marcus Brubaker]

This looks like maybe it is grabbing the wrong wine executable or something?  Do you have multiple versions of wine installed?  

 

The problem seems to be that it is making a 32bit executable with a 64bit compiler on a 64bit OS

goodrich@CYBERPOWERPC:/opt/stan/build-mingw32$ file stanc.exe
stanc.exe: PE32+ executable (console) x86-64, for MS Windows

I also tried to make a 32bit wine prefix, but Stan didn't seem to work with that either. I did not see anything in the Toolchain file about the bitness. Maybe it also needs to specify the archiver?

That says it's a 64bit executable (note the x86-64), not 32bit.

Cheers,

Marcus

[Ben Goodrich]

On Friday, April 18, 2014 6:02:01 PM UTC-4, Marcus Brubaker wrote:

This looks like maybe it is grabbing the wrong wine executable or something?  Do you have multiple versions of wine installed?  

I do. I can fix it by setting WINEARCH="wine64", but I don't know why that is not necessary on the develop branch. Then there is a problem where libwinpthread-1.dll can't be dynamically linked but I pushed a fix for that.

The broader problem (which has little to do with the cmake issue) is that I can't really test mingw because anything multivariate takes 10GB of RAM to compile when using a mingw based on g++-4.8.x. If CRAN ever decides to update Rtools from their unsupported, incompletely standardized mingw that is based on a prerelease of g++-4.6.3, then at least a third of Stan users will be unable to compile many models. We should be proactive about this.

Ben

[Ben Goodrich]

I got a mingw based on g++-4.9 and it is much better (i.e. clang++-like) in its memory requirements iff CFLAGS += -flto=jobserver -ffat-lto-objects

@Daniel, can you try that on Jenkins once the current pull requests finish? I don't think it will improve much with mingw based on g++-4.6 but I think it won't break the build, and if so, we should enable these options in the Makefile.

@Marcus, can you incorporate that into the CMake branch or tell me how to do it?

Ben


[Bob Carpenter]

Nice! Does that mean we can

On May 7, 2014, at 6:47 AM, Ben Goodrich <goodri...@gmail.com> wrote:

> On Saturday, April 19, 2014 5:04:54 PM UTC-4, Ben Goodrich wrote:
> On Friday, April 18, 2014 6:02:01 PM UTC-4, Marcus Brubaker wrote:
> This looks like maybe it is grabbing the wrong wine executable or something? Do you have multiple versions of wine installed?
>
> I do. I can fix it by setting WINEARCH="wine64", but I don't know why that is not necessary on the develop branch. Then there is a problem where libwinpthread-1.dll can't be dynamically linked but I pushed a fix for that.
>
> The broader problem (which has little to do with the cmake issue) is that I can't really test mingw because anything multivariate takes 10GB of RAM to compile when using a mingw based on g++-4.8.x. If CRAN ever decides to update Rtools from their unsupported, incompletely standardized mingw that is based on a prerelease of g++-4.6.3, then at least a third of Stan users will be unable to compile many models. We should be proactive about this.
>
> I got a mingw based on g++-4.9 and it is much better (i.e. clang++-like) in its memory requirements iff CFLAGS += -flto=jobserver -ffat-lto-objects

Nice. I have no idea how you find all those flags. The doc for
these compilers seems thousands of pages long!

Can we just release a set of StanTools along the lines of RTools?
There's not that much in RTools that we need --- just the basic
unix stuff for make and the C++ compilers, I think.

- Bob

[Ben Goodrich]

On Wednesday, May 7, 2014 5:03:56 AM UTC-4, Bob Carpenter wrote:

> I got a mingw based on g++-4.9 and it is much better (i.e. clang++-like) in its memory requirements iff CFLAGS += -flto=jobserver -ffat-lto-objects

Nice.  I have no idea how you find all those flags.  The doc for
these compilers seems thousands of pages long!

I was only dimly aware of LTO until the 4.9 release notes mentioned that they could compile Firefox with 20% of the RAM. We seem to be getting a similar improvement.
 

Can we just release a set of StanTools along the lines of RTools?
There's not that much in RTools that we need --- just the basic
unix stuff for make and the C++ compilers, I think.  

We could, but I doubt it is worth the hassle. It is reasonably easy to install a more recent version of mingw over the mingw based on 4.6 that comes with Rtools. I don't think many R users do this, but we might consider it for Jenkins just so the tests run faster.

Ben

[Marcus Brubaker]

That has been added for any mingw compile now.  You could have also added it on a build-by-build case by using ccmake (or the GUI) to edit the CMAKE_CXX_FLAGS variable on an existing build dir or setting it when calling cmake to setup a new build directory.

Cheers,

Marcus

[Marcus Brubaker]

Actually, Ben, should those LTO flags be passed to the linker as well or just at object compile time?

Cheers,

Marcus

[Ben Goodrich]

On Thursday, May 8, 2014 7:25:33 PM UTC-4, Marcus Brubaker wrote:

Actually, Ben, should those LTO flags be passed to the linker as well or just at object compile time?

They need to be passed at every step. Under the current makefile regime, putting them into CFLAGS worked. I tried them with g++-4.6 and it threw an error about not recognizing -ffat-lto-objects, so my guess is that mingw based on g++-4.6 would do the same. I think we can omit -ffat-lto-objects.

Ben

[Ben Goodrich]

 
For g++-4.6, it seems to work with just -flto. For g++-4.9, it seems to also need -ffat-lto-objects.

Ben

[Marcus Brubaker]

Ok, the latest push should include the flags in the linker as well.  I've also checked for -ffat-lto-objects separately so it will use it if the compiler recognizes it, but not otherwise.

Is it worth turning on LTO even for non-mingw compilers?  It would be easy to do that but I'm not really familiar with what it does and what the tradeoffs are.

Also, is -lwinpthread always the right thing to link against?  On my machine (Fedora 20) it fails because it can't find the library (it's there, but probably somewhere strange) and seems to work just fine without it.

Cheers,

Marcus

[Ben Goodrich]

On Friday, May 9, 2014 11:33:00 AM UTC-4, Marcus Brubaker wrote:

Ok, the latest push should include the flags in the linker as well.  I've also checked for -ffat-lto-objects separately so it will use it if the compiler recognizes it, but not otherwise.

Is it worth turning on LTO even for non-mingw compilers?  It would be easy to do that but I'm not really familiar with what it does and what the tradeoffs are.

Also, is -lwinpthread always the right thing to link against?  On my machine (Fedora 20) it fails because it can't find the library (it's there, but probably somewhere strange) and seems to work just fine without it.

The definitive blog posts about what GCC is doing with LTO is

http://hubicka.blogspot.ca/2014_04_01_archive.html

I think clang++ is a little behind here, but it seems to work with -flto (no =jobserver and no -ffat-lto-objects).

For me, -lwinpthread is necessary but that may either be due to Debian's multilibification or due to having the most recent version of mingw. I can't remember if it was ever not essential for me. You can add a path for the linker on your machine via make/local but I don't know how to incorporate local modifications into CMake.

Ben