g++ segfault with -O2 -DNDEBUG on Mac

[Jiqiang Guo]

It is not a big deal, but it might be good to know that g++ 4.2.1 in Xcode 4.5.2 on Mountain Lion will fail to compile the following file under flag -O2 and -DNDEBUG.  I came across this when I failed to build rstan because I forgot to call set_cppo() again after I used set_cppo('presentation2'), in which -O2 and -DNDEBUG are used.  

In general, I looked at compiling file in Stan directly under these flags. 

mmac2:stan jq$ g++ -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix.cpp 

mmac2:stan jq$ g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix.cpp 

src/stan/agrad/matrix.cpp:136: internal compiler error: Segmentation fault: 11

Please submit a full bug report,

with preprocessed source if appropriate.

See <URL:http://developer.apple.com/bugreporter> for instructions.

mmac2:stan jq$ g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O3 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix.cpp 

mmac2:stan jq$ 

mmac2:stan jq$ g++ -v

Using built-in specs.

Target: i686-apple-darwin11

Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2336.11~28/src/configure --disable-checking --enable-werror --prefix=/Applications/Xcode.app/Contents/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2336.11~28/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1

Thread model: posix

gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)

mmac2:stan jq$ xcodebuild -version

Xcode 4.5.2

Build version 4G2008a

--

Jiqiang

[Bob Carpenter]

On 11/2/12 5:25 PM, Jiqiang Guo wrote:
> It is not a big deal,

I get the same problem on my machine (g++ segfaulting)
with the same config (Xcode 4.5.2 + OS X Mountain Lion).
Here's a fuller report:

options | result
--------------------------
-O0 | OK
-O0 -DNEBUG | OK
--------------------------
-O2 | OK
-O2 -DNEBUG | fail
--------------------------
-O3 | OK
-O3 -DNEBUG | OK


Oddly, it's only -O2 -DNDEBUG failing.

So I guess we can work around this, but it's annoying.

Is -O2 -DNEBUG the default for R before fiddling?
If so, it's probably going to frustrate a lot of users.

> but it might be good to know that g++ 4.2.1 in Xcode 4.5.2 on Mountain Lion will fail to compile
> the following file under flag -O2 and -DNDEBUG. I came across this when I failed to build rstan because I forgot to
> call set_cppo() again after I used set_cppo('presentation2'), in which -O2 and -DNDEBUG are used.

The failure is in agrad/matrix.hpp, which is the most
problematic of all of our files because it involves
auto-dif templating on top of Eigen's templating.

- Bob

[Jiqiang Guo]

In general there is no problem for rstan since in O3 is used by default.  The only case is that if O2 is set by the user in $HOME/.R/Makevars.  (I guess this problem has been there for long, but we don't know.)   

--

Jiqiang  

[Ben Goodrich]

On Friday, November 2, 2012 8:12:00 PM UTC-4, Jiqiang Guo wrote:

In general there is no problem for rstan since in O3 is used by default.  The only case is that if O2 is set by the user in $HOME/.R/Makevars.  (I guess this problem has been there for long, but we don't know.)

Or if the user doesn't ever call set_cppo(). Can you narrow down the problem any with gdb? Do

gdb --args g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix.cpp

Then inside gdb do

run # until the segfault is hit
bt # to see the backtrace

Ben

[Jiqiang Guo]

On Fri, Nov 2, 2012 at 10:05 PM, Ben Goodrich <goodri...@gmail.com> wrote:

On Friday, November 2, 2012 8:12:00 PM UTC-4, Jiqiang Guo wrote:

In general there is no problem for rstan since in O3 is used by default.  The only case is that if O2 is set by the user in $HOME/.R/Makevars.  (I guess this problem has been there for long, but we don't know.)

Or if the user doesn't ever call set_cppo().

package rstan uses a Makeefile in which the setting of O3 will overrule R's default O2. 

So there is not a problem if users do not call set_cppo() at all.  Once rstan is installed, 

this file need not be compiled.

 

Can you narrow down the problem any with gdb? Do

gdb --args g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix.cpp

Then inside gdb do

run # until the segfault is hit
bt # to see the backtrace

I got nothing from using bt inside gdb: it says 'No stack', thought the error message says (I guess it is saying line 136). 

src/stan/agrad/matrix.cpp:136: internal compiler error: Segmentation fault: 11

--

Jiqiang

 

Ben

--
 
 

[Ben Goodrich]

On Friday, November 2, 2012 10:21:50 PM UTC-4, Jiqiang Guo wrote:

I got nothing from using bt inside gdb: it says 'No stack', thought the error message says (I guess it is saying line 136). 

src/stan/agrad/matrix.cpp:136: internal compiler error: Segmentation fault: 11

Yes, although that is the last line, so it doesn't help much. You could try Mav bisecting it. Just comment out the first half of the functions and try to compile. If it works, you know that the problem is in the first half so try commenting out the first quarter. If it fails, the problem is in the last half ...

Ben

[Jiqiang Guo]

Ok, Ben's suggestion tells use it is function eigenvalues_sym that g++ 4.2.1 cannot deal with. 

--

Jiqiang 

$ g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix_tmp.cpp 

src/stan/agrad/matrix_tmp.cpp:17: internal compiler error: Segmentation fault: 11

Please submit a full bug report,

with preprocessed source if appropriate.

See <URL:http://developer.apple.com/bugreporter> for instructions.

$ more src/stan/agrad/matrix_tmp.cpp 

#include "stan/agrad/matrix.hpp"

namespace stan {

  namespace agrad {

    vector_v eigenvalues_sym(const matrix_v& m) {

      stan::math::validate_nonzero_size(m,"eigenvalues_sym");

      stan::math::validate_square(m,"eigenvalues_sym");

      Eigen::SelfAdjointEigenSolver<matrix_v> solver(m,Eigen::EigenvaluesOnly);

      return solver.eigenvalues().real();

    }

  }

}

Ben

--
 
 

[Ben Goodrich]

On Saturday, November 3, 2012 11:19:00 AM UTC-4, Jiqiang Guo wrote:

Ok, Ben's suggestion tells use it is function eigenvalues_sym that g++ 4.2.1 cannot deal with. 

 

    vector_v eigenvalues_sym(const matrix_v& m) {      stan::math::validate_nonzero_size(m,"eigenvalues_sym");

      stan::math::validate_square(m,"eigenvalues_sym");

      Eigen::SelfAdjointEigenSolver<matrix_v> solver(m,Eigen::EigenvaluesOnly);

      return solver.eigenvalues().real();

    }

Try eliminating the .real() thing. For a SelfAdjoint (really symmetric) matrix, all the eigenvalues are real by construction. I would think .real() would be a no-op but maybe it is a segfault.

Ben

[Jiqiang Guo]

That is not the reason.  I went ahead figuring out it is line 7.  See the following line 7 being commented out and not. 

$ nl src/stan/agrad/matrix_tmp.cpp | more

     1  #include "stan/agrad/matrix.hpp"

        

     2  namespace stan {

        

     3    namespace agrad {

        

     4      vector_v eigenvalues_sym(const matrix_v& m) {

     5        stan::math::validate_nonzero_size(m,"eigenvalues_sym");

     6        stan::math::validate_square(m,"eigenvalues_sym");

     7        // Eigen::SelfAdjointEigenSolver<matrix_v> solver(m,Eigen::EigenvaluesOnly);

     8        return vector_v();

     9        // return solver.eigenvalues();// .real();

    10      }

        

    11    }

    12  }

$ g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix_tmp.cpp 

$ vi src/stan/agrad/matrix_tmp.cpp 

$ nl src/stan/agrad/matrix_tmp.cpp | more                                                                                           1  #include "stan/agrad/matrix.hpp"

        

     2  namespace stan {

        

     3    namespace agrad {

        

     4      vector_v eigenvalues_sym(const matrix_v& m) {

     5        stan::math::validate_nonzero_size(m,"eigenvalues_sym");

     6        stan::math::validate_square(m,"eigenvalues_sym");

     7        Eigen::SelfAdjointEigenSolver<matrix_v> solver(m,Eigen::EigenvaluesOnly);

     8        return vector_v();

     9        // return solver.eigenvalues();// .real();

    10      }

        

    11    }

    12  }

$ g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix_tmp.cpp 

src/stan/agrad/matrix_tmp.cpp:18: internal compiler error: Segmentation fault: 11

Please submit a full bug report,

with preprocessed source if appropriate.

See <URL:http://developer.apple.com/bugreporter> for instructions.

Ben

--
 
 

[Ben Goodrich]

Does this compile?

Eigen::Matrix<double,Eigen::Dynamic,1> eigenvalues_sym(const Eigen::MatrixXd& m) {

  Eigen::SelfAdjointEigenSolver<Eigen::MatrixXd> solver(m,Eigen::EigenvaluesOnly);

  return solver.eigenvalues();

}

If not, it looks like a bug in Eigen.

Ben

[Jiqiang Guo]

It is good. 

$ g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function   -c -o bin/stan/agrad/matrix.o src/stan/agrad/matrix_tmp.cpp 

$ more src/stan/agrad/matrix_tmp.cpp 

#include "stan/agrad/matrix.hpp"

Eigen::Matrix<double,Eigen::Dynamic,1> eigenvalues_sym(const Eigen::MatrixXd& m) {

  Eigen::SelfAdjointEigenSolver<Eigen::MatrixXd> solver(m,Eigen::EigenvaluesOnly);

  return solver.eigenvalues();

}

[Ben Goodrich]

I guess it does not like custom scalar types.

[Bob Carpenter]

Nice detective work! It's working for me now with the
current version.

This is one of those issues where I really wonder what's
going on so that it worked everywhere but g++ -O2 -NDEBUG.
But not enough to dig deeper.

- Bob

On 11/3/12 12:32 PM, Ben Goodrich wrote:
> I guess it does not like custom scalar types.
>
> On Saturday, November 3, 2012 12:23:05 PM UTC-4, Jiqiang Guo wrote:
>
> It is good.
>
> $ g++ -DNDEBUG -I src -I lib/eigen_3.1.1 -I lib/boost_1.51.0 -O2 -Wall -Wno-unused-function -c -o
> bin/stan/agrad/matrix.o src/stan/agrad/matrix_tmp.cpp
> $ more src/stan/agrad/matrix_tmp.cpp
> #include "stan/agrad/matrix.hpp"
> Eigen::Matrix<double,Eigen::Dynamic,1> eigenvalues_sym(const Eigen::MatrixXd& m) {
> Eigen::SelfAdjointEigenSolver<Eigen::MatrixXd> solver(m,Eigen::EigenvaluesOnly);
> return solver.eigenvalues();
> }
>
>
> On Sat, Nov 3, 2012 at 12:10 PM, Ben Goodrich <goodri...@gmail.com <javascript:>> wrote:
>
> Eigen::Matrix<double,Eigen::Dynamic,1> eigenvalues_sym(const Eigen::MatrixXd& m) {

> Eigen::__SelfAdjointEigenSolver<Eigen::MatrixXd> solver(m,Eigen::__EigenvaluesOnly);
> return solver.eigenvalues();
> }
>
>
> --
>
>

[Ben Goodrich]

On Saturday, November 3, 2012 1:27:35 PM UTC-4, Bob Carpenter wrote:

Nice detective work!  It's working for me now with the
current version.

This is one of those issues where I really wonder what's
going on so that it worked everywhere but g++ -O2 -NDEBUG.
But not enough to dig deeper. 

Most likely it is a compiler bug. But there is a nontrivial chance that the traits of the var scalar type are not set right or something. So it would be good to figure it out.

Ben