make clean-all not cleaning CVODES binaries

[Daniel Lee]

Hi all,

I just submitted 3 pull requests to fix this. There's one on math, stan, and cmdstan.

Daniel

On Tue, Aug 16, 2016 at 4:51 AM, Bob Carpenter <ca...@alias-i.com> wrote:

Yes, please --- clean-all should really clean everything.

I have no idea how it ran at all then without cleaning all
the object files.

- Bob

> On Aug 16, 2016, at 9:12 AM, Sebastian Weber <sdw....@gmail.com> wrote:
>
>
> That's the problem - make clean-all DOES NOT remove all binaries. Just observe what gets deleted; these are only the .a files which are essentially archives of the object .o files. The .o object files of CVODES are NOT deleted by clean-all. I can fix that.
>
> And yes, I have run gcc 6.1 from MacPorts on Mac OS X El Capitan and everything runs just fine.
>
> Sebastian
>
> On Tuesday, August 16, 2016 at 1:30:09 AM UTC+2, Bob Carpenter wrote:
>> I was cleaning everything using make clean-all.  Is this
>> not something that make clean-all takes care of?  I'd have
>> thought if not that my previous clang++ versions would've conflicted.
>>
>> Did you get it working on Mac OS X with g++6?  Ben reports
>> that it works on the Linux (?) versions as is.
>>
>> - Bob
>>
>>> On Aug 15, 2016, at 9:23 PM, Sebastian Weber
>>>
>>> Hi Bob!
>>>
>>> I was just able to reproduce your error - and also found a very easy solution: Recompile *everything* with the gcc 6 and you get things working again. Our make clean-all is not cleaning everything as advertised. You have to add a
>>>
>>> find lib/cvodes_2.8.2/ -name "*o" -exec rm {} \;
>>>
>>> to get rid of all the object files. So far the clean-all merely cleans the built libraries. So really cleaning everything triggers a rebuild of cvodes 2.8.2 using gcc 6. If you then run the tests in question, they will be successful without changing any code in Stan at all.
>>>
>>> I hope this clears up matters for you as well. Maybe we should fix the makefile...
>>>
>>> Best,
>>> Sebastian
>>>
>>> On Monday, August 15, 2016 at 2:44:09 AM UTC+2, Bob Carpenter wrote:
>>>> Also, if I replace the theta.at(0) line with:
>>>>
>>>>   throw std::out_of_range("foo bar baz");
>>>>
>>>> it's not caught and I get this:
>>>>
>>>> terminate called after throwing an instance of 'std::out_of_range'
>>>> what():  foo bar baz
>>>> test/unit/math/rev/mat/functor/foo_test --gtest_output="xml:test/unit/math/rev/mat/functor/foo_test.xml" failed
>>>>
>>>> So std::vector isn't the problem, it's how the ODE system
>>>> function is being called.
>>>>
>>>> And also, this is relevant:
>>>>
>>>> http://stackoverflow.com/questions/6426835/what-happens-if-exception-gets-thrown-through-c-code
>>>>
>>>> Sorry to be spamming our lists like this.  I'm giving up.
>>>> This is too complicated with the call of C.  I think that's
>>>> where the problem's arising --- g++6 changed the behavior
>>>> somehow.
>>>>
>>>> - Bob
>>>>
>>>>> On Aug 15, 2016, at 2:25 AM, Bob Carpenter
>>>>>
>>>>> Oh, and the error arises in our code on line 22 of
>>>>>
>>>>> test/unit/math/prim/arr/functor/harmonic_oscillator.hpp
>>>>>
>>>>> whenit tries to access theta[0]:
>>>>>
>>>>>  res.push_back(-y_in.at(0) - theta.at(0)*y_in.at(1));
>>>>>
>>>>> If I replace the above line with:
>>>>>
>>>>>  res.push_back(-y_in.at(0) - 0.15 * y_in.at(1));
>>>>>
>>>>> then there's no exception thrown, no error, and everything's hunky-dory.
>>>>>
>>>>> This oscillator's being used as a callback inside the C code
>>>>> of Sundials.
>>>>>
>>>>> What a mess.  I'm very very tired of looking all this stuff.
>>>>>
>>>>> - Bob
>>>>>
>>>>>> On Aug 15, 2016, at 1:45 AM, Bob Carpenter
>>>>>>
>>>>>> I can't figure out what's going on with the tests for
>>>>>> integrate_ode_bdf.  They fail on g++6 because a std::out_of_range
>>>>>> exception isn't being caught the same way it is in earlier compilers.
>>>>>> I don't think this has anything to do with the int arguments
>>>>>> that set me down this path in this issue:
>>>>>>
>>>>>> https://github.com/stan-dev/math/issues/327
>>>>>>
>>>>>> Otherwise, I have everything working for g++6 and int arguments (see
>>>>>> below for more details and how I tested this) pushed to
>>>>>> this branch:
>>>>>>
>>>>>> stan_math(bugfix/0327-log-int)
>>>>>>
>>>>>> The problem I'm having is that I can't seem to catch the
>>>>>> std::out_of_range exception thrown by integrate_ode_bdf with
>>>>>> g++6.  This is a simplified form of the existing unit test,
>>>>>> which works just fine with other compilers.
>>>>>>
>>>>>> test/unit/math/rev/mat/functor/foo_test.cpp
>>>>>> ==============================================================
>>>>>> #include <stan/math/rev/mat.hpp>
>>>>>> #include <test/unit/math/prim/arr/functor/harmonic_oscillator.hpp>
>>>>>> #include <gtest/gtest.h>
>>>>>> #include <iostream>
>>>>>> #include <vector>
>>>>>> #include <stdexcept>
>>>>>>
>>>>>> TEST(foo, bar) {
>>>>>> using stan::math::integrate_ode_bdf;
>>>>>> harm_osc_ode_fun harm_osc;
>>>>>> std::vector<double> y0;
>>>>>> y0.push_back(1.0);
>>>>>> y0.push_back(0.0);
>>>>>> double t0 = 0;
>>>>>> std::vector<double> ts;
>>>>>> for (int i = 0; i < 100; i++)
>>>>>> ts.push_back(t0 + 0.1 * (i + 1));
>>>>>> std::vector<double> x(3,1);
>>>>>> std::vector<int> x_int(2,0);
>>>>>> std::vector<double> theta_bad;
>>>>>> // theta_bad.push_back(0.15);  // no abort with integrate_ode if included
>>>>>>
>>>>>> try {
>>>>>> // cathces out_of_range
>>>>>> // theta_bad.at(4);
>>>>>>
>>>>>> // *** ABORTS ***
>>>>>> integrate_ode_bdf(harm_osc, y0, t0, ts, theta_bad,
>>>>>>                   x, x_int, 0, 1e-8, 1e-10, 1e6);
>>>>>> } catch (const std::out_of_range& e) {
>>>>>> std::cout << "Got out_of_range" << std::endl;
>>>>>> }
>>>>>> std::cout << "FINI" << std::endl;
>>>>>> }
>>>>>> ================================================================
>>>>>>
>>>>>> With the theta_bad.at(4) uncommented, the out_of_range exception is caught
>>>>>> and the test prints this:
>>>>>>
>>>>>>
>>>>>> [ RUN      ] foo.bar
>>>>>> Got out_of_range
>>>>>> FINI
>>>>>> [       OK ] foo.bar (1 ms)
>>>>>>
>>>>>>
>>>>>> With theta_bad.at(4) commented out so that integrate_ode_bdf runs,
>>>>>> I get this cryptic result:
>>>>>>
>>>>>>
>>>>>> [ RUN      ] foo.bar
>>>>>> terminate called after throwing an instance of 'std::out_of_range'
>>>>>> what():  vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)
>>>>>> test/unit/math/rev/mat/functor/foo_test --gtest_output="xml:test/unit/math/rev/mat/functor/foo_test.xml" failed
>>>>>>
>>>>>>
>>>>>> What has me so confused is that it says that the termination happens
>>>>>> after throwing std::out_of_range.  I can't even catch it with catch (...).
>>>>>> I've tried to look this up and can't seem to find any help online.
>>>>>>
>>>>>> Anyone have a clue as to what's going on?
>>>>>>
>>>>>> Otherwise, I have everything working with the int specializations
>>>>>> of log(), exp(), log1p() and log2(), which for some reason, are the only
>>>>>> four of the functions that seem to cause problems with int arguments.
>>>>>> I made sure the following generalization of Ben's test with all the cmath
>>>>>> functions we support (from C++03 and C++11) would compile from CmdStan:
>>>>>>
>>>>>>
>>>>>> transformed data {
>>>>>> real x;
>>>>>> x = 0;
>>>>>> for (i in 1:2)
>>>>>> x = x + cos(i) + sin(i) + tan(i)
>>>>>>   + acos(i) + asin(i) + atan(i)
>>>>>>   + atan2(x, i) + atan2(i, x) + atan2(i, i)
>>>>>>   + cosh(i) + sinh(i) + tanh(i)
>>>>>>   + acosh(i) + asinh(i) + atanh(i)
>>>>>>   + exp(i) + log(i) + log10(i) + log1p(i) + log2(i)
>>>>>>   + pow(x, i) + pow(i, x) + pow(i, i) + sqrt(i) + cbrt(i)
>>>>>>   + hypot(x, i) + hypot(i, x) + hypot(i, i)
>>>>>>   + erf(i) + erfc(i) + tgamma(i) + lgamma(i)
>>>>>>   + ceil(i) + floor(i) + round(i)
>>>>>>   + fmax(i, x) + fmax(x, i) + fmax(i, i)
>>>>>>   + fmin(i, x) + fmin(x, i) + fmin(i, i);
>>>>>> }
>>>>>> model {
>>>>>> }
>>>>>>
>>>>>>
>>>>>> - Bob
>>>>>>
>>>>>> P.S.  In case you care, I installed g++6 from MacPorts and here's its version:
>>>>>>
>>>>>> ~/cmdstan/stan/lib/stan_math(bugfix/0327-log-int)$ /opt/local/bin/g++-mp-6 --version
>>>>>> g++-mp-6 (MacPorts gcc6 6.1.0_0) 6.1.0
>>>>>> Copyright (C) 2016 Free Software Foundation, Inc.
>>>>>> This is free software; see the source for copying conditions.  There is NO
>>>>>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>>>>>>
>>>>>> It runs all the unit tests and integration tests for stan-dev/stan
>>>>>> and all the unit tests for stan-dev/math:
>>>>>>
>>>>>>
>>>>>> P.P.S.  And in case you want to see the Stan test build after a clean-all:
>>>>>>
>>>>>>
>>>>>> ~/cmdstan/stan/lib/stan_math(bugfix/0327-log-int)$ make clean-all
>>>>>> removing test executables
>>>>>> rm -f -r doc/api
>>>>>> removing dependency files
>>>>>> rm -f
>>>>>> removing generated test files
>>>>>> rm -f test/gtest.o test/libgtest.a
>>>>>> rm -f lib/cvodes_2.8.2/lib/libsundials_cvodes.a lib/cvodes_2.8.2/lib/libsundials_nvecserial.a
>>>>>>
>>>>>>
>>>>>> ~/cmdstan/stan/lib/stan_math(bugfix/0327-log-int)$ ./runTests.py -j4 test/unit/math/rev/mat/functor/foo_test.cpp
>>>>>> ------------------------------------------------------------
>>>>>> make -j4 test/unit/math/rev/mat/functor/foo_test
>>>>>> ar -rs lib/cvodes_2.8.2/lib/libsundials_nvecserial.a lib/cvodes_2.8.2/src/nvec_ser/nvector_serial.o lib/cvodes_2.8.2/src/sundials/sundials_math.o
>>>>>> /opt/local/bin/g++-mp-6 -I . -isystem lib/eigen_3.2.8 -isystem lib/boost_1.60.0 -isystemlib/cvodes_2.8.2/include -Wall -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DNO_FPRINTF_OUTPUT -pipe -Wno-unused-function   -c -O0 -DGTEST_USE_OWN_TR1_TUPLE -isystem lib/gtest_1.7.0/include -isystem lib/gtest_1.7.0 lib/gtest_1.7.0/src/gtest-all.cc -o test/gtest.o
>>>>>> ar: creating archive lib/cvodes_2.8.2/lib/libsundials_nvecserial.a
>>>>>> ar -rs lib/cvodes_2.8.2/lib/libsundials_cvodes.a lib/cvodes_2.8.2/src/cvodes/cvodes.o lib/cvodes_2.8.2/src/cvodes/cvodes_io.o lib/cvodes_2.8.2/src/cvodes/cvodea.o lib/cvodes_2.8.2/src/cvodes/cvodea_io.o lib/cvodes_2.8.2/src/cvodes/cvodes_direct.o lib/cvodes_2.8.2/src/cvodes/cvodes_band.o lib/cvodes_2.8.2/src/cvodes/cvodes_dense.o lib/cvodes_2.8.2/src/cvodes/cvodes_diag.o lib/cvodes_2.8.2/src/cvodes/cvodes_spils.o lib/cvodes_2.8.2/src/cvodes/cvodes_spbcgs.o lib/cvodes_2.8.2/src/cvodes/cvodes_spgmr.o lib/cvodes_2.8.2/src/cvodes/cvodes_sptfqmr.o lib/cvodes_2.8.2/src/cvodes/cvodes_sparse.o lib/cvodes_2.8.2/src/cvodes/cvodes_bandpre.o lib/cvodes_2.8.2/src/cvodes/cvodes_bbdpre.o lib/cvodes_2.8.2/src/sundials/sundials_band.o lib/cvodes_2.8.2/src/sundials/sundials_dense.o lib/cvodes_2.8.2/src/sundials/sundials_direct.o lib/cvodes_2.8.2/src/sundials/sundials_iterative.o lib/cvodes_2.8.2/src/sundials/sundials_math.o lib/cvodes_2.8.2/src/sundials/sundials_nvector.o lib/cvodes_2.8.2/src/sundials/sundials_pcg.o lib/cvodes_2.8.2/src/sundials/sundials_sparse.o lib/cvodes_2.8.2/src/sundials/sundials_spbcgs.o lib/cvodes_2.8.2/src/sundials/sundials_spfgmr.o lib/cvodes_2.8.2/src/sundials/sundials_spgmr.o lib/cvodes_2.8.2/src/sundials/sundials_sptfqmr.o
>>>>>> /opt/local/bin/g++-mp-6 -I . -isystem lib/eigen_3.2.8 -isystem lib/boost_1.60.0 -isystemlib/cvodes_2.8.2/include -Wall -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DNO_FPRINTF_OUTPUT -pipe -Wno-unused-function   -c -O0 -DGTEST_USE_OWN_TR1_TUPLE -isystem lib/gtest_1.7.0/include -isystem lib/gtest_1.7.0 test/unit/math/rev/mat/functor/foo_test.cpp -o test/unit/math/rev/mat/functor/foo_test.o
>>>>>> ar: creating archive lib/cvodes_2.8.2/lib/libsundials_cvodes.a
>>>>>> ar rv test/libgtest.a test/gtest.o
>>>>>> ar: creating archive test/libgtest.a
>>>>>> a - test/gtest.o
>>>>>> /opt/local/bin/g++-mp-6 -I . -isystem lib/eigen_3.2.8 -isystem lib/boost_1.60.0 -isystemlib/cvodes_2.8.2/include -Wall -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DNO_FPRINTF_OUTPUT -pipe -Wno-unused-function    -O0 lib/gtest_1.7.0/src/gtest_main.cc test/unit/math/rev/mat/functor/foo_test.o -DGTEST_USE_OWN_TR1_TUPLE -isystem lib/gtest_1.7.0/include -isystem lib/gtest_1.7.0 -o test/unit/math/rev/mat/functor/foo_test test/libgtest.a  lib/cvodes_2.8.2/lib/libsundials_nvecserial.a lib/cvodes_2.8.2/lib/libsundials_cvodes.a
>>>>>> rm test/unit/math/rev/mat/functor/foo_test.o
>>>>>> ------------------------------------------------------------
>>>>>> test/unit/math/rev/mat/functor/foo_test --gtest_output="xml:test/unit/math/rev/mat/functor/foo_test.xml"
>>>>>> Running main() from gtest_main.cc
>>>>>> [==========] Running 1 test from 1 test case.
>>>>>> [----------] Global test environment set-up.
>>>>>> [----------] 1 test from foo
>>>>>> [ RUN      ] foo.bar
>>>>>> terminate called after throwing an instance of 'std::out_of_range'
>>>>>> what():  vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)
>>>>>> test/unit/math/rev/mat/functor/foo_test --gtest_output="xml:test/unit/math/rev/mat/functor/foo_test.xml" failed
>>>>>> exit now (08/15/16 01:38:10 CEST)
>>>>>>
>>>>>>
>>>>>> ~/cmdstan/stan/lib/stan_math(bugfix/0327-log-int)$
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.