MKL and Stan-math

[Sebastian Weber]

Hi!

The test mdivide_left_grad_vv and mdivide_left_grad_vd do fail for small deviations from the expected precision (1.5e-12 > 1.0e-12) when I run the Stan-Math tests with -DEIGEN_USE_MKL_ALL (which is equal to EIGEN_USE_BLAS, EIGEN_USE_LAPACKE, and EIGEN_USE_MKL_VML). So my two question:

- Is such a high precision needed by Stan-Math or could it be relaxed to, let's say 1.0E-10?
- Can I safely run Stan with the lower precision?

For more information on Eigen+MKL, see

http://beamteam.usask.ca/mark/acquaman/dd/d88/_topic_using_intel_m_k_l.html

Even using the more "strict" EIGEN_USE_BLAS, EIGEN_USE_LAPACKE_STRICT and EIGEN_USE_MKL_VML triggers this test to fail. I can only get successfull Stan-Math tests iff I use

EIGEN_USE_BLAS and EIGEN_USE_MKL_VML

which is not optimal I guess, as the LAPACK routines from MKL would not be used. This stuff may be included in the cmdstan documentation on MKL for other users.

Best,
Sebastian

[ RUN ] AgradRevMatrix.mdivide_left_grad_vv
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:86: Failure
The difference between Cd(i,j) and g[k] is 1.5916157281026244e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to -273.0000000000008,
g[k] evaluates to -273.00000000000239, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:86: Failure
The difference between Cd(i,j) and g[k] is 1.3073986337985843e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to 210.00000000000054,
g[k] evaluates to 210.00000000000185, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:86: Failure
The difference between Cd(i,j) and g[k] is 1.3073986337985843e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to -280.0000000000008,
g[k] evaluates to -280.0000000000021, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:86: Failure
The difference between Cd(i,j) and g[k] is 1.1084466677857563e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to 217.00000000000054,
g[k] evaluates to 217.00000000000165, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:86: Failure
The difference between Cd(i,j) and g[k] is 1.0516032489249483e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to 195.00000000000054,
g[k] evaluates to 195.00000000000159, and
1.0E-12 evaluates to 9.9999999999999998e-13.

[ RUN ] AgradRevMatrix.mdivide_left_grad_vd
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:178: Failure
The difference between Cd(i,j) and g[k] is 1.5916157281026244e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to -273.0000000000008,
g[k] evaluates to -273.00000000000239, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:178: Failure
The difference between Cd(i,j) and g[k] is 1.3073986337985843e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to 210.00000000000054,
g[k] evaluates to 210.00000000000185, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:178: Failure
The difference between Cd(i,j) and g[k] is 1.3073986337985843e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to -280.0000000000008,
g[k] evaluates to -280.0000000000021, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:178: Failure
The difference between Cd(i,j) and g[k] is 1.1084466677857563e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to 217.00000000000054,
g[k] evaluates to 217.00000000000165, and
1.0E-12 evaluates to 9.9999999999999998e-13.
test/unit/math/rev/mat/fun/mdivide_left_test.cpp:178: Failure
The difference between Cd(i,j) and g[k] is 1.0800249583553523e-12, which exceeds 1.0E-12, where
Cd(i,j) evaluates to 195.00000000000054,
g[k] evaluates to 195.00000000000162, and
1.0E-12 evaluates to 9.9999999999999998e-13.
[ FAILED ] AgradRevMatrix.mdivide_left_grad_vd (1 ms)

[Daniel Lee]

Hi Sebastian,

On Thu, Jul 16, 2015 at 4:28 AM, Sebastian Weber <sdw....@gmail.com> wrote:

Hi!

The test mdivide_left_grad_vv and mdivide_left_grad_vd do fail for small deviations from the expected precision (1.5e-12 > 1.0e-12) when I run the Stan-Math tests with -DEIGEN_USE_MKL_ALL (which is equal to EIGEN_USE_BLAS, EIGEN_USE_LAPACKE, and EIGEN_USE_MKL_VML). So my two question:

- Is such a high precision needed by Stan-Math or could it be relaxed to, let's say 1.0E-10?

I don't think it needs that high of a precision.

If you changed it locally to 1.0E-10, does everything pass?

 

- Can I safely run Stan with the lower precision?

I believe so.

 

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sebastian Weber]

Hi!

Great to hear that the precision does not need to be that high. I created an issue for easier tracking. I am currently running tests with full MKL and decreased precision to 1e-10 - I will let you know once done.

Best,
Sebastian

[Bob Carpenter]

Just to be clear, we don't have a good handle on what
losing a couple decimal places of accuracy in lower-level
computations will do to things like the accuracy of the leapfrog
integrator, which determines accept/reject thresholds and
thus impacts sampling efficiency. Maybe we can run some
experiments on models that matter to us with both clang++
and Intel to see if they get the same answers in the same time.

- Bob

[Sebastian Weber]

Hi!

Is there an example you have in mind which we can be easily kicked off? I am happy to run such an experiment. Of course, it should contain some linear algebra to make sense.

Best,
Sebastian

On Thursday, July 16, 2015 at 8:42:55 PM UTC+2, Bob Carpenter wrote:
> Just to be clear, we don't have a good handle on what
> losing a couple decimal places of accuracy in lower-level
> computations will do to things like the accuracy of the leapfrog
> integrator, which determines accept/reject thresholds and
> thus impacts sampling efficiency. Maybe we can run some
> experiments on models that matter to us with both clang++
> and Intel to see if they get the same answers in the same time.
>
> - Bob
>
> > On Jul 16, 2015, at 8:44 AM, Sebastian Weber
> >

[Bob Carpenter]

Afraid not --- this is a big research problem and I'm guessing
the answer will vary by model.

Overall, I doubt it'll be a big deal. If you lost half the precision
(say seven decimal places, as in going to a float from a double), then
things can get dicey. Beyond that is real trouble.

- Bob