Stan and Intel MKL?

[Sebastian Weber]

Hi!

I was wondering if we should tweak Stan such that it can specifically take advantage of the MKL? I know this thing cost money, but those who have it get some speedups here for free (assuming its there). Use cases

1. Enabling the MKL for Eigen gave me ~10-20% speedup. Depending on your model you may get even more.

2. It is possible to make CVODES use the Intel MKL. I have never tried this, but from talking to people this may give a 2X speedup for ODE solving; again "for free".

Specifically what I have in mind to do is to let the build system define a macro, called

STAN_USE_MKL

for example. If this macro is defined, then CVODES is instructed to be built slightly different and will then use MKL routines which hopefully expedite things.

Another advantage to deploy this in a pre-configured way is that we can ensure that the MKL is used in the right way as you can shoot yourself in the foot otherwise. At least the unit tests of stan-math are not successful whenever you switch on all optimizations from the Intel compiler.

Would that be an appropiate approach?

Sebastian

[Bob Carpenter]

> On Aug 22, 2016, at 10:40 AM, Sebastian Weber <sdw....@gmail.com> wrote:
>
> Hi!
>
> I was wondering if we should tweak Stan such that it can specifically take advantage of the MKL?

We can discuss at the meeting, but it'd be OK with me
to have conditional builds if you're willing to maintain
them. It sounds like there are multiple versions of MKL.
Are you proposing to just add build support for the one
that Novartis uses or also the more recent ones?

> I know this thing cost money,

Not just the compiler, also the tweaking. Though the latter
is more in our time (really, that's going to be your time and
whoever else wants to customize for MKL).

> but those who have it get some speedups here for free (assuming its there). Use cases
>
> 1. Enabling the MKL for Eigen gave me ~10-20% speedup. Depending on your model you may get even more.

That's nothing to sneeze at.

> 2. It is possible to make CVODES use the Intel MKL. I have never tried this, but from talking to people this may give a 2X speedup for ODE solving; again "for free".

That's a huge speedup, so I'd be curious if you can get
that with your ODEs and your version of MKL.

> Specifically what I have in mind to do is to let the build system define a macro, called
>
> STAN_USE_MKL
>
> for example. If this macro is defined, then CVODES is instructed to be built slightly different and will then use MKL routines which hopefully expedite things.

That sounds reasonable to me, but we have to run that by Daniel.

> Another advantage to deploy this in a pre-configured way is that we can ensure that the MKL is used in the right way as you can shoot yourself in the foot otherwise. At least the unit tests of stan-math are not successful whenever you switch on all optimizations from the Intel compiler.

Is there a way to fix them (even with conditionals) so that
they all pass? And I don't mean editing out the ones that don't.
This would be more ongoing maintenance from whoever is working
on the MKL.

- Bob

[Sebastian Weber]

Hi!

Yes, this is a good topic for the meetings. Let me clarify that two things are involved here:

- MKL
- Intel compiler

The MKL can come independent of the Intel compiler while the Intel compiler usually brings with it the MKL. As far as I am aware neither are free, only if you use it for academic only purposes there are some possibilities to use them free of charge.

I myself was always curious at what the real speedup for CVODES is - the 2X sounds to good to be true, but I am already happy with a 1.5X. The changes this implies, I can easily maintain; but I can't do magic, i.e. I can do so with what we have here which is the 2015.3 version (the second latest to my knowledge).

Wrt. to stan-math and the Intel compiler: You have to enable the options "-fp-model precise -fp-model source" in order to get units tests to pass (I am not sure if the cmdstan manual states this). As far as I recall, I did not dare to throw in the MKL on top of that, but I can investigate should we want this.

Best,
Sebastian

On Monday, August 22, 2016 at 1:04:14 PM UTC+2, Bob Carpenter wrote:
> > On Aug 22, 2016, at 10:40 AM, Sebastian Weber
> >

[Daniel Lee]

Hi Sebastian,

As Bob mentioned, it's OK with conditional builds. See the Math / Stan / CmdStan makefiles in each of those repos and in make/os_*. We're doing conditional builds now.

For the Intel Compiler, we can just add settings that only apply when using Intel.

For the tests -- if you need to add that to the tests, we should update the manual. I think you might have written that part about MKL, so please feel free to update it.

What would you suggest happens so that this works? I looked into the licenses a while ago and it looked like we'd have to pay to have it. We don't have that many users asking for it, so it's much less priority for me to maintain the code for yet another compiler.

Daniel

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bob Carpenter]

> On Aug 22, 2016, at 5:23 PM, Daniel Lee <bea...@alum.mit.edu> wrote:
>
> Hi Sebastian,
>
> As Bob mentioned, it's OK with conditional builds. See the Math / Stan / CmdStan makefiles in each of those repos and in make/os_*. We're doing conditional builds now.
>
> For the Intel Compiler, we can just add settings that only apply when using Intel.
>
> For the tests -- if you need to add that to the tests, we should update the manual. I think you might have written that part about MKL, so please feel free to update it.

That can be done in the pull request to add conditions to
the compiler.

> What would you suggest happens so that this works? I looked into the licenses a while ago and it looked like we'd have to pay to have it. We don't have that many users asking for it, so it's much less priority for me to maintain the code for yet another compiler.

Are you worried about setting up another test server config?
Already these Travis jobs seem pretty fragile, so my main concern
isn't maintaining the scripts, but continually have to restart
issues.

- Bob

[Daniel Lee]

No. I'm worried about the nonstandard / not implemented like the other compilers parts of each compiler.

Remember your g++-6 hack? There are a few msvc hacks in the code. When we run into these problems, it's always a pain to figure out what's going on. If we can't get the tool chain locally or access it immediately, it really becomes a guessing game.

Of course, Intel might have implemented their computer without any issues. If that's the case, we're good and those are hypothetical concerns that can be ignored.

> - Bob
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Sebastian Weber]

Hi!

Let's go in steps here:

1. Ensure in the built system that the intel compiler is called with the right options

2. Include in the build system a flag for having the MKL which allows to instruct Eigen to use MKL as well

3. Evaluate the performance gain for CVODES when using the MKL; only if this is promising, we decide for keeping this

However, in any case I do not think that we need to have travis changed or any of the test systems. The reason being that not many users will use the MKL and hence this is not of such a major concern. Nonetheless, I will run once in a while tests on our MKL installation. Does that make sense?

Sebastian

On Monday, August 22, 2016 at 5:42:14 PM UTC+2, Bob Carpenter wrote:
> > On Aug 22, 2016, at 5:23 PM, Daniel Lee
> >

[Ben Goodrich]

On Monday, August 22, 2016 at 7:50:49 AM UTC-4, Sebastian Weber wrote:

Yes, this is a good topic for the meetings. Let me clarify that two things are involved here:

- MKL
- Intel compiler

The MKL can come independent of the Intel compiler while the Intel compiler usually brings with it the MKL. As far as I am aware neither are free, only if you use it for academic only purposes there are some possibilities to use them free of charge.

I don't think there would be (m)any users of MKL without the Intel compiler or users of the Intel compiler without MKL, in which case we can just infer from CC=icc that we should link Eigen to the MKL.

Ben

[Michael Betancourt]

To add a sour voice — this sounds like a huge amount of work

and maintenance with very little benefit for our general user base.

Not to mention potential issues with the interfaces and testing

going forward (none of the CI services support Intel, right?).

It would be one thing if it requires a bunch of work and everyone

benefitted, but these kinds of microoptmizations seems like poor

use of our _very_ limited person-power at the moment.  Might be

viable from a consulting perspective, but I’m not sure I’d support

it as is.

[Avraham Adler]

Non-programmer here, but what isn't the MKL just a highly tuned BLAS/LAPACK combination? If Stan is already calling the local BLAS & LAPACK the speedup should happen.

Possibly the reason why you don't see as much is that Stan made the decision to use Eigen, which specifically does not use the system BLAS but for philosophical reasons builds its own routines. See these two messages by the authors of Armadillo (which uses BLAS/LAPACK) and RcppEigen on the R-devel list from 2012:

* http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2012-June/003899.html
* http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2012-June/003909.html

So in Stan's case, unless Eigen has greatly changed from 2012, the benefit of using the MKL (or OpenBLAS or ATLAS) will be restricted to the non-Eigen sections of the code, I believe.

Thanks,

Avi

[Bob Carpenter]

Stan does not call BLAS/LAPACK. It uses Eigen, the C++
library.

Those philosophical reasons are what let us do a bunch
of the autodiff we're doing now and let us get Stan off
the ground in the first place! Having a pure C++ solution
is also much easier from a build and installation perspective.

I can't even enumerate how many times systems fail for me
because they can't find some Fortran-related library related
to BLAS/LAPACK.

- Bob

[Bob Carpenter]

I thought we were just talking about some conditional builds
here and that Sebastian was volunteering to do the work. If
not, then I agree with Michael.

- Bob

[Avraham Adler]

It seems Eigen has changed since I last looked and, given the MKL, "substitutes" some of its routines for MKL versions. See https://eigen.tuxfamily.org/dox/TopicUsingIntelMKL.html

Again, I am neither a programmer nor do I play one on the radio, but if some of that C++ ability allows Stan to get out of the gate, and behind the scenes something is passed to the MKL, it is remotely possible that mayhem may ensue.

That's why YOUZ GUYZ get the big bucks 8-)

Avi

[Sebastian Weber]

Correct. Eigen replaces some of its calls with MKL optimized routines. The same holds true for CVODES if you compile it correctly.

In summary, we are talking about conditional builds which I have already done, will anyway maintain and are suggesting to add to Stan.

@Ben: Yes, wherever the MKL is around, the Intel compiler will be close and vice versa. However, from my experience with Stan and R I would NOT setup a production system to build with Intel anymore. If you make as a company the decision to go with open-soruce, then you should actually use the compilers of that community. This means the optimal system from my perspective is built on the Rtools compiler (gcc 4.9 as of now) and then you simply link in the MKL. That would have saved me (and other here, sorry) a lot of trouble.

Cheers,
Sebastian

[Avraham Adler]

On Monday, August 22, 2016 at 3:00:52 PM UTC-4, Sebastian Weber wrote:
However, from my experience with Stan and R I would NOT setup a production system to build with Intel anymore. If you make as a company the decision to go with open-soruce, then you should actually use the compilers of that community. This means the optimal system from my perspective is built on the Rtools compiler (gcc 4.9 as of now) and then you simply link in the MKL. That would have saved me (and other here, sorry) a lot of trouble.
>
> Cheers,
> Sebastian

Hmmm. Eigen is not set up to recognize an open-source optimized BLAS like OpenBLAS, is it?

Thanks,

Avi

[Bob Carpenter]

> On Aug 22, 2016, at 8:34 PM, Avraham Adler <avraha...@gmail.com> wrote:
>
> It seems Eigen has changed since I last looked and, given the MKL, "substitutes" some of its routines for MKL versions. See https://eigen.tuxfamily.org/dox/TopicUsingIntelMKL.html

They added this feature a couple years after we started using Eigen for
Stan and I didn't even notice. They do qualify that it now only works for
float, double, complex<float>, and complex<double>.

Now that we're using more matrix derivatives and fewer algorithms passing
our autodiff variables directly to Eigen, the more that might help.

> Again, I am neither a programmer nor do I play one on the radio, but if some of that C++ ability allows Stan to get out of the gate, and behind the scenes something is passed to the MKL, it is remotely possible that mayhem may ensue.

I trust the Eigen devs have done things properly. Usually things
break at compile time rather than runtime.

> That's why YOUZ GUYZ get the big bucks 8-)

Yup, about one-third market rates. Isn't academia wonderful?

- Bob

[Bob Carpenter]

[back on stan-dev]

If we offer an option to build with Intel MKL and users
try it and it doesn't work, they'll send us questions and we
don't want to ignore users questions. In this case, we'd just
direct them all to you. This is what happens with various
R configs, MSVC compilers, edge-case g++ compilers on linux,
etc.

Then for this to be useful, there needs to be instructions on
how to build for CmdStan, PyStan, RStan, etc.

Do you think we also need conditional tests or will all the
unit tests for the math lib, Stan, and interfaces pass as is?

Or even worse, will the code need to be hacked to make it work?

- Bob

[Sebastian Weber]

To get the benefit for Eigen we only need to define a few macros during the build. The story with CVODES needs 3 lines of code being changed (the mkl backends need to be initialized); that's it - at least that's the theory, I am going to find out that bit on CVODES.

I hope I need not to sign any contract... but I will certainly be responsible for this stuff.

Sebastian

[Bob Carpenter]

Nobody's signing any contracts for support :-) I just meant
that if you don't do it, it's unlikely anyone else will, so we'd
just remove the feature.