Hi, I am using C++ for numerical computing. However, after having compared with Matlab, I found some algorithms(like multiplication of matrices) implemented with C++ are not faster then the same one in Matlab. I counted the time spending on the both algorithm and result that
Matlab C++ multipication of matrices(500x500) 1.4s 5s(not including building the matrix)
It's disappointing that C++ is much slower than a intepreted languages (like Matlab). If so, why so many people devote them in developing a numerical library in C/C++/Fortran? Why don't they use Matlab instead?
Rex_chaos wrote: > Hi, > I am using C++ for numerical computing. However, after having > compared with Matlab, I found some algorithms(like multiplication of > matrices) implemented with C++ are not faster then the same one in > Matlab. I counted the time spending on the both algorithm and result > that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s(not including > building the matrix)
> It's disappointing that C++ is much slower than a intepreted languages > (like Matlab). If so, why so many people devote them in developing a > numerical library in C/C++/Fortran? Why don't they use Matlab instead?
There's probably more than one reason. But note that the overhead of the interpreter is probably negligible when compared to the time required to perform the matrix multiplication. It is possible in either language to write code which runs as fast as the processor is capable -- so my conclusion is that there *is* a need to develop a (fast) numerical library in C/C++, not that no one should write one.
-- There are two things you must never attempt to prove: the impossible -- and the obvious. http://www.crbond.com
rex_ch...@21cn.com (Rex_chaos) writes: > Hi, > I am using C++ for numerical computing. However, after having > compared with Matlab, I found some algorithms(like multiplication of > matrices) implemented with C++ are not faster then the same one in > Matlab. I counted the time spending on the both algorithm and result > that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s(not including > building the matrix)
> It's disappointing that C++ is much slower than a intepreted languages > (like Matlab).
the interpreter merely parses "A*B" and then calls a compiled subroutine to perform the multiplication. matlab is written in C.
you might want to check out ATLAS. it is an optimizing BLAS (basic linear algrebra). modern processors can be finicky beasts due to the cache layout and instruction scheduling and BLAS tries a number of feeding patterns to get high performance for particular CPUs.
> If so, why so many people devote them in developing a > numerical library in C/C++/Fortran? Why don't they use Matlab > instead?
> I am using C++ for numerical computing. However, after having > compared with Matlab, I found some algorithms(like multiplication of > matrices) implemented with C++ are not faster then the same one in > Matlab.
Matlab is slightly faster for basic operations like multiplication.
> I counted the time spending on the both algorithm and result > that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s(not including > building the matrix)
I am surprised by the difference. Could you send me your C++ code? I think your C++ code should be a lot faster.
> It's disappointing that C++ is much slower than a intepreted languages > (like Matlab). If so, why so many people devote them in developing a > numerical library in C/C++/Fortran? Why don't they use Matlab instead?
Matlab simple operations are efficient because they are optimized. If you write basic operations in C++ by yourself, you will never get the same performances. Matlab matrix/vector multiplication was probably written in assembler or, at least, with advanced considerations on memory operations. That is why it is faster.
Nevertheless, you can reach the same performances providing you use the right routines. If you have an optimized matrix/vector product (as in BLAS), you will reach the same performances. That's the reason why using C++ is not a problem. But, you should use optimized routines and, for example, use C++ to manage memory and higher level calculations. C++ libraries often provides a convenient interface and use BLAS/LAPACK (for example) in order to be efficient.
Finally, Matlab turns out to be very slow when you use high level routines. Matrices manipulations are convenient but not as fast as what you could do in C++. For professional projects, you cannot imagine working with Matlab. Many interesting routines are written in Fortran, C or C++. Creating a project, adding components is quite convenient in C++. And, Matlab program are interpreted which is of course slow (and you lose memory). As for I/O operations, Matlab is not as convenient as C++. And so on and so forth...
There are many other reasons why one should use C++ (or C or Fortran 77/90) instead of Matlab (slow, licences, ...).
Notice that you can compile your Matlab functions. And, to be convinced that Matlab is slow, you can compile some of your Matlab functions... [ If you compile Matlab functions, do not use notations like M(5:10) = 10, use loops instead (even if it is slower while running under Matlab...)! ]
> > Hi, > > I am using C++ for numerical computing. However, after having > > compared with Matlab, I found some algorithms(like multiplication of > > matrices) implemented with C++ are not faster then the same one in > > Matlab. I counted the time spending on the both algorithm and result > > that
> > Matlab C++ > > multipication of matrices(500x500) 1.4s 5s(not including > > building the matrix)
> > It's disappointing that C++ is much slower than a intepreted languages > > (like Matlab).
> the interpreter merely parses "A*B" and then calls a compiled > subroutine to perform the multiplication. matlab is written in C.
> you might want to check out ATLAS. it is an optimizing BLAS (basic > linear algrebra). modern processors can be finicky beasts due to the > cache layout and instruction scheduling and BLAS tries a number of > feeding patterns to get high performance for particular CPUs.
rex_ch...@21cn.com (Rex_chaos) wrote in message <news:f7a7417.0206100230.39bbb6cd@posting.google.com>... > Hi, > I am using C++ for numerical computing. However, after having > compared with Matlab, I found some algorithms(like multiplication of > matrices) implemented with C++ are not faster then the same one in > Matlab. I counted the time spending on the both algorithm and result > that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s(not including > building the matrix)
> It's disappointing that C++ is much slower than a intepreted languages > (like Matlab). If so, why so many people devote them in developing a > numerical library in C/C++/Fortran? Why don't they use Matlab instead?
While I can't speak much about Matlab, it seems C++ is always a difficult language to use when performance is an issue. For the case of matrix multiplication on Intel processors, it is very likely that Matlab uses hardware features such as MMX, SSE, or SSE2 to speed up the calculation while your C++ matrix libraries do not. For Motorola PowerPC processors the hardware to look out for is the Altivec unit.
Check out this link for some ideas on improving performance on your own:
Rex_chaos wrote: > I am using C++ for numerical computing. > However, after having compared with Matlab, > I found some algorithms(like multiplication of matrices) > implemented with C++ are not faster then the same one in Matlab. > I counted the time spending on the both algorithm and result that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s
> It's disappointing that C++ are much slower > than a interpreted languages (like Matlab).
Matrix-matrix multiplication is NOT interpreted in Matlab. It is an atomic operation. I believe that Matlab is still using an optimized version of the BLAS library subroutine dgemm. Install ATLAS and call dgemm from it. You should get precisely the same performance that you get from Matlab. The ATLAS version of dgemm is implemented in C and is as fast or faster than most implementations in Fortran 77.
The problem is not that C++ is slow but that you are not a very good numerical programmer yet. It takes a lot of time and experience. Be patient.
Is C++ a best choice for numerical computing?
It is certainly as good as any other choice for numerical computing.
> Hi, > I am using C++ for numerical computing. However, after having > compared with Matlab, I found some algorithms(like multiplication of > matrices) implemented with C++ are not faster then the same one in > Matlab. I counted the time spending on the both algorithm and result > that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s(not including > building the matrix)
> It's disappointing that C++ is much slower than a intepreted languages > (like Matlab). If so, why so many people devote them in developing a > numerical library in C/C++/Fortran? Why don't they use Matlab instead?
> > > Hi, > > > I am using C++ for numerical computing. However, after having > > > compared with Matlab, I found some algorithms(like multiplication of > > > matrices) implemented with C++ are not faster then the same one in > > > Matlab. I counted the time spending on the both algorithm and result > > > that
> > > Matlab C++ > > > multipication of matrices(500x500) 1.4s 5s(not including > > > building the matrix)
> > > It's disappointing that C++ is much slower than a intepreted languages > > > (like Matlab).
> > the interpreter merely parses "A*B" and then calls a compiled > > subroutine to perform the multiplication. matlab is written in C.
> > you might want to check out ATLAS. it is an optimizing BLAS (basic > > linear algrebra). modern processors can be finicky beasts due to the > > cache layout and instruction scheduling and BLAS tries a number of > > feeding patterns to get high performance for particular CPUs.
> Also note that modern matlab uses ATLAS.
> Jack Walker
It's been nearly ten years, but I once tried to match the MSDOS Matlab matrix multiply speed by tweaking the code for a competing product produced by the software company I worked for at the time. In pure C code, using pointers in somewhat skillful fashion, I could do no better than a factor of 6 slower than Matlab. Using inline assembler in the innermost loop of the multiply, I could only get to a factor of 3. This was back in an era when Borland's C++ compiler was king, and Microsoft C was essentially unusable for serious production. Things have changed a lot, but I believe that Intel makes a hand-coded assembler BLAS that is impossible to match with any high level language compiler.
I agree with Johan and others who point out that the "interpreted" aspects of Matlab are beside the point here, as the basic linear algebra subroutine is consuming the major CPU cycles.
Matlab can be used to develop applications ("toolboxes") of a low level of aesthetic sensibility, but of course using the toolbox requires a Matlab license. So perhaps the most common reason to do an application in C++ is to make it freely redistributable.
>>Hi, >> I am using C++ for numerical computing. However, after having >>compared with Matlab, I found some algorithms(like multiplication of >>matrices) implemented with C++ are not faster then the same one in >>Matlab. I counted the time spending on the both algorithm and result >>that
>> Matlab C++ >> multipication of matrices(500x500) 1.4s 5s(not including >>building the matrix)
>>It's disappointing that C++ is much slower than a intepreted languages >>(like Matlab). If so, why so many people devote them in developing a >>numerical library in C/C++/Fortran? Why don't they use Matlab instead?
> Hi, > I am using C++ for numerical computing. However, after having > compared with Matlab, I found some algorithms(like multiplication of > matrices) implemented with C++ are not faster then the same one in > Matlab. I counted the time spending on the both algorithm and result > that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s(not including > building the matrix)
> It's disappointing that C++ is much slower than a intepreted languages > (like Matlab). If so, why so many people devote them in developing a > numerical library in C/C++/Fortran? Why don't they use Matlab instead?
Hi,
As the others say, Matlab doesn't really do the mult in Matlab language. It calls up an very well optimized subroutine for that. That's the case for every Matlab built-in function
If you really want to compare, try writing your own matrix mult routine in pure scalar Matlab (no vectorization), with 'for' loops, etc.
You will most probably find out that Matlab is _way_ slower than C for that type of low level programming ! Careful Matlab programmers use built-in functions and vectorization whenever possible, to speed up the execution and make up for Matlab's slowness. The drawback of vectorization is that the code may be less clear to read.
Daniel Jaeggi <jae...@spamfilter.embl-heidelberg.de> wrote: > I'll completely second that - for matrix work, F90 is fast, easy and > elegant.
Except that I've seen too many examples where the compiler makes dozens of needless copies every time you do array sections. And gone is your performance.
>> I'll completely second that - for matrix work, F90 is fast, easy and >> elegant.
>Except that I've seen too many examples where the compiler makes dozens >of needless copies every time you do array sections. And gone is your >performance.
>V.
If you are doing non-continuous sections... that's pretty much what you should expect. If your data is continuous ( and not addressed with s l o w pointer-ridden code ) you shouldn't have that problem unless you have a crappy compiler.
If unsure... don't use array sections, especially if you want fast runtime codes. After all... what other language can do efficient array sections??
>Date: 6/10/02 3:30 AM Pacific Daylight Time >Message-id: <f7a7417.0206100230.39bbb...@posting.google.com>
>Hi, > I am using C++ for numerical computing. However, after having >compared with Matlab, I found some algorithms(like multiplication of >matrices) implemented with C++ are not faster then the same one in >Matlab. I counted the time spending on the both algorithm and result >that
> Matlab C++ > multipication of matrices(500x500) 1.4s 5s(not including >building the matrix)
>It's disappointing that C++ is much slower than a intepreted languages >(like Matlab). If so, why so many people devote them in developing a >numerical library in C/C++/Fortran? Why don't they use Matlab instead?
Out of curiousity, I just wrote a fortran program to do your described multiply. I used the Lahey F95 compiler with no special optimizations set and took special steps to be sure that the compiler would not "optimize out" of the calculations. I also used the Standard Matrix Multiplication function of fortran rather than attempt to write my own loops. Using 4 byte Floating point data, it took between 1.3 and 1.37 seconds per run. I also did it using 8 byte reals. It consistently took only 1.7 seconds to run.
These test were on a Windows98 computer with an Athlon 750 cpu, running several applications in the background, including AOL, Explorer, DOS and a mid-sized editor.
> >Except that I've seen too many examples where the compiler makes dozens > >of needless copies every time you do array sections. And gone is your > >performance.
> >V.
> If you are doing non-continuous sections... that's pretty much what you > should expect.
No, if A(5,5) and you take A(2:3,2:3) no copy should be necessary, but it usually is still taken. That array section could be described internally in the time-honoured Lapack way with M=2, LDA=5.
> No, if A(5,5) and you take A(2:3,2:3) no copy should be necessary, but > it usually is still taken. That array section could be described > internally in the time-honoured Lapack way with M=2, LDA=5.
> V.
Fair points, but then any language has features with which you can clobber your performance. But in my experience, there are far fewer of these (when talking about numerical computation) in Fortran.
Without turning this into a fortran discussion, I take it your solution to avoiding sections is to write as you would in f77 when passing arrays?
Back to the original point, you really can't get much simpler than C = matmul(A,B)
Daniel Jaeggi <jae...@spamfilter.embl-heidelberg.de> wrote: > Without turning this into a fortran discussion, I take it your solution > to avoiding sections is to write as you would in f77 when passing arrays?
rex_ch...@21cn.com (Rex_chaos) wrote: > I am using C++ for numerical computing. However, after having > compared with Matlab, I found some algorithms(like multiplication of > matrices) implemented with C++ are not faster then the same one in > Matlab. [...] > It's disappointing that C++ is much slower than a intepreted languages > (like Matlab). If so, why so many people devote them in developing a > numerical library in C/C++/Fortran? Why don't they use Matlab instead?
As others have pointed out, Matlab executes vectorized expressions using highly optimized, special purpose subroutines. That's why it is much faster than a naive algorithm written in C++.
Matlab scripts are also much easier to comprehend than equivalent programs in C++. You need not worry about memory allocation, declarations, and other such baggage. Also, writing a vectorized expression is much clearer that an equivalent for-loop in C++.
The only down side is that Matlab is pretty expensive. I recommend Octave, an open source implementation of the Matlab script language, very highly. Octave, like Matlab, uses BLAS to achieve high performance. See http://www.octave.org/ for more info.
I have written a variety of number crunching programs in a variety of languages, and I strongly recommend that you start your work in an appropriate high-level language such as Matlab/Octave, S+/R, IDL, or what have you. Maybe later on you will decide to rewrite some or all the code in a C/C++, Fortran, Java, etc. Or you might never need to do so.
For what it's worth,
Robert Dodier -- ``He wins most who toys with the dies.'' -- David O'Bedlam
>>>>> "Robert" == Robert Dodier <robert_dod...@yahoo.com> writes:
Robert> declarations, and other such baggage. Also, writing a vectorized Robert> expression is much clearer that an equivalent for-loop in C++.
This goes both ways. Because matlab is intepreted (it was when I last used it, maybe it's different now?), loops don't run so fast, so you go out of your way to operate on matrices, slicing an array this way, reshaping it, slicing it another, etc. The equivalent C/C++ code would have been a simple loop.
The fact that matrices/vectors in Matlab are 1-based indices sometimes also makes things less clear when the algorithm descriptions use 0-based indices.
> From: rex_ch...@21cn.com (Rex_chaos) > >Date: 6/10/02 3:30 AM Pacific Daylight Time > >Message-id: <f7a7417.0206100230.39bbb...@posting.google.com>
> >Hi, > > I am using C++ for numerical computing. However, after having > >compared with Matlab, I found some algorithms(like multiplication of > >matrices) implemented with C++ are not faster then the same one in > >Matlab. I counted the time spending on the both algorithm and result > >that
> > Matlab C++ > > multipication of matrices(500x500) 1.4s 5s(not including > >building the matrix)
> >It's disappointing that C++ is much slower than a intepreted languages > >(like Matlab). If so, why so many people devote them in developing a > >numerical library in C/C++/Fortran? Why don't they use Matlab instead?
> Out of curiousity, I just wrote a fortran program to do your described > multiply. I used the Lahey F95 compiler with no special optimizations set and > took special steps to be sure that the compiler would not "optimize out" of the > calculations. I also used the Standard Matrix Multiplication function of > fortran rather than attempt to write my own loops. Using 4 byte Floating point > data, it took between 1.3 and 1.37 seconds per run. I also did it using 8 > byte reals. It consistently took only 1.7 seconds to run.
> These test were on a Windows98 computer with an Athlon 750 cpu, running several > applications in the background, including AOL, Explorer, DOS and a mid-sized > editor.
> Dan :-)
Can you send me the source code? I have written a program to do the same thing with Fortran 90. However, I found it's as slow as my C++ version.
>From: rex_ch...@21cn.com (Rex_chaos) >Date: 6/18/02 4:28 AM Pacific Daylight Time >Message-id: <f7a7417.0206180328.75aac...@posting.google.com>
>dant...@aol.com (Dan Tex1) wrote in message ><news:20020612235006.07541.00000288@mb-df.aol.com>... >> From: rex_ch...@21cn.com (Rex_chaos) >> >Date: 6/10/02 3:30 AM Pacific Daylight Time >> >Message-id: <f7a7417.0206100230.39bbb...@posting.google.com>
>> >Hi, >> > I am using C++ for numerical computing. However, after having >> >compared with Matlab, I found some algorithms(like multiplication of >> >matrices) implemented with C++ are not faster then the same one in >> >Matlab. I counted the time spending on the both algorithm and result >> >that
>> > Matlab C++ >> > multipication of matrices(500x500) 1.4s 5s(not including >> >building the matrix)
>> >It's disappointing that C++ is much slower than a intepreted languages >> >(like Matlab). If so, why so many people devote them in developing a >> >numerical library in C/C++/Fortran? Why don't they use Matlab instead?
>> Out of curiousity, I just wrote a fortran program to do your described >> multiply. I used the Lahey F95 compiler with no special optimizations set >and >> took special steps to be sure that the compiler would not "optimize out" of >the >> calculations. I also used the Standard Matrix Multiplication function of >> fortran rather than attempt to write my own loops. Using 4 byte Floating >point >> data, it took between 1.3 and 1.37 seconds per run. I also did it using 8 >> byte reals. It consistently took only 1.7 seconds to run.
>> These test were on a Windows98 computer with an Athlon 750 cpu, running >several >> applications in the background, including AOL, Explorer, DOS and a >mid-sized >> editor.
>> Dan :-) >Can you send me the source code? I have written a program to do the >same thing with Fortran 90. However, I found it's as slow as my C++ >version.
The CHECK is in the mail.... email that is ( source and executable ).
