[SciPy-user] Why is SciPy better than Matlab or IDL? ;)
Vincent van Beveren
Hello everyone,
I’m an engineer at Rijnhuizen, which is a research institute for plasma physics. We use Python to drive one of our main research projects, however, in the scientific circles in this institute Python (and SciPy) are largely unknown. Time to change this, I think J. However, since I am an engineer and not a scientist I my viewpoint on Python and SciPy are more of an engineering perspective, like its Open Source, Free, modern language, functional programming, etc... I’m not entirely sure these are compelling arguments for a scientist to start working with Python (or atleast not on it self). So I was wondering, if I was to promote Python in the scientific community here at Rijnhuizen. So I have a few questions:
- In what aspects does SciPy excel, compared to say IDL or matlab?
- In what ways allows it a scientist to be more effective?
- How usable is SciPy for Plasma physics, molucular dynamics and nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
- How stable is it compared to other packages. (bugs, computation)?
Any comments and insights are welcome!
Regards,
Vincent
PS. Posted in scipy-dev by accident.
___
Ing. V. van Beveren
Software Engineer, FOM Rijnhuizen
ms
> - In what aspects does SciPy excel, compared to say IDL or matlab?
I would say that its main strength is being a library of a
full-featured, well-supported, cross-platform programming language. This
means you can build arbitrarily complex applications using Python and
Scipy, while it would be very painful with Matlab.
> - In what ways allows it a scientist to be more effective?
See above.
> - How usable is SciPy for Plasma physics, molucular dynamics and nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
I don't think one could write molecular dynamics engines in python, it's
anyway too slow (I hope someone can tell me I'm wrong!)
But it can be very useful for all kinds of data analysis and
not-enormously-heavy computations (that's what I use it for).
m.
_______________________________________________
SciPy-user mailing list
SciPy...@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Gökhan SEVER
_______________________________________________
SciPy-user mailing list
SciPy...@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user
Vincent,
Here a couple of sources that has inspired me to choose Python for my scientific data analysis and visualization tasks:
http://conference.scipy.org/proceedings/SciPy2008/
https://cirl.berkeley.edu/fperez/
A Demonstration of the 'IPython' Interactive Shell
Scientific Computing with Python
Matplotlib by John D. Hunter (watched on April 12, 2009)
''Sage: Unifying Mathematical Software''', video part 1, video part 2
--
Gökhan
Hoyt Koepke
> - In what aspects does SciPy excel, compared to say IDL or matlab?
>
> - In what ways allows it a scientist to be more effective?
A while ago, I wrote up a quick article on a few reasons that I think
python is great for research stuff. It's a bit unpolished, but it
gives 10 points that I think are really advantageous. It's at
http://www.stat.washington.edu/~hoytak/blog/whypython.html.
Let me know if anyone has feedback or comments on it; it's a work in
progress at this point.
--Hoyt
++++++++++++++++++++++++++++++++++++++++++++++++
+ Hoyt Koepke
+ University of Washington Department of Statistics
+ http://www.stat.washington.edu/~hoytak/
+ hoy...@gmail.com
++++++++++++++++++++++++++++++++++++++++++
David Warde-Farley
> - In what aspects does SciPy excel, compared to say IDL or
> matlab?
Vectorized operations are (almost) as fast. Depending on what BLAS you
use, linear algebra can be just as fast. String processing in Python
is so incredibly better than Matlab that the thought of doing string
operations in Matlab almost makes me cry. Python has bindings to
nearly everything -- a dozen different GUI toolkits to low-level
hardware interfaces. Likewise there are a handful of different ways
to interface your code to code written in other languages -- ctypes,
swig, boost::python, cython for C/C++, PyObjC for Objective C, f2py
for Fortran, scikits.mlabwrap for Matlab, rpy and rpy2 for R...
> - In what ways allows it a scientist to be more effective?
As has been echoed here, writing large programs will be easier. Python
is a real programming language and was designed that way from the
ground up. Matlab was designed as an interactive environment first and
programming language features were grafted on as an afterthought.
> - How stable is it compared to other packages. (bugs,
> computation)?
I won't lie in that I occasionally run into bugs in SciPy, but one of
three things usually happens:
- I look at the code myself -- all of which is available to view and
modify, not the case with Matlab -- and file a bug report and/or patch
- I look at the code and realize it's not a bug but me
misunderstanding something, at which point I might edit the
documentation at http://docs.scipy.org/scipy/ to make it clearer if
necessary.
- If the code is incomprehensible to me, I ask on the mailing lists
and (either already or shortly thereafter) it's fixed by someone else
in Subversion.
Bruce Southey
On 7-Jul-09, at 10:03 AM, Vincent van Beveren wrote:- In what aspects does SciPy excel, compared to say IDL or matlab?Vectorized operations are (almost) as fast. Depending on what BLAS you use, linear algebra can be just as fast. String processing in Python is so incredibly better than Matlab that the thought of doing string operations in Matlab almost makes me cry. Python has bindings to nearly everything -- a dozen different GUI toolkits to low-level hardware interfaces. Likewise there are a handful of different ways to interface your code to code written in other languages -- ctypes, swig, boost::python, cython for C/C++, PyObjC for Objective C, f2py for Fortran, scikits.mlabwrap for Matlab, rpy and rpy2 for R...
As a shameless plug, Frontiers in Neuroinformatics had a special issue on Python:
http://www.frontiersin.org/neuroinformatics/
http://www.frontiersin.org/neuroinformatics/specialtopics/8/
Bruce
Robin
Beveren<V.vanB...@rijnhuizen.nl> wrote:
> - In what aspects does SciPy excel, compared to say IDL or matlab?
>
> - In what ways allows it a scientist to be more effective?
>
> - How usable is SciPy for Plasma physics, molucular dynamics and
> nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
>
> - How stable is it compared to other packages. (bugs, computation)?
I think it's probably been covered by others but some of the key
things for me are:
- how easy it is to extend performance critical portions in C or
FORTRAN (cython, ctypes, f2py, weave etc.)
http://www.scipy.org/PerformancePython may be slightly out of date but
I think it's a great illustration.
- how easy it is (relatively speaking) to employ parralel processing.
http://www.scipy.org/ParallelProgramming . Using the multiprocessing
module (available as a backport for 2.5) it is literally 2 lines to
have a for loop parallellised over 8 cores (or 16 if I had them). In
MATLAB anything more than 4 cores gets into really silly money (not
usually included in site license). This is probably the major factor
driving interest in Python from colleagues in my lab. Also of course
there are parralel python, ipython etc. and python bindings to MPI,
cuda, opencl etc. which I think would usually allow much quicker
progress than working in C.
- just how nice it is to use; broadcasting... even just how nice it is
to handle default arguments in functions and update code with new
parameters in a backwards compatible was was a bit of a revelation
for me after matlab.
- slices as views - in my work I usually have one long data set which
I slice and view in different ways. In matlab I think slices make
copies (at least they used to - possibly its changed now, but another
thing is that you can never really know what matlabs going to do,
whereas in Python you really can control how memory is laid out and
manipulated at a low level) but in Python all my different views point
to the same data.
cheers
Robin
David Warde-Farley
On 7-Jul-09, at 12:39 PM, Bruce Southey wrote:
> As an example, you can very easily 'bolt on' web stuff to an
> existing peice of Python code to create an almost instant web
> application. (I have mixed linear algebra with html processing
> including parsing cgi and creating html tags and images to provide a
> couple of web apps.) So suddenly instead of saying I can do solve
> this problem (and perhaps here is the code), you can just say 'go to
> this web site and do it'. Plus, I can easily fix bugs and add new
> code without the user doing anything.
Indeed, I've done this too. Getting matplotlib to render PNGs for
TurboGears/Django/your-favourite-web-framework is dead simple, too so
you can provide really quite fancy visualization in your web-based
interface without having to do anything extra.
David
Tim Michelsen
David Warde-Farley <dwf <at> cs.toronto.edu> writes:
> On 7-Jul-09, at 12:39 PM, Bruce Southey wrote:
> > As an example, you can very easily 'bolt on' web stuff to an
> > existing peice of Python code to create an almost instant web
> > application.
[...]
> Indeed, I've done this too. Getting matplotlib to render PNGs for
> TurboGears/Django/your-favourite-web-framework is dead simple, too so
> you can provide really quite fancy visualization in your web-based
> interface without having to do anything extra.
@Bruce, David
Do you have a particular site that is one-line and
available to the gereral public?
I am intending something similar.
Although it is not specially topic of this list, I am particulary interested in
how you generate the matplotlib graphics on the fly.
To you store them in a temporary directory or do create
the files somehow on the fly?
Would be nice to see your implementation.
Thanks,
Timmie
David Warde-Farley
> Would be nice to see your implementation.
The way I did it was to do the plotting onto a FigureCanvasAgg and
then printing it to a fake file using cStringIO.
from matplotlib.backends.backend_agg import FigureCanvasAgg
import matplotlib.pyplot as plt
import cStringIO
def plot(...):
fig = plt.figure(...) # figure out size and such here
canvas = FigureCanvasAgg(fig)
fig.set_canvas(canvas) # I forget whether this is needed
# Do some plotting commands here
...
...
canvas.draw()
imgdata = cStringIO.StringIO()
canvas.print_png(imgdata)
return imgdata.get_value()
There's something very similar in the Matplotlib cookbook or the
TurboGears cookbook, but I forget which.
David
David Warde-Farley
> @Bruce, David
> Do you have a particular site that is one-line and
> available to the gereral public?
> I am intending something similar.
http://www.zunzun.com uses matplotlib extensively.
My own frankenstein creation has been superseded by a J2EE version
designed by a whole team of programmers, unfortunately. :)
David
Ken Dere
> Hello everyone,
>
> I'm an engineer at Rijnhuizen, which is a research institute for plasma
> physics. We use Python to drive one of our main research projects,
> however, in the scientific circles in this institute Python (and SciPy)
> are largely unknown. Time to change this, I think :). However, since I am
> an engineer and not a scientist I my viewpoint on Python and SciPy are
> more of an engineering perspective, like its Open Source, Free, modern
> language, functional programming, etc... I'm not entirely sure these are
> compelling arguments for a scientist to start working with Python (or
> atleast not on it self). So I was wondering, if I was to promote Python in
> the scientific community here at Rijnhuizen. So I have a few questions:
>
> - In what aspects does SciPy excel, compared to say IDL or
> matlab?
> - In what ways allows it a scientist to be more effective?
> - How usable is SciPy for Plasma physics, molucular dynamics and
> nanolayer Surface and Interface Physics (the 3 main areas at Rijnhuizen)?
> - How stable is it compared to other packages. (bugs,
> computation)?
>
> Any comments and insights are welcome!
>
> Regards,
> Vincent
>
> PS. Posted in scipy-dev by accident.
> ___
> Ing. V. van Beveren
> Software Engineer, FOM Rijnhuizen
> E: V.vanB...@rijnhuizen.nl<mailto:V.vanB...@rijnhuizen.nl>
The real question is "why is Python better than Matlab or IDL?"
I don't have an opinion on Matlab since I never used it. However, I used
IDL professionally for about 20 years or so (just for reference, I am a
solar physicist).
A few years ago, having gotten some proficiency in Python, I pretty much
dropped my usage of IDL and now use Python. This means python, ipython,
numpy, scipy, matplotlib and whatever other modules I need. One basic
difference is that I can use it at home, on the road and at work without
need of an expensive license. Also, I enjoy using it.
I have seem some object-oriented implementations is IDL and they are
pathetic. It may just be the way it was done but I don't know. The use of
objects in Python is very straightforward.
Ken Dere
--
K. Dere
Vincent van Beveren
Thanks everyone for your comments. Because of the many responses I can’t reply to everyone personally, but thank you for your input. I have summarised your comments into a list, if you have any corrections, or additions, I would like to hear:
- Completely Free: No ridiculous licenses or restrictions
- Open source:
o If there is a bug you can fix it yourself, or ask the community
o You can always go back to the source
- Stable:
o Originally written by commercial company, before released opensource,
o The base of EPD (Enthought Python Distribution)
- Large user community:
o well supported
o actively developed
- Easy to build arbitrary complex applications (IDL and Matlab make this more difficult)
- As a language it:
o has many modules out of the box
o is very readable, and modular
o easy parallel programming
o well featured:
§ has support for many data structures (Lists, Tuples, Sets, Dicts)
§ Object Orientated (OO in IDL is a pain, Matlab: what objects?)
§ has compact array manipulation notation, functional programming and support for complex numbers
- Cross-platform (So is IDL? I don’t know about Matlab)
- Integration with many different things into a unified interface
o has many additional modules which can be installed
§ HDF5, Numpy, SciPy, Matplotlib (2D plotting), VTK (3D graphics)
o Easy to interface with other languages (Fortran, C, R)
- Complete applications for numerical computation build on SciPy freely available:
o SAGE
o Python XY
Thanks so far, you all have been really helpful.
Regards,
Vincent
David Cournapeau
>
> Thanks everyone for your comments. Because of the many responses I
> can’t reply to everyone personally, but thank you for your input. I
> have summarised your comments into a list, if you have any
> corrections, or additions, I would like to hear:
>
>
>
> - Completely Free: No ridiculous licenses or restrictions
>
> - Open source:
>
> o If there is a bug you can fix it yourself, or ask the community
>
> o You can always go back to the source
>
> - Stable:
>
> o Originally written by commercial company, before released
> opensource,
>
It depends on which parts of the stack you are talking about. AFAIK, the
original design for numpy was done ~15 years by Jim Hugunin, and scipy
was born as an open source project from the start. The "official"
history can be found here for scipy:
http://www.scipy.org/History_of_SciPy
David
Brian Blais
Thanks everyone for your comments. Because of the many responses I can’t reply to everyone personally, but thank you for your input. I have summarised your comments into a list, if you have any corrections, or additions, I would like to hear:
- Completely Free: No ridiculous licenses or restrictions
- Open source:
o If there is a bug you can fix it yourself, or ask the community
o You can always go back to the source
- Stable:
o Originally written by commercial company, before released opensource,
o The base of EPD (Enthought Python Distribution)
- Large user community:
o well supported
o actively developed
- Easy to build arbitrary complex applications (IDL and Matlab make this more difficult)
- As a language it:
o has many modules out of the box
o is very readable, and modular
o easy parallel programming
o well featured:
§ has support for many data structures (Lists, Tuples, Sets, Dicts)
§ Object Orientated (OO in IDL is a pain, Matlab: what objects?)
§ has compact array manipulation notation, functional programming and support for complex numbers
- Cross-platform (So is IDL? I don’t know about Matlab)
- Integration with many different things into a unified interface
o has many additional modules which can be installed
§ HDF5, Numpy, SciPy, Matplotlib (2D plotting), VTK (3D graphics)
o Easy to interface with other languages (Fortran, C, R)
- Complete applications for numerical computation build on SciPy freely available:
o SAGE
o Python XY
John Hunter
> On 8-Jul-09, at 7:48 AM, Tim Michelsen wrote:
>
>> Would be nice to see your implementation.
>
> The way I did it was to do the plotting onto a FigureCanvasAgg and
> then printing it to a fake file using cStringIO.
>
> from matplotlib.backends.backend_agg import FigureCanvasAgg
> import matplotlib.pyplot as plt
> import
Apologies since this is OT for scipy, so please followup to
matplolib-users if you have further questions or comments, but there
is a slight problem with the example above. If you are using mpl in a
web server, and want explicit control of the canvas, you can either
use pyplot and the canvas like so::
import matplotlib.use('Agg')
import matplotlib.pyplot as plt
fig = plt.figure()
fig.canvas # use thie canvas (it is FIgureCanvasAgg) rather than
creating your own
Or you can use the API and manually create your own figure and canvas
(this is recommended since you have maximum control over object
management)::
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure
fig = Figure()
canvas = FigureCanvas(fig)
ax = fig.add_subplot(111)
The code you posted is a hybrid of using pyplot and managing your own
Canvas creation, which can work under certain configurations but is
brittle and not supported. This is mainly because pyplot creates and
manages its own canvas instances.
JDH
David Warde-Farley
On 11-Jul-09, at 7:41 AM, John Hunter wrote:
> The code you posted is a hybrid of using pyplot and managing your own
> Canvas creation, which can work under certain configurations but is
> brittle and not supported. This is mainly because pyplot creates and
> manages its own canvas instances.
Oops. I suppose it was an instance where I fiddled until it worked and
never gave it a second look. Thanks for the clarification John.
David