Choosing Perl/Python for my particular niche
Fred Ma
This is not a troll posting, and I've refrained from
asking because I've seen similar threads get all
nitter-nattery. But I really want to make a decision
on how best to invest my time. I'm not interested on
which language is better in *general*, just for my
purpose. My area of research is in CAD algorithms,
and I'm sensing the need to resort to something more
expedient than C++, bash scripting, or sed scripting.
What I hope to do is a bit multifaceted. What I don't
do dabble in is web apps (though I'm not sure of the
similarities). No XML processing or database
interaction in what I do.
One thing I expect to have to do is to modify design
files. For example, there is a tool which takes ASCII
hardware desscription language (HDL) and converts it
to a C++ (augmented by hardware simulation library).
The translator is freeware, so has limitations which I
have to make up for by tweaking the HDL code. In the
past, I've eeked out sed scripts for such tasks, but
would appreciate a more traditional language. Since
Perl is used alot in digital IC design, I took a stab
at that, motivated by the simple need to convert my
mail aliases from one mail reader to the other. It
took a while to do, and I'm concerned at the difficulty
level. If I did this kind of thing constantly, I
would probably get proficient and use the power behind
it, but it's not my main area.
The alternative is Python, which is easier to read
from what I've read. My concern there is that I cut
myself off from large availability of stuff that's
out there. For example, I use a PC-to-solaris
viewer called VNC, and I've banged my head against the
startup script to change it a bit for my situation.
Likewise, the above translator uses Perl extensively
in its operation, as well as it's building and
installation. If I wasn't passingly familiar with
Perl, I would have had a much harder time installing
it. Being in the approximate area digit circuits,
I'm concerned about being on the band wagon, if only
to avoid reinventing things, or impediments to sharing
things.
An additional usage scenario is to (if reasonable)
replace my shell scripting e.g. I just converted to
bash from tcsh to write scripts that push a document
through a series of filters, or simply as a wrapper
around a tough-to-use utility e.g. pstops. A final
example is to take a file of design information and
do more than tweak it e.g. extract all the information
about interconnections between circuit building blocks,
including which is the source block, and which are the
destination blocks. Typically, this information will
be read by matlab scripts and passed to my C++ code
(I've managed to avoid writing code to parse the
input file from C++).
Whichever way I go, I would like to avoid the overhead
of learning both Perl and Python. I will sculpt out some
time in a miserly fashion to slowly get to know one. Since
I spend most of my time exploring the algorithm in Matlab or
C++/STL, there's only so much time to pick up higher level
languages (it took years before I shelled out the time
to switch from tcsh to bash). One of the things that
makes the decision not clear is that despite Python's
claim to a gentler learning curve and clearer code, I
often like the expedience of sed e.g. a terse one-liner
that can be pipelined with other shell commands. I
also note that Perl's unweldiness only comes for big
projects, and I don't expect to using Perl quite that
way.
I've seen mention of parrot and perl6, which is quite a
ways off. I'm not sure how much that should weigh into
my decision, since it isn't real yet.
Thanks for any comments/suggestions.
--
Fred Ma
Dept. of Electronics, Carleton University
1125 Colonel By Drive, Ottawa, Ontario
Canada, K1S 5B6
Fred Ma
I encounter alot of. As in many computer languages,
various kinds of scopes are delineated by things
like braces or begin/end statements. I often find
it valuable to find an occurance of certain things
(e.g. certain data declaration) within certain
scope structures. Sed is convenient for that:
/SomeStartPattern/,/SomeEndPattern/{
Do some stuff,
including conditional branching
}
To avoid the hassle of writing a script file and
filtering the input source file to an output source
file, I often use gvim's awesome pattern matching
and macro recording abilities. A sed script file
can look pretty cryptic, especially when I want
to search across physical lines (lots of conditional
flow to append things to the hold buffer, etc..).
For that reason, I tend to stay with quick/dirty
practices in gvim. Crypticness (though not of the
same variety) is one of the things that I found
hard about ramping up on Perl, to the degree that
I did. As I hinted earlier, a fair comparison would
account for the fact that its power would be more
apparent if I became adequately guru-like, but
realistically, I will not spend enough time with
it on a regular basis to expect that. Here again
is a dichotomy; I appreciate the sed terseness because
I can make a command line pipe stage out of it, but
the same brevity in a scripting language can make for
rough going. Maybe it's wishful thinking to expect
one language to cover all the bases that I initially
mentioned.
Fred
Tassilo v. Parseval
> This is not a troll posting, and I've refrained from
> asking because I've seen similar threads get all
> nitter-nattery. But I really want to make a decision
> on how best to invest my time. I'm not interested on
> which language is better in *general*, just for my
> purpose. My area of research is in CAD algorithms,
> and I'm sensing the need to resort to something more
> expedient than C++, bash scripting, or sed scripting.
> What I hope to do is a bit multifaceted. What I don't
> do dabble in is web apps (though I'm not sure of the
> similarities). No XML processing or database
> interaction in what I do.
Who knows, though. The desire to do certain things grows with the skills
one acquired.
> One thing I expect to have to do is to modify design
> files. For example, there is a tool which takes ASCII
> hardware desscription language (HDL) and converts it
> to a C++ (augmented by hardware simulation library).
> The translator is freeware, so has limitations which I
> have to make up for by tweaking the HDL code. In the
> past, I've eeked out sed scripts for such tasks, but
> would appreciate a more traditional language. Since
> Perl is used alot in digital IC design, I took a stab
> at that, motivated by the simple need to convert my
> mail aliases from one mail reader to the other. It
> took a while to do, and I'm concerned at the difficulty
> level. If I did this kind of thing constantly, I
> would probably get proficient and use the power behind
> it, but it's not my main area.
That sounds like a text-processing task. Perl's strengths in this area
are well-known, so there's no need to go into that deeper. Most
scripting languages (and that includes Python) can be used here.
> The alternative is Python, which is easier to read
> from what I've read. My concern there is that I cut
> myself off from large availability of stuff that's
> out there. For example, I use a PC-to-solaris
> viewer called VNC, and I've banged my head against the
> startup script to change it a bit for my situation.
> Likewise, the above translator uses Perl extensively
> in its operation, as well as it's building and
> installation. If I wasn't passingly familiar with
> Perl, I would have had a much harder time installing
> it. Being in the approximate area digit circuits,
> I'm concerned about being on the band wagon, if only
> to avoid reinventing things, or impediments to sharing
> things.
This is a strong reason to use Perl. While it may not always be
desirable to toe the line, here it is. The availability of tools that
can easily be integrated into one's own work will save you a lot of
work on the long run. I didn't know that Perl was particularly strong
in the field of IC design. If so, good then.
Apart from this particular case, it's generally a good thing when
pre-written code exists that can be used. The amount of such code for
Perl is immense (several thousand libraries) and gathered all in one
place (the CPAN) with a unified interface to access and install them and
a vast infrastructure built around it.
I don't know how much or whether at all the situation for Python has
significantly improved over the recent past. A while ago at least there
was no such thing.
> An additional usage scenario is to (if reasonable)
> replace my shell scripting e.g. I just converted to
> bash from tcsh to write scripts that push a document
> through a series of filters, or simply as a wrapper
> around a tough-to-use utility e.g. pstops. A final
> example is to take a file of design information and
> do more than tweak it e.g. extract all the information
> about interconnections between circuit building blocks,
> including which is the source block, and which are the
> destination blocks. Typically, this information will
> be read by matlab scripts and passed to my C++ code
> (I've managed to avoid writing code to parse the
> input file from C++).
As with text processing, both Python and Perl offer all the essential
things needed for that. On a cursory glance, the score of them is tied.
> Whichever way I go, I would like to avoid the overhead
> of learning both Perl and Python. I will sculpt out some
> time in a miserly fashion to slowly get to know one. Since
> I spend most of my time exploring the algorithm in Matlab or
> C++/STL, there's only so much time to pick up higher level
> languages (it took years before I shelled out the time
> to switch from tcsh to bash). One of the things that
> makes the decision not clear is that despite Python's
> claim to a gentler learning curve and clearer code, I
> often like the expedience of sed e.g. a terse one-liner
> that can be pipelined with other shell commands. I
> also note that Perl's unweldiness only comes for big
> projects, and I don't expect to using Perl quite that
> way.
Perl wins when it comes to one-liners and maybe replacing shell scripts.
As I tend to forget the syntax of bash's scripting language easily, I've
replaced it entirely with Perl.
Perl's bad reputation for big projects is probably due to the variance
of those people using Perl. One large group is sys-admins who use Perl
in an entirely different way as compared to someone building larger
systems. However, there are enough very large Perl projects that show
that it can be used for that just as easily (at least as long as one
keeps the sys-admins away from the source;-).
> I've seen mention of parrot and perl6, which is quite a
> ways off. I'm not sure how much that should weigh into
> my decision, since it isn't real yet.
It shouldn't. Perl6 is mostly yet another leap towards fitness for large
projects. I don't think it will gain much for those more hackish tasks.
Also, no one quite knows when Perl6 will be due.
Tassilo
--
$_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
$_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
Cameron Laird
Tassilo v. Parseval <tassilo....@post.rwth-aachen.de> wrote:
>Also sprach Fred Ma:
.
.
.
>> purpose. My area of research is in CAD algorithms,
>> and I'm sensing the need to resort to something more
>> expedient than C++, bash scripting, or sed scripting.
.
.
>> One thing I expect to have to do is to modify design
>> files. For example, there is a tool which takes ASCII
>> hardware desscription language (HDL) and converts it
>> to a C++ (augmented by hardware simulation library).
>> The translator is freeware, so has limitations which I
>> have to make up for by tweaking the HDL code. In the
>> past, I've eeked out sed scripts for such tasks, but
>> would appreciate a more traditional language. Since
>> Perl is used alot in digital IC design, I took a stab
>> at that, motivated by the simple need to convert my
>> mail aliases from one mail reader to the other. It
.
.
>> The alternative is Python, which is easier to read
>> from what I've read. My concern there is that I cut
>> myself off from large availability of stuff that's
>> out there. For example, I use a PC-to-solaris
>> viewer called VNC, and I've banged my head against the
>> startup script to change it a bit for my situation.
.
.
>work on the long run. I didn't know that Perl was particularly strong
>in the field of IC design. If so, good then.
>
>Apart from this particular case, it's generally a good thing when
>pre-written code exists that can be used. The amount of such code for
>Perl is immense (several thousand libraries) and gathered all in one
>place (the CPAN) with a unified interface to access and install them and
>a vast infrastructure built around it.
.
.
>> An additional usage scenario is to (if reasonable)
>> replace my shell scripting e.g. I just converted to
>> bash from tcsh to write scripts that push a document
>> through a series of filters, or simply as a wrapper
>> around a tough-to-use utility e.g. pstops. A final
.
.
>> destination blocks. Typically, this information will
>> be read by matlab scripts and passed to my C++ code
>> (I've managed to avoid writing code to parse the
>> input file from C++).
.
.
>> Whichever way I go, I would like to avoid the overhead
>> of learning both Perl and Python. I will sculpt out some
>> time in a miserly fashion to slowly get to know one. Since
>> I spend most of my time exploring the algorithm in Matlab or
>> C++/STL, there's only so much time to pick up higher level
>> languages (it took years before I shelled out the time
>> to switch from tcsh to bash). One of the things that
>> makes the decision not clear is that despite Python's
>> claim to a gentler learning curve and clearer code, I
>> often like the expedience of sed e.g. a terse one-liner
>> that can be pipelined with other shell commands. I
.
.
Where to begin? I'm seriously surprised that you haven't
mentioned Tcl, originally created to serve CAD needs, and
still more prevalent than either Perl or Python in CAD,
from all I know. Perl is indeed present, too, although,
in some ways, PythonCAD <URL: http://www.pythoncad.org/ >
is the most interesting action in this domain. All three
languages have enough to recommend them, and are close
enough in their capabilities, that I feel comfortable ad-
vising you to choose the one that suits *you* best. If
you like thinking in Perl, choose that; you'll miss out
on a few Python capabilities, but you'll be as productive
as with any other choice.
Either Perl or Python is going to satisfy you for algorithmic
experiments much, MUCH more than Matlab or C++. Make the
switch, *now*.
Either Perl or Python can adequately substitute for shell
in the uses you describe. They pipeline fine themselves;
'fact, I'm working on an essay on how they're *better* at
pipelining.
"The translater is freeware, so has limitations ...": I
don't understand how to read this. Has it been your
experience that commercial CAD packages are perfect, and
without limitations?
I work with VNC a lot, too. You seem to believe that
there's a special relation between VNC and Perl; do I
have that right? That surprises me. If anything, I'd
say Tcl leads in that domain <URL:
http://www-106.ibm.com/developerworks/linux/library/l-vnc/ >.
I generally favor Python over Perl; in the absence of
more details, I think there's mild evidence that those
who don't specialize in programming are happier over the
long term with the readability of the former. They're
very close, though. The one most certain advantage
Python boasts is a dimension that I'd think important
to you, although you haven't mentioned it: Python's
interfaces to C++ are MUCH easier to manage than Perl's.
Perl6 will change this. For now, though, it's FAR
easier to practice "dual-level programming" with Python
and C++. If you have a large existing library of C++
work, I think that tips the balance toward Python.
Understand, SWIG and other alternatives make Perl-to-C++
links possible; with Python's facilities, though,
including Pyrex, they're *fun*.
There's more to say; I'll be back later.
--
Cameron Laird <cla...@phaseit.net>
Business: http://www.Phaseit.net
Fred Ma
>
> Where to begin? I'm seriously surprised that you haven't
> mentioned Tcl, originally created to serve CAD needs, and
> still more prevalent than either Perl or Python in CAD,
I was under the impression (rightly or wrongly) that Tcl
was good at interfacing to, and controlling, CAD tools.
I was more motivated by the need to massage data, though
they probably overlap greatly. My experience with Tcl is
limited to using Synopsys DC shell to do unconventional
things, like traverse my design hierarchy (originally
in verilog) flattening and stripping out things. At the
time, the documentation for their dcshell was more
complete than for their Tcl version of the same. But
it's good to know that Tcl is an option to Perl and Python,
at least for some things.
> from all I know. Perl is indeed present, too, although,
> in some ways, PythonCAD <URL: http://www.pythoncad.org/ >
> is the most interesting action in this domain. All three
> languages have enough to recommend them, and are close
> enough in their capabilities, that I feel comfortable ad-
> vising you to choose the one that suits *you* best. If
> you like thinking in Perl, choose that; you'll miss out
> on a few Python capabilities, but you'll be as productive
> as with any other choice.
Good to know that at a personal level, I can run faster with
whichever one I feel more comfortable with (though I have
yet to explore Python for personal comfort). I was also
trying to take into consideration the prevalence of Perl
in digital design.
Along this line, I should clarify that by CAD, I don't mean
general purpose computer aided drawing. I mean software
that helps designers in digital design, much of which is not
graphical. In fact, my area is more combinatoric
optimization and graph theory-ish than drawing. Sometimes,
I forget that CAD has a much more general meaning than this.
> Either Perl or Python is going to satisfy you for algorithmic
> experiments much, MUCH more than Matlab or C++. Make the
> switch, *now*.
I'm surprised to hear that. Perhaps it's related to the
confusion above (due to the initial generality of my
description). My thesis research deals with evolutionary
algorithms (EAs) for programming coarse grain reconfigurable
logic platforms. They tend to be heavy on computation, and
I migrated to C++ for the speedup compared to matlab.
Matlab has been made fast for highly vectorizable
computation, but the restrictive conditions for such speedup
was, too restrictive. I saw some examples of genetic
algorithms in Perl (or perhaps it was Python), but assumed
(maybe prematurely) that it was much slower than C++. Other
factors also contributed to this impression, including the
fact that I augment the evolutionary search with local searh
heuristics, giving rise to complicated control flow in the
code. So I've used C++ for the computation engine code, and
shell/data-massaging scripts for "glue" activity. Matlab
I'm using because I've got alot of familiarity invested in
its graphing capabilities (to assess data rather than
drawing). I'm also pretty familiar with its compact
vectorized operations, which helps for data massaging in the
post-processing phase.
> Either Perl or Python can adequately substitute for shell
> in the uses you describe. They pipeline fine themselves;
> 'fact, I'm working on an essay on how they're *better* at
> pipelining.
Actually, I was thinking more along the lines of pipelining
Perl commands in a bash command pipeline, along with other
commands. But it's good to know that they pipeline within
their own scripts (if I understand you correctly), and for
general shell scripting.
> "The translater is freeware, so has limitations ...": I
> don't understand how to read this. Has it been your
> experience that commercial CAD packages are perfect, and
> without limitations?
The man page for "verilator" says this: "many limitations and
lack of features compared to a commercial simulator". One
of the things that prompted me to ask about Perl/Python is
because I'm looking at making many changes throughout a
number of files in order to accommodate the limitations.
Regardless of limitations carried by commercial tools, these
are the ones that I have to deal with.
> I work with VNC a lot, too. You seem to believe that
> there's a special relation between VNC and Perl; do I
> have that right? That surprises me. If anything, I'd
> say Tcl leads in that domain <URL:
> http://www-106.ibm.com/developerworks/linux/library/l-vnc/ >.
I described my experience with VNC just as an example of the
various situation in which I had to make sense of Perl
scripts. The vncserver script is written in Perl, at least
for TightVNC, and for older versions of RealVNC I seem to
recall. That's the extent to which I delved into VNC, since
my area of study (and thus the area where I spend most of my
time) is in "CAD" algorithms, as described above. I was
driven more by a need to expediently get a certain
kind/level of functionality from VNC rather than the
intention of developing it.
> I generally favor Python over Perl; in the absence of
> more details, I think there's mild evidence that those
> who don't specialize in programming are happier over the
> long term with the readability of the former. They're
> very close, though. The one most certain advantage
> Python boasts is a dimension that I'd think important
> to you, although you haven't mentioned it: Python's
> interfaces to C++ are MUCH easier to manage than Perl's.
> Perl6 will change this. For now, though, it's FAR
> easier to practice "dual-level programming" with Python
> and C++. If you have a large existing library of C++
> work, I think that tips the balance toward Python.
> Understand, SWIG and other alternatives make Perl-to-C++
> links possible; with Python's facilities, though,
> including Pyrex, they're *fun*.
That's good news, from my standpoint. I don't know what
level of programming qualifies a person to be "specializing
in programming", thus leading to more satisfaction with
Python. I've tried to read my own C++ code after half a
year, and have gained a new appreciation for the importance
of readability. So I'm still fine-tuning my idea of
adequate commenting. Since I'm working on my thesis rather
than commercial code, I don't have large amounts of generic
C++ library code. Just a big-ish heap of routines/functions
for the problem that I'm investigating, as the algorithms
that I choose to try morph.
My impression is that you think Python would be more
appropriate for my situation. Just wondering if you can
comment on the merit of having lots of Perl code around. It
seems to be prevelant in a digital design tool environment.
It may not matter all that much to me right now, but it
would be nice to be on the same wavelength as others if I
find myself in a team situation. As I mentioned, there have
already been situations where a nodding familiarity with
Perl has helped in making use of other people's tools.
> There's more to say; I'll be back later.
Thanks for your comments, and looking forward to hear any
further comments.
Fred
Paul McGuire
news:40652B0D...@doe.carleton.ca...
<snip>
> One thing I expect to have to do is to modify design
> files. For example, there is a tool which takes ASCII
> hardware desscription language (HDL) and converts it
> to a C++ (augmented by hardware simulation library).
> The translator is freeware, so has limitations which I
> have to make up for by tweaking the HDL code.
If you eventually find yourself in the Python realm, please look into the
pyparsing text parsing module, more information at
http://pyparsing.sourceforge.net/. I have implemented an easily-extended
99% Verilog parser using this module, and it may provide some shortcuts for
you in dealing with your HDL files.
-- Paul McGuire
Roy Smith
> I was under the impression (rightly or wrongly) that Tcl
> was good at interfacing to, and controlling, CAD tools.
Tcl is an interesting language, and well worth learning, although it
certainly has its share of quirks.
The product I work on has close to 20,000 lines of Tcl code doing
network discovery via SNMP. The low-level SNMP protocol is accessed via
hooks done in C. It's not very fast, but it's as fast as it has to be,
and I shudder to think how many lines of C or C++ the whole system would
have been.
The real strengths are twofold:
1) It's easy to learn. Even easier than Python. We take guys who are
networking types with no programming experience and get them doing
useful development in Tcl in just a few days.
2) It's dead simple to embed in a C/C++ application. This is one place
where it really outshines Python. It's literally two lines of C to get
a Tcl interpreter up and running. It's equally trivial to extend Tcl by
providing functions in C. Compare this to the native Python/C
interface, boost.python, or swig, and you really appreciate the
difference!
For most general purpose tasks, I reach for Python first. But if I
needed to provide a quick scripting interface to an existing C library,
or wanted to embed some easy scriptability into a large C/C++ app, Tcl
is what I would be looking at.
Jim Keenan
>
[snip 4 paragraphs]
>
> Whichever way I go, I would like to avoid the overhead
> of learning both Perl and Python. I will sculpt out some
> time in a miserly fashion to slowly get to know one.
Okay. You are at Carleton University, which was the site of Yet
Another Perl Conference::Canada in May 2003. So you have an excellent
pool of local Perl experts to draw upon -- and there are many more not
far away in Montreal, Toronto and Kitchener/Waterloo. Look them up
via http://www.pm.org/groups/north_america.html .
Jim Keenan
Brooklyn NY
Cameron Laird
Fred Ma <f...@doe.carleton.ca> wrote:
.
.
.
>I was under the impression (rightly or wrongly) that Tcl
>was good at interfacing to, and controlling, CAD tools.
>I was more motivated by the need to massage data, though
>they probably overlap greatly. My experience with Tcl is
>limited to using Synopsys DC shell to do unconventional
>things, like traverse my design hierarchy (originally
>in verilog) flattening and stripping out things. At the
>time, the documentation for their dcshell was more
>complete than for their Tcl version of the same. But
>it's good to know that Tcl is an option to Perl and Python,
>at least for some things.
.
.
.
Several people have posted good advice already.
Thanks for the clarification; NOW I understand considerably
better your focus on graph-theoretic and related work.
When I characterize you as not primarily a software person,
I'm just trying to say that your aim is engineering or
mathematical understanding. You have a life beyond pro-
gramming (good thing!).
My current instinct is that you'll be happiest with Perl--
but it's a close choice, all around. I'm most accomplished
in Tcl, and would happily use it, or Perl, or Python. It
seems to me, though, that it's crucial that you be able
to exploit libraries that pertain to your domain, and, in
particular, optimized engines for linear algebra, graph
theory, and linear and mathematical programming. Tcl
lacks the weight that Perl and, to an increasing extent,
Python can boast in their libraries.
The most important thing I can do is to continue to push
you in the direction of exploring at least one of Perl
and Python. YES genetic algorithms and other adaptive
methods are available. Some C++ is more mature--but some-
times Perl, or Python, have taken the lead. I understand
your concern with performance. Switch to one of Perl or
Python now, and I think you'll look back on the decision
as a correct one.
That's much more to say on precisely this issue. It
likely will take weeks to get it all out. I'm quite
certain of my conclusion, though, even without making the
time to explain it in detail: you'll be happier basing
your experiments in combinatoric and graph-theoretic
algorithms on Perl or Python than if you focused on C++.
Cameron Laird
Roy Smith <r...@panix.com> wrote:
.
.
.
>2) It's dead simple to embed in a C/C++ application. This is one place
>where it really outshines Python. It's literally two lines of C to get
>a Tcl interpreter up and running. It's equally trivial to extend Tcl by
>providing functions in C. Compare this to the native Python/C
>interface, boost.python, or swig, and you really appreciate the
>difference!
>
>For most general purpose tasks, I reach for Python first. But if I
>needed to provide a quick scripting interface to an existing C library,
>or wanted to embed some easy scriptability into a large C/C++ app, Tcl
>is what I would be looking at.
Me, too, to a large extent. I think you overstate Tcl's
advantage, though, Roy, in a couple of ways. First, I
agree that Tcl is objectively easier and more reliable
in embedding; Python still has blemishes in ease-of-use
and portability in this role. It can be hard getting
all of Python embedded correctly, especially, in my ex-
perience, when working on multiple platforms.
However, Tcl has historically been so easy that it hasn't
bothered documenting the process. Python has a definitive
(if aging) embedding manual that I regard as more inviting
than what Tcl offers.
Also, ctypes, Boost and Pyrex *are* successful, and appear
to me progressing faster than any of their Tcl cousins.
Yet another way to look at the comparison is that a sig-
nificant number of people rely on SWIG, and, to them,
Tcl and Python are indistinguishable.
Follow-ups narrowed. Most of Perl's response to all of
this is, "Wait 'til 6!"
Cameron Laird
.
.
I *really* don't know how to respond to this; a
sentence with "sed", "conditional" and "convenient"
leaves me at a loss.
Both Perl and Python are good fits for large popu-
lations. I have a lot of confidence that, in the
near future, you'll find the one you've choosen
handier than either sed or Matlab, *even for most
of the tasks at which sed and Matlab are good*.
Perl and Python are that strong.
Python's population has a greater propensity to
regard its readability as an advantage, while
Perlites often revel in concise power. Try 'em
both briefly, choose one that makes a good impres-
sion, and get back to your engineering work.
You'll be glad you upgraded to a 'P' language.
zatoichi
> In article <40657D14...@doe.carleton.ca>,
> My current instinct is that you'll be happiest with Perl--
> but it's a close choice, all around. I'm most accomplished
> in Tcl, and would happily use it, or Perl, or Python. It
> seems to me, though, that it's crucial that you be able
> to exploit libraries that pertain to your domain, and, in
> particular, optimized engines for linear algebra, graph
> theory, and linear and mathematical programming. Tcl
> lacks the weight that Perl and, to an increasing extent,
> Python can boast in their libraries.
Since you happily use Perl, Python or Tcl, I have a question. How do you
decided which one to use? :-)
Robert
Cameron Laird
Most often, whatever is already at hand. It'll be interesting
some day to come across an organization that doesn't already
have a standard in place.
Tassilo v. Parseval
> I generally favor Python over Perl; in the absence of
> more details, I think there's mild evidence that those
> who don't specialize in programming are happier over the
> long term with the readability of the former. They're
> very close, though. The one most certain advantage
> Python boasts is a dimension that I'd think important
> to you, although you haven't mentioned it: Python's
> interfaces to C++ are MUCH easier to manage than Perl's.
> Perl6 will change this. For now, though, it's FAR
> easier to practice "dual-level programming" with Python
> and C++. If you have a large existing library of C++
> work, I think that tips the balance toward Python.
> Understand, SWIG and other alternatives make Perl-to-C++
> links possible; with Python's facilities, though,
> including Pyrex, they're *fun*.
When it comes to embedding C/C++ into Perl, SWIG is the last thing that
I'd recommend. It may be capable of transforming such a library into a
Perl module on its own with no user intervention. However, C/C++ doesn't
map particularly well onto Perl from an interface point-of-view: A 1:1
translation of a C++ API doesn't look particularly good.
Those things should be done manually. How much work this is depends on
the library. Some only require a few lines of XS. XS' learning curve is
admittedly a bit steep. Not because it is difficult in itself but rather
because the documentation leaves a few things to be desired. There are
some actions on completing perlxstut.pod on its way right now, though.
There is also Inline::CPP, although I don't think it is as mature as
Inline::C. And since you have mentioned Tcl, even that can be inlined
with the help of Inline::Tcl.
Cameron Laird
.
.
>When it comes to embedding C/C++ into Perl, SWIG is the last thing that
>I'd recommend. It may be capable of transforming such a library into a
>Perl module on its own with no user intervention. However, C/C++ doesn't
>map particularly well onto Perl from an interface point-of-view: A 1:1
>translation of a C++ API doesn't look particularly good.
>
>Those things should be done manually. How much work this is depends on
>the library. Some only require a few lines of XS. XS' learning curve is
>admittedly a bit steep. Not because it is difficult in itself but rather
>because the documentation leaves a few things to be desired. There are
>some actions on completing perlxstut.pod on its way right now, though.
>
>There is also Inline::CPP, although I don't think it is as mature as
>Inline::C. And since you have mentioned Tcl, even that can be inlined
>with the help of Inline::Tcl.
.
.
As I think about it more, maybe Inline::C (or Tcl's
Critcl, for that matter) *is* something that would
suit the original poster roughly as well as Pyrex.
Incidentally, I've heard that Inline::Tcl might be
an area of renewed interest; it has an inverse in
tclperl.
Fred Ma
"Eric @ Zomething" wrote:
>
> Roy Smith wrote:
>
> > For most general purpose tasks, I reach for Python first. But if I needed
> > to provide a quick scripting interface to an existing C library,
>
> I would suggest that one might take an option other than the either/or
> (Perl/Python/____). It sounds like having at least some comfort with Perl is
> important to OP based on relevant code base. However, Python is so simple to
> learn, at least at the beginner/intermediate level, that there is not much
> cost to learning some Python as well.
>
> Personal experience: I struggled to becoming proficient with Perl due to its
> syntax and perhaps my lack of bottomless commitment (knowing Perl would not be
> my bread and butter). Somehow I stumbled upon a little bit of Python and got
> sucked in. Now I am probably more proficient at Perl than I was before I
> started using Python (except for needing to look up Perl syntax from time to
> time) because working with Python took my head out of the syntax and let my
> mind focus on the problem solving... and more importantly, the objective.
>
> Python has me feeling like I can make a program to do _anything_ I want, even
> though I doubt it is any more capable than Perl, etc. It's matter of clarity
> and better being able to see the forest, rather than a whole lot of trees.
>
> This might sound like something less than a firm choice, but perhaps OP might
> decide to start digging into Perl in April, but allow one weekend in that
> month for Python and let things evolve as best suits his situation.
Eric, you voiced exactly the opinion that was forming in my mind upon reading
the responses. There is a complicating factor (actually the most pressing
factor) behind my either/or approach. That is, I must wrap up my thesis very
soon, and what I've proposed is alot more than what there is time for. However,
given the responses, I agree that that most reasonable approach is to adopt one
and keep tabs on the other. Perl seems to be the natural choice for the first,
since I see it in alot of places, meaning I will probably have to poke through
it often. And the ease of Python will make it feasible to keep in touch with
it, if not immediately, then with time.
Aside from hacking other people's Perl code, I've already bloodied my nose on
Perl composition (quite a bit harder than small modifications to other people's
code, I found), so I'm already down that path a bit. I expect to be using this
primarily for "glue" work i.e. to mulch information meant for one utility (e.g.
verilog) so that it can be used for another e.g. verilator, or my own code.
For the computation engine, I will stick with C++ and matlab (for its graphing),
again due to the lack of time to ramp up thoroughly on something else. Also,
despite the availability of Perl code for evolutionary algorithms, I found that
in exploring algorithms, one can very quickly depart from the funcionality
offered by a standard package. Even C/C++ libraries for these algorithms
presume a certain general framework which can impact speed; I didn't use them,
but if I did, I expect I would have diverged from the code base very early.
On the topic of speed, It's surprising to hear that this can be possibly rivaled
by Perl/Python, considering that even my matlab code is about 10x slower than
C++. That's with extensive profiling and round-about coding styles to exploit
vectorization tricks, and no such effort in the C++ code. Some of the
limitations are pretty fundamential e.g. large speed penalties for invoking
nonbuiltin functions. Such limitations discourage clear coding practice e.g.
huge swaths of code to avoid encapsulating some functionality in its own
function. Together with the sometimes roundabout coding style, it can detract
from the matlab's high level ease of programming and clarity, especially
considering that C++ can be brought to a high level with its STL library. And
this speed disparity shows up despite matlab's mission to develop in a way that
rivals 3G languages. Speed considerations aside, however, matlab is infinitely
easier to use and to read. With time, perhaps the limitations will shrink.
This is not to say that Perl/Python will have the same limitations. They are,
however, more general purpose languages, so I expect that speed is less of a
priority than in scientific computing (compared to other important priorities).
Hence, learning curve aside, some time needs to be invested in exploring its
speed in a realistic scientific applications. For me, this is an option for the
more distant future rather than the immediate term. I spent quite alot of time
in this phase with matlab, trying to squeeze out all possible speed before
conceding that some of the limits were fundamental, at least in the current
release.
One thing I learned was that speed characterization was not a simple. Many of
the tasks found in scientific computing are done blistering fast by matlab. It
is not until you put together a larger application when you start running into
the limitations. Aside from the speed penalties of calling nonbuiltins,
roundabout coding style is not such a big deal for small chunk of code. It can
be debilitating when you're trying to wrap your head around something large.
The large chunks of inline code exacerbate this.
A final reason to stick with C++ is the hardware simuation library ("SystemC").
The Perl translator I alluded to in an earlier post translates code in hardware
description language ("verilog") to C++ specifically for use with SystemC (well,
it does more than that, but that's my main use of it). This allows my C++ code
to more directly control the hardware model and its simulation, in contrast to
using a standalone simulator to simulate the original verilog. Depending on the
level of poking and prodding, it can be extremely arduous to go through the API
of the verilog simulator. This is especially true if you intend to change the
hardware models alot between simulations (or do things to achieve the same
effect), and all the more true if the functionality outside of the verilog
simuation is meant to supervise the various simulations instead of being
subordinate to them. An alternative is to have the C++ (or whatever) parent
program generate different verilog code between simulations and invoke the
simulator each time. This would be far too slow.
I realize that my inclination to write my own code sort of contradicts my want
to springboard off of existing Perl code base. I only had a vague idea of the
benefits of this, since I don't actually program in Perl right now. There are
several points I've tried to determine the relevance of. On one hand, I've
looked at and hacked Perl code to get things done, so knowing Perl helps. On
the other hand I need a good way to repackage data for different applications,
in which case I'll compose code from scratch. For this, I would appricate the
lesser pain and greater ease of Python (I imagine). On yet another hand, I use
Perl-based packages like the verilog-to-C++ translator, but I don't expect to
delve into such a complicated package, whose complexity probably lies in more
than just use of the Perl language. Finally, and most vaguest of all, I have
this idea that I might eventually be sharing Perl code, collaborating in its
composition, or revising/maintaining it. If "synergy" in that form is actually
typical in my area of specialization, then of course it would make sense to
start with Perl first, since that is the prevalent language in this area.
Paul Prescod
>...
> .
> .
> .
> As I think about it more, maybe Inline::C (or Tcl's
> Critcl, for that matter) *is* something that would
> suit the original poster roughly as well as Pyrex.
Neither Pyrex nor Inline::C are going to be great for C++. They were
both created by C programmers for C programmers (I'd like to see this
change about Pyrex some day). BOOST::Python, on the other hand, is
probably a good fit.
Paul Prescod
Fred Ma
"Tassilo v. Parseval" wrote:
>
> Also sprach Fred Ma:
>
>
> Who knows, though. The desire to do certain things grows with the skills one
> acquired.
You might be right. Some digital CAD data is stored in XML.
> This is a strong reason to use Perl. While it may not always be desirable to
> toe the line, here it is. The availability of tools that can easily be
> integrated into one's own work will save you a lot of work on the long run. I
> didn't know that Perl was particularly strong in the field of IC design. If
> so, good then.
>
> Apart from this particular case, it's generally a good thing when pre-written
> code exists that can be used. The amount of such code for Perl is immense
> (several thousand libraries) and gathered all in one place (the CPAN) with a
> unified interface to access and install them and a vast infrastructure built
> around it.
>
> I don't know how much or whether at all the situation for Python has
> significantly improved over the recent past. A while ago at least there was no
> such thing.
SNIP
> As with text processing, both Python and Perl offer all the essential things
> needed for that. On a cursory glance, the score of them is tied.
>
> Perl wins when it comes to one-liners and maybe replacing shell scripts. As I
> tend to forget the syntax of bash's scripting language easily, I've replaced
> it entirely with Perl.
>
> Perl's bad reputation for big projects is probably due to the variance of
> those people using Perl. One large group is sys-admins who use Perl in an
> entirely different way as compared to someone building larger systems.
> However, there are enough very large Perl projects that show that it can be
> used for that just as easily (at least as long as one keeps the sys-admins
> away from the source;-).
>
> > I've seen mention of parrot and perl6, which is quite a ways off. I'm not
> > sure how much that should weigh into my decision, since it isn't real yet.
>
> It shouldn't. Perl6 is mostly yet another leap towards fitness for large
> projects. I don't think it will gain much for those more hackish tasks. Also,
> no one quite knows when Perl6 will be due.
Thanks for the commentary on Perl, for clarifying the source of unweldiness in
big Perl projects, as well as its nonuniversality.
Fred
Petri
> One thing I expect to have to do is to modify design
> files. For example, there is a tool which takes ASCII
> hardware desscription language (HDL) and converts it
> to a C++ (augmented by hardware simulation library).
> The translator is freeware, so has limitations which I
> have to make up for by tweaking the HDL code.
The translator, is it doing the Verilog stuff you mention later in the thread?
Why not check if you can't replace the translator altogether with an existing
Perl module:
http://search.cpan.org/search?query=verilog&mode=all
Well, I know nothing about your problem area, but it's a tip in case you hadn't
thought about checking CPAN.
Petri
Cameron Laird
Fred Ma <f...@doe.carleton.ca> wrote:
.
.
.
.
'Couple of reactions: first, I think it's entirely healthy
for you to focus on finishing your dissertation. Everything
else is and should be subordinate to that. Picking up as
much of Perl as you need to achieve that goal, but not more,
is thoroughly sensible.
Second, be aware that I'm the only one in this thread who has
made explicit claims that it's reasonable to expect Perl- or
Python-based applications to rival those in C++ for
performance. Maybe I'm wrong, or, more charitably, maybe I'm
making assumptions that I've kept implicit. It certainly is
possible to exhibit comparable-looking C and Python programs,
where the former is over a hundred times as fast as the latter.
Still, I'm sticking with my claim: for practical, effective
algorithmic experimentation, PDL, Numpy, and comparable exten-
sions bring 'P' programs' achieved speed up to the level of
C++.
Ultimately, you're right: "speed characterization [is] not
simple."
Perl and Python feel like breaths of fresh air after Matlab
precisely for their scalability and maintainability. Matlab
just doesn't generalize and abstract and express as agilely
as the more general-purpose programming languages.
Paddy McCarthy
If there are particular C/C++ packages of algorithms that you use then
you might try searching for the package name +python on the web. You
might find someone who has already turned the library into aPython
module.
There are several Scientific libraries for Python that have good
graphing capabilities as well as speed advantages for certain
operations. - Try installing the Enthought Python distribution for
windows at: http://www.enthought.com/python/
Another good page is ScriptEDA:
http://www-cad.eecs.berkeley.edu/~pinhong/scriptEDA/ - I was using the
gate-level verilog parser a month ago.
On Scripting language choice, I find TCL to be in many EDA tools, but
its datastructures and syntax make it a language I use when I have
to, rather than a language of choice.
Perl IS used a lot in EDA but I like to think of it as being dangerous
in a lot of hands as great Engineers start scripting larger and larger
applications in Perl and usually end up with something that is hard to
maintain and hard to write. Perl does have its sweet spot - I do like
the `perl -p -i -e` mewthod of making sed like alterations to files
in-place, butthe languages support for subroutine arguments - compared
to Pythons; and the need to compute and pass around references to
lists and hashes for simple nested datastructures - things like that
shout small programs only to me.
Python is my language of choice for a large amount of programming in
the EDA field (when given the choince :-).
Perl has a large library, Python has a large library - what you need
to do is do some web searches for libraries in your field to see which
is more appropriate -
When I have to write an algorithm from scratch then I do it in Python.
It does get out of the way and allow me to concentrate on the
algorithm most of the time. (Although I wish it had constrained random
generation of integers like Vera or Specman :-).
I hope I've helped, Paddy.
Cameron Laird
>to springboard off of existing Perl code base. I only had a vague idea of the
>benefits of this, since I don't actually program in Perl right now. There are
>several points I've tried to determine the relevance of. On one hand, I've
>looked at and hacked Perl code to get things done, so knowing Perl helps. On
>the other hand I need a good way to repackage data for different applications,
>in which case I'll compose code from scratch. For this, I would appricate the
>lesser pain and greater ease of Python (I imagine). On yet another hand, I use
>Perl-based packages like the verilog-to-C++ translator, but I don't expect to
>delve into such a complicated package, whose complexity probably lies in more
>than just use of the Perl language. Finally, and most vaguest of all, I have
>this idea that I might eventually be sharing Perl code, collaborating in its
>composition, or revising/maintaining it. If "synergy" in that form is actually
>typical in my area of specialization, then of course it would make sense to
>start with Perl first, since that is the prevalent language in this area.
.
.
One of the hardest things in science (and business,
and romance, and ...) is to deal with uncertainty
and incompleteness. Yup: you have countervailing
criteria. Figuring out the true optimum is a bigger
project than the CAD engineering you claim as your
goal. At some point--probably this weekend, from
the sound of it--you need to make decisions, and
move forward. They might be wrong decisions, on
narrow technical grounds. Good scientists make
wrong decisions all the time. Don't let 'em get in
the way of your science--that's all.
You've done a good job thinking about what you want,
and what's available. Good luck with your combina-
torics.
Cameron Laird
freeware, VNC, ...
that no longer seems
urgent]
.
.
>Python. I've tried to read my own C++ code after half a
>year, and have gained a new appreciation for the importance
>of readability. So I'm still fine-tuning my idea of
>adequate commenting. Since I'm working on my thesis rather
>than commercial code, I don't have large amounts of generic
>C++ library code. Just a big-ish heap of routines/functions
>for the problem that I'm investigating, as the algorithms
>that I choose to try morph.
>
>My impression is that you think Python would be more
>appropriate for my situation. Just wondering if you can
>comment on the merit of having lots of Perl code around. It
>seems to be prevelant in a digital design tool environment.
>It may not matter all that much to me right now, but it
>would be nice to be on the same wavelength as others if I
>find myself in a team situation. As I mentioned, there have
.
.
Some scientists are very successful with "slash-and-burn"
techniques. That manifests itself in this area as "con-
science-free" coding: they write whatever programs get
them the results they're after, and don't care that no
one else can reproduce their results, nor can they them-
selves, six months later. A LOT of that goes on.
It makes me uncomfortable. A lesson I've learned over
and over is that computer programs live far longer than
you expect. It pays to do 'em better at the beginning,
'cause you're likely to live with them quite a while.
I'm not a successful scientist, though; odds are long
against me ever being one.
Being "on the same wavelength" is yet another of the
dualities you face. It has its advantages. So does
leadership in establishing new and better ways. You
might never know which is the right choice. Most cer-
tain is that *you* are the one who'll live with the
choices you make.
Fred Ma
Wow, there's more information coming in than I expected. Having
a bit of a nonengineering day (at least not at the ground level)
keeping up. I did ask for it.
Paddy McCarthy wrote:
>
> Hi Fred,
> If there are particular C/C++ packages of algorithms that you use then
> you might try searching for the package name +python on the web. You
> might find someone who has already turned the library into aPython
> module.
Actually, verilator (does verilog to SystemC, which is a C++ library
that provides HDL simulation capability) is a Perl-based module.
As for the glue functionality I need to modify the my original verilog
source, there's no specific functionality. I'm still trying to think
of ways in which to modify to code to work around the translation
limitations of verilator. So the filter will be very much ad-hoc.
> There are several Scientific libraries for Python that have good
> graphing capabilities as well as speed advantages for certain
> operations. - Try installing the Enthought Python distribution for
> windows at: http://www.enthought.com/python/
>
> Another good page is ScriptEDA:
> http://www-cad.eecs.berkeley.edu/~pinhong/scriptEDA/ - I was using the
> gate-level verilog parser a month ago.
Wow. The screen shots on for enthought are impressive. The software at
scriptEDA reminds me that I'm an a newbie in this area. The level to
which interfacing environments and languages have been brought boggles
the mind. I think that I need to take a bite of Perl and/or Python
before I even consider looking at the offerings there with any degree
of appreciation. For the computation engine and post-process/graphing,
I'll stick with C++/matlab for now, due to lack of thesis time, and
because I've got a familiar system (or "idiom"?) setup to use them
together. As I mentioned in another post, I'll start with Perl as
the starting point, and keep tabs on Python, something that is feasible
because of its readability.
> On Scripting language choice, I find TCL to be in many EDA tools, but
> its datastructures and syntax make it a language I use when I have
> to, rather than a language of choice.
>
> Perl IS used a lot in EDA but I like to think of it as being dangerous
> in a lot of hands as great Engineers start scripting larger and larger
> applications in Perl and usually end up with something that is hard to
> maintain and hard to write. Perl does have its sweet spot - I do like
> the `perl -p -i -e` mewthod of making sed like alterations to files
> in-place, butthe languages support for subroutine arguments - compared
> to Pythons; and the need to compute and pass around references to
> lists and hashes for simple nested datastructures - things like that
> shout small programs only to me.
I got the same impression, but there have been other explanations given
in this thread for the unweldiness of big projects, as well as the fact
that this is not universal. Myself, I will be using "P..." for glue
functionality, so I won't run the risk of creating dangerously
unmaintainable code (finger crossed here). Besides, it's for the thesis,
which has to be finished soon, so I won't have the problem of looking
back a ways to figure out code written long ago. By the time I get to
an industrial situation (fingers crossed, since jobs are leaving North
America), I hope to have a bit for breathing time to ramp up on Python.
> Python is my language of choice for a large amount of programming in
> the EDA field (when given the choince :-).
>
> Perl has a large library, Python has a large library - what you need
> to do is do some web searches for libraries in your field to see which
> is more appropriate -
Well...I did it...verilog to SystemC boils down to verilator unless one
has money to shell out. Despite its professed limitations, I am still
astounded to read even the help pages. That degree of sophistication,
for free. Kind of makes the thesis look like small time, though of
course, the challenges are of a different nature.
> When I have to write an algorithm from scratch then I do it in Python.
> It does get out of the way and allow me to concentrate on the
> algorithm most of the time. (Although I wish it had constrained random
> generation of integers like Vera or Specman :-).
>
> I hope I've helped, Paddy.
A great deal. Thanks.
Fred
Fred Ma
Hi, Paul,
I took a look at your website. I've decided to go Perl for now,
and ramp up on Python on the side. I think a parser has a higher
level of intelligence than regex'ing, but I hesitate to jump into
it at the moment because thesis time is running out. I may do some
adhoc regex'ing, either with sed/Perl/gvim (it's quick and dirty,
but suitable for the time crunch of my current circumstance, which
is a hard deadline on the thesis). Also, I'm doing the quick-and-
dirty because of limitations of an *existing* verilog parser (and
translator) which I don't want to delve into right now, for the
same reason. But thanks for the heads up. If things work out in
the long run, and I get to know Python, I know there is a verilog
parser to keep an eye out for.
Fred
Fred Ma
>
> You are at Carleton University, which was the site of Yet
> Another Perl Conference::Canada in May 2003. So you have an excellent
> pool of local Perl experts to draw upon -- and there are many more not
> far away in Montreal, Toronto and Kitchener/Waterloo. Look them up
> via http://www.pm.org/groups/north_america.html .
Thanks for the pointer, Jim.
Fred
Fred Ma
Actually, I was spending much of last week installing many
of those tools! The translater (verilator) is Perl-based,
written by Wilson Snyder (author of many of those tools)
and relies on many of those tools. But thanks for the tip...
Fred
Fred Ma
>
> One of the hardest things in science (and business,
> and romance, and ...) is to deal with uncertainty
> and incompleteness. Yup: you have countervailing
> criteria. Figuring out the true optimum is a bigger
> project than the CAD engineering you claim as your
> goal. At some point--probably this weekend, from
> the sound of it--you need to make decisions, and
> move forward. They might be wrong decisions, on
> narrow technical grounds. Good scientists make
> wrong decisions all the time. Don't let 'em get in
> the way of your science--that's all.
Sounds like the story of life, actually: making
decisions based on the available info, and the
residual uncertainty; making the most of the
decision, as well as knowing when things have
to change despite past decisions. Not a bad thing,
really. Life would be dead boring otherwise.
> You've done a good job thinking about what you want,
> and what's available. Good luck with your combina-
> torics.
Glad you think so. I know many who would have thought
it was too much digging to make the decision, but now I
have a guiding "plan" if it can be called that. Hopefully
it will optimize the use of my time going forward. Now I
gotta run like the wind. Thanks for all your input.
Fred
Fred
Fred Ma
>
> Neither Pyrex nor Inline::C are going to be great for C++. They were
> both created by C programmers for C programmers (I'd like to see this
> change about Pyrex some day). BOOST::Python, on the other hand, is
> probably a good fit.
I just checked out Boost (visited there before based on a pointer
by Scott Meyers, author of a book on the STL). I wasn't quite
prepared to put the time into getting to know libraries at
that time, and I really needed to get to know basic standard
C++ library before delving into expansions of them, but the
example for the python interface looks very cool. My current
use of a "P..." type scripting language, though, wouldn't be to
replace my matlab wrapper around C++ (the main engine until I
graduate). It would be to mulch data to get it in a palatable
form for the engine code. It's good to know about the python
interface, though, for future investigation. Thanks.
Fred
Michele Simionato
> Some scientists are very successful with "slash-and-burn"
> techniques. That manifests itself in this area as "con-
> science-free" coding: they write whatever programs get
> them the results they're after, and don't care that no
> one else can reproduce their results, nor can they them-
> selves, six months later. A LOT of that goes on.
>
> It makes me uncomfortable. A lesson I've learned over
> and over is that computer programs live far longer than
> you expect. It pays to do 'em better at the beginning,
> 'cause you're likely to live with them quite a while.
>
> I'm not a successful scientist, though; odds are long
> against me ever being one.
>
The relation between programming and science is rather complex.
There are still lots of sciences where the programming part is
very minor and the "slash-and-burn" technique is the right thing
to do. For instance, I have worked with very successful scientists
using "slash-and-burn". Their requirements were the following:
1. we want to spend at most 5% of our time doing coding and 95% doing
science;
2. our group is composed by two persons and our code will never be
released;
3. if a Ph. D. student reach the group and has to study the code, we
will explain it to him in person;
4. the largest program we will ever write is a thousand lines long;
for the rest we will use standard routines and write as little code
as possible;
5. we don't care at all about fancy graphics: the output will be a list
of numbers write on a file and displayed by gnuplot;
6. all we care is numerical performance in floating point computations;
the logic of the program will be trivial.
They actually used old fashioned Fortran 77 and I think they made the perfect
choice given their requirements; they essentially had to do numerical
integrations and solving differential equations, tasks were Fortran is
incredibly strong. Other persons I know doing particle Physics and doing a
good deal of programming (good deal means something like 50% of their time
spent on programming; spending more than 50% means that you are actually
a programmer, and scientists don't want to do that) were using a mixture
of Fortran and C. Here I am referring to people doing numerically
intensive tasks; I have seen a trend to switch to C++ for large projects,
since there is a perception than C++ scales better than Fortran. In
any case, the core routines are still in Fortran and always will be.
For scripting tasks, scripting languages are used; I know a few
astronomers who were using Perl to control the telescope movement.
Many physicists/astronomers use Python too. Specialized languages
and tools (Mathematica, Maple, Matlab, Octave, Yorick, etc. etc.)
have their place.
Finally, it may be weird in XXI century, but there are still scientists
not doing programming. For instance, the only programming language I used
for my scientific research was Latex, to write the papers; my reseach tools
were paper and pencil and blackboard, which are still as effective as
ever.
Michele Simionato
Ville Vainio
>> Since you happily use Perl, Python or Tcl, I have a
>> question. How do you decided which one to use? :-)
>>
>> Robert
Cameron> Most often, whatever is already at hand. It'll be
Cameron> interesting some day to come across an organization that
Cameron> doesn't already have a standard in place.
Many organizations don't, esp. if the actual product they produce is
not written in either of the languages. Some code may exist, but there
is no "official" standard. And if there is, there is a good
possibility of changing it if you are able to demonstrate clear
technical superiority in skunkworks projects. And w/ Python, you
are. Been there, done that :-).
--
Ville Vainio http://tinyurl.com/2prnb
Fred Ma
I want to start by emphasizing that I don't disagree with you.
I've spent a great deal of a previous degree in solution
methods for couple PDEs, as well as problem formulations that
come up with PDEs. I know of the tools that you mentioned.
I would like to point out that many science problem are not
as much numerically based. Many of them can be regarded still
as "physically" based science. For example, physical design
phase of digital circuit design relies on optimization
algorithms, albeit applied to a specialized class of problems.
There are genetic algorithms that evolve neural nets for
various applications (many of them physically based). These
less numerical applications use more abstraction in computational
models, so C++ has a place for these.
Of course, it is arguable that the computation models and
methods are a separate thing from the physical problems
that motivate them. So it all depends on how each person
interprets "science".
Fred
Charlton Wilbur
CL> Try 'em both briefly, choose one that makes a
CL> good impression, and get back to your engineering work.
CL> You'll be glad you upgraded to a 'P' language.
You, sir, have written a clear, fair-minded, and reasonable post. The
Backbone Cabal will be revoking your Usenet license shortly.
(I concur, by the way: Python has a couple of traits that would drive
me, with close to 10 years of Perl experience, bonkers; but they don't
drive everyone bonkers, and they obviously suit some people, or they
wouldn't be in the language. The OP ought to spend a couple weeks
playing with each language and pick the one that seems to suit his
style better.)
Charlton
--
cwilbur at chromatico dot net
cwilbur at mac dot com
Aahz
Fred Ma <f...@doe.carleton.ca> wrote:
>
>On the topic of speed, It's surprising to hear that this can be
>possibly rivaled by Perl/Python, considering that even my matlab code
>is about 10x slower than C++. That's with extensive profiling and
>round-about coding styles to exploit vectorization tricks, and no such
>effort in the C++ code.
The reason people say this is because algorithmic experimentation is
cheap in Python/Perl. An O(NlogN) algorithm in Python/Perl will lose to
an O(N^2) algorithm in C/C++ with a few elements, but as the data set
grows, the O(NlogN) algorithm will pull ahead.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"usenet imitates usenet" --Darkhawk
Fred Ma
>
> In article <40661C14...@doe.carleton.ca>,
> Fred Ma <f...@doe.carleton.ca> wrote:
> >
> >On the topic of speed, It's surprising to hear that this can be
> >possibly rivaled by Perl/Python, considering that even my matlab code
> >is about 10x slower than C++. That's with extensive profiling and
> >round-about coding styles to exploit vectorization tricks, and no such
> >effort in the C++ code.
>
> The reason people say this is because algorithmic experimentation is
> cheap in Python/Perl. An O(NlogN) algorithm in Python/Perl will lose to
> an O(N^2) algorithm in C/C++ with a few elements, but as the data set
> grows, the O(NlogN) algorithm will pull ahead.
The STL library brings in some efficiently implemented algorithms
to the C++ world instead of having programmings roll their own code.
The problem I saw with matlab was the overhead in between calling
such algorithms. For example, all the individual built-in's work
fast, but the moment you call your own function, you lose all the
speed that was painfully obtained by carefully crafted code. In
fact, if you rely on loops containing complex control flow, it
hinders matlab's acceleration technology (which vectorizes loop
operations). Same with many operators. In another response, I
mentioned that to ensure that one wouldn't run into similar
difficulties with Perl/Python would require a real-sized
application to be implemented and tested, with all the features
such as overhead between function calls -- a nontrivial task. So
that's put off as a possibility until after graduation, since
there's more than enough to try to achieve before then.
Anectdotal accounts are welcome, of course. Meantime, I'm
looking at Perl/Python for mulching design data to feed various
apps (notably, my algorithms, or to adequately change verilog
files to accommodate limitations in a tool like verilator, as
an example).
Fred
Fred Ma
Well, it looks like trying them both out in the long term is in fact
what I'll be doing. Thanks.
Peter Hickman
> I appreciate the sed terseness because
> I can make a command line pipe stage out of it, but
> the same brevity in a scripting language can make for
> rough going.
Both perl and python will allow you to do this. in perl
#!/usr/bin/perl
while(<>) { print uc $_; }
this script will read from stdin and write an uppercased version of the
input to stdout and can be used in a pipeline.
You can to the same in python and of course the guts of the loop can be
as big or small as you like. There is also a s2p tool that converts sed
scripts into perl, but rewiting would be a better bet as s2p can produce
some crufty code.
To be honest you could just flip a coin and go with that if you have no
specific requirements.
Perl may be easier to sell to your boss than python but if you just went
ahead with it you could use anything.
Robin Munn
> Hello,
>
> This is not a troll posting, and I've refrained from
> asking because I've seen similar threads get all
> nitter-nattery.
[snip: poster is basically asking "Should I learn Perl or Python?"]
While availability of libraries for your specific tasks is a major
factor to take into consideration, I'd say that even more important is
to figure out which language fits your brain.
I first learned Perl, then Python. I now use Python almost exclusively,
because I found that Python fits my mental "map" much better than Perl
does. To me, it seemed like in Perl, everything was a string. Sure, you
could have objects and classes, but they felt strange and awkward -- how
do I translate the "bless" keyword into my mental model? Whereas
Python's object model translated perfectly into my mental map, to the
point where programming in Python gave a clarity that I never had when
using Perl. It's hard to explain why, exactly -- it just seemed like
when using Perl, I was always going back down to the level of details,
the trees in the forest. Python allows me to look at the entire forest.
I can still go down and look at trees if I need to, but I don't have to
operate in that mode all the time.
Your mileage *will* vary. There's no substitute for experience. Set
yourself a simple task (say, writing a sieve of Eratosthenes program)
that you could do in about an hour with a language you're familiar with.
Then write the same program in Perl and in Python, and see which one
fits *your* brain better.
--
Robin Munn
rm...@pobox.com
Fred Ma
Hi, Peter,
My boss is my thesis supervisor. I'm going with Perl and keeping a
tab on Python. I expect to do most of my hacking in Perl, since I
see it around everywhere, composing in Python, and one-liners in
Perl/sed. About invoking Perl as part of a simple command in a
pipeline, I mean that one doesn't even have to write a script for it,
similar to sed
Command1 | sed -e Expression1 -e Expression2 ... | Command_N
This might also be possible in Python, in which case so much the better.
I'll find out (or someone might answer in a follow-up post). Thanks
for your view on the two.
Fred Ma
Well, my bout with Perl took much, much more than an hour.
It worked, though. It's probably not enough experience to
get a good look at the strength of Perl. For example, I
am a vim user (an editor), which is cryptic at first, but
let's you fly when you get to know it. I'm not saying that
all things cryptic are efficient in the end, just that a
brief bout won't always uncover the strengths. As a
"working model", I've decided to use Perl as the workhorse,
and keep tabs on Python as time permits. If my mental map
works better with Python, it's ease of ramping up will
allow the better fit to shine through despite the lesser
time I spend with it. Then it will naturally take a more
prominent role. Thanks for the advice on checking out both.
It seems to be the most realistic, and in line with what
many have suggested.
Cameron Laird
Fred Ma <f...@doe.carleton.ca> wrote:
.
.
.
>Well, my bout with Perl took much, much more than an hour.
>It worked, though. It's probably not enough experience to
>get a good look at the strength of Perl. For example, I
>am a vim user (an editor), which is cryptic at first, but
>let's you fly when you get to know it. I'm not saying that
>all things cryptic are efficient in the end, just that a
>brief bout won't always uncover the strengths. As a
.
.
Absolutely: a brief bout is a poor guide to long-term
strengths. The consensus of our follow-ups, though, is
this: Perl and Python both have so many, and so com-
parable, strengths "in the large", and they both have
such interesting cosmetics, that it *is* meaningful for
you to spend an hour or two and get a clear first impres-
sion of each. Some things (people, cultures, foods, ...)
can't be known at a first glance. When restricting your
attention to Perl and Python, a first glance *is* help-
ful.
Cameron Laird
>pipeline, I mean that one doesn't even have to write a script for it,
>similar to sed
>
> Command1 | sed -e Expression1 -e Expression2 ... | Command_N
>
>This might also be possible in Python, in which case so much the better.
>I'll find out (or someone might answer in a follow-up post). Thanks
.
.
# python -c "print 3 + 5"
8
Fred Ma
>
> In article <4069F1FD...@doe.carleton.ca>,
> Fred Ma <f...@doe.carleton.ca> wrote:
> .
> .
> .
> >Perl/sed. About invoking Perl as part of a simple command in a
> >pipeline, I mean that one doesn't even have to write a script for it,
> >similar to sed
> >
> > Command1 | sed -e Expression1 -e Expression2 ... | Command_N
> >
> >This might also be possible in Python, in which case so much the better.
> >I'll find out (or someone might answer in a follow-up post). Thanks
> .
> .
> .
> # python -c "print 3 + 5"
> 8
I was 99.99% sure that it was possible (heck, if tcsh and bash can do
it, python surely can). It's a little easier to squish it into a simple
command in a pipeline for a terse/compact "language" e.g. sed. On a
side-note, I've abused sed before by embedded sed scripts into bash
scripts without actually writing the sed script:
Bash_Command | \
sed \
-e 'Expression1' \
-e 'Expression2' \
-e 'Conditional1 {' \
-e 'Body_of_Conditional1' \
-e 'More_of_Body' \
-e '}' \
| More_Bash_Commands
Not that I want to do everything in sed, of course. And for
perl/python, there is probably not much reason to mix commands
with bash, since (unlike sed) they have the shell functionality.
I've put in a request to my sysadmin to install python. The
way to get hooked on it is to start small, with mundane tasks
(same as with any new shell, I suppose). If it doesn't get
installed due to lack of priority, I can always get it
bundled with my next cygwin update. Cygwin is an absolute
blessing. Everything comes prebuilt (lots of stuff) and
you have superuser priveleges, as well as simultaneous
access to Windows. Just as long as your HDD isn't FAT
(that prevents chmod from working).
But I digress.....
Cameron Laird
>way to get hooked on it is to start small, with mundane tasks
>(same as with any new shell, I suppose). If it doesn't get
>installed due to lack of priority, I can always get it
>bundled with my next cygwin update. Cygwin is an absolute
>blessing. Everything comes prebuilt (lots of stuff) and
>you have superuser priveleges, as well as simultaneous
>access to Windows. Just as long as your HDD isn't FAT
>(that prevents chmod from working).
.
.
Oh! Apparently Win* is the platform of interest; that
hadn't occurred to me.
There's a LOT to say specifically on the topic of Python-
for-Windows-system-administration. You might want to
start with Mark Hammond's book.
I'm no great fan of CYGWIN for my own use, although I
certainly recognize it's been historically important.
It occurs to me that perhaps we haven't made adequately
clear how easy it is to install native Python on Windows.
It should take less time doing it than talking about do-
ing it. I'm not arguing with your approach; I just want
to ensure we've been clear about the alternatives.
Follow-ups narrowed.