Typing system vs. Java
John Goerzen
I am currently familiar with Perl, C, and Java and am looking at
Python. I have some questions about its "strong-typedness" (or lack
thereof).
To me, the benefit of Java or Python over Perl comes into play with
large apps. For my purposes (no flames please!), Perl works great for
small scripts and stuff, but its almost total lack of typing,
prototyping, etc. causes major trouble with larger projects.
One nice thing about Java is that the programmer can control exactly
how tight the type-checking is and whether it occurs at runtime or at
compile time (dynamic vs. static). I can, for instance, write methods
that care naught about what is passed in (they can take an Object),
which essentially renders the type checking there little tighter than
Perl. I can write methods that return Objects, and the caller will
then have to cast them back to the appropriate type -- and a runtime
error will result if they're casted to the wront type.
Or, I could write methods that accept objects of a type anywhere up
and down the object hierarchy, being as specific or as general as
desired. The compiler will do as much checking as possible at compile
time. Usually I want it to do lots of checking, but sometimes it is
nice to be able to override.
From what I hear about Python, it is weakly-typed like Perl. There
are many times where I would like strong, static typing and some times
when I'd like to override that. Java has a lot of problems (its
bloated and inconsistent API is the largest) and I'd much rather use
some other language for large projects. However, it's valuable to me
that as many things as possible be caught before runtime. Do you
think that Python would be a worthwhile language to investigate given
these criteria? If so, how would I go about asserting type
compatibility at "compile time"? (Or as close as Python gets to that)
I want to be able to write methods that take only certain types of
objects (or their descendants). It is also useful to be able to cast
back from a general object to a more specific one (dynamic type
checking). Or it could be just basic things as enforcing passing an
int rather than a float or a string.
Thoughts?
Thanks!
--
John Goerzen <jgoe...@complete.org> GPG: 0x8A1D9A1F www.complete.org
bru...@tbye.com
> large apps. For my purposes (no flames please!), Perl works great for
> small scripts and stuff, but its almost total lack of typing,
> prototyping, etc. causes major trouble with larger projects.
Hi John,
Can you elaborate on this "major trouble" with some specific examples?
This is sort of straying from your original post but I've had people
tell me similar things before but I've yet to run into the problems they
feared. Maybe it's a knee-jerk reaction to all that sudden freedom and
flexibility that comes with a loosely-typed languaged?
In most jobs I've used C/C++ or Java and for me the stronger the typing
rules the more problems (and in turn bugs) it caused. I haven't sat down
to really ponder why but I have noticed a few recurring problems.
One is that stronger types introduce a lot more complexity. In Java I
spend a lot of time creating new classes just to act as simple data
structures. For example, if I need to store a list of points and color
values I can't just make an array of (x,y,color) tuples. I need to
create a class to hold them and then create instances and store them in
an array or Vector. And darn it, if I want to pull them out of the
Vector I have to cast them back to their class - even though both I and
the objects in the Vector already know what they are.
Now if I want those point/color tuples to sometimes have a color index and
other times use an RGB value, what do I do? Once again I just can't store
the correct value in there, I have to go and extend my class to take
either, or if the resulting classes are different enough I have to create
a whole new class and form both into some sort of class hierarchy. Or I
could have them both implement a common interface. Sheesh! Too often I've
seen rather convoluted class hierarchies or next-to-worthless interfaces
because there's some functions to which different objects need to be
passed so the language forces me to relate the classes in some way, even
when it doesn't make much sense. Forcing the developer to explicitly state
that common ground really adds to the complexity - too often it forces the
developer to make design decisions he/she is not really ready to make.
Strong typing seems to prevent a very small class of errors at great
expense, and none of the errors it prevents should escape good testing
(which you should have anyway).
With Python you can add any sort of checking that you want. You can have
your methods reject objects that aren't of a certain type or that don't
have certain methods/member variables, but I just haven't seen too many
cases where this is a problem.
Like I said, I haven't devoted a ton of time to thinking about why the
weaker typing of Python is actually a huge advantage to me in large
projects, but it seems that stronger types force you to define everything
in more exact terms, thereby adding complexity and forcing you to specify
your design in more detail than you may really have.
If nothing else, for large projects the Python versions are much smaller
and simpler, and as a result are usually less buggy and easier to manage.
-Dave
Christopher Barber
> However, it's valuable to me
> that as many things as possible be caught before runtime. Do you
> think that Python would be a worthwhile language to investigate given
> these criteria?
AFAIK there is no way to do compile-time type checking in Python.
If this is important to you, you might be interested in the Curl language. It
lets you omit type declarations when rapid prototyping or writing throwaway
code and add them later to get compile-time type checking and code
optimizations.
- Christopher
Chris Barker
> To me, the benefit of Java or Python over Perl comes into play with
> large apps.
I think Python is far superior to Perl for large projects, but because
is has nice support for clean syntax and modular programming, not
because of strong typing. The previous poster has a good discussion the
question of whether strong typing is as advantageous as you may think.
> From what I hear about Python, it is weakly-typed like Perl.
Python is dynamically typed, and NOT like Perl!
> are many times where I would like strong, static typing and some times
> when I'd like to override that.
As the previous poster mentioned, you can do type checking in your code
if you want to:
import types
def function(i,j):
if type(i) <> types.IntType or type(j) <> types.IntType:
raise TypeError("both input values must be integers")
#do something with i and j
Clearly this is something of a pain, so if you really think you are
going it a lot, you are using the wrong language for your problem and or
programming style.
> that as many things as possible be caught before runtime. Do you
> think that Python would be a worthwhile language to investigate given
> these criteria?
If compile time type checking is an important criteria: Python is not
your language! Even the above example is a runtime check.
> If so, how would I go about asserting type
> compatibility at "compile time"? (Or as close as Python gets to that)
Essentially, it can't be done.
My major thought is this: Compile time type checking is vastly over
rated: I suspect you won't miss it much, and the dynamicism of Python
will buy you much more that you lose: try it, you'll probably like it.
One more thought: it is important when trying a new language to learn
to program in a style appropriate to that language -- write Python that
looks like C (or JAVA), and you won't like the language.
As an example, my graduate advisor started using MATLAB, he like it at
first, but started out by re-writing a bunch of FORTRAN routines in
MATLAB. Soon he found that they ran really slow, and he never really go
the hang of writing MATLAB like MATLAB. After a couple of years, he went
back to using FORTRAN, and using MATLAB for interactive plotting only.
I, on the other hand, put some effort into learning MATLAB techniques,
and even re-wrote a number of his translated-from-FORTRAN routines in
more MATLAB native syntax, and they ran much faster. I used MATLAB for
years, and like it a lot. (Now I use Python + NumPy for most of what I
used to do in MATLAB, of course)
-good luck, Chris
--
Christopher Barker,
Ph.D.
ChrisH...@home.net --- --- ---
http://members.home.net/barkerlohmann ---@@ -----@@ -----@@
------@@@ ------@@@ ------@@@
Oil Spill Modeling ------ @ ------ @ ------ @
Water Resources Engineering ------- --------- --------
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
stalin
June Kim
combined with proper disciplines -- such as UnitTesting,
TestFirstProgramming
and RefactorMercilessly from XP. XP is one of the new agile methdologies,
which is quite a hot issue over the IT world.
Dave Thomas, who wrote the wonderful book "The Pragmatic Programmer", wrote
about the benefits of dynamic typing:
http://www.c2.com/cgi/wiki?DavidThomasOnTheBenefitsOfDynamicTyping.
And there are some other stuffs including hot discussions worth while to
read on the
same site:
http://www.c2.com/cgi/wiki?DynamicTypesEaseRefactoring,
http://www.c2.com/cgi/wiki?BenefitsOfDynamicTyping
My link is broken, but I believe you can find the forum lead by Rob C.
Martin
(aka Uncle Bob) at http://forums.itworld.com/, which discussed about cons
and
pros of dynamic typing (mainly on pros) and how to take advantage from it.
It might look like a deficiency that Python doesn't have static typing at
first, as it looked so to me, but you'll see it's rather liberating than
constraning, moreover, with strong confidence on the code.
Best wishes,
June Kim
ps. Python doesn't have compile-time type checking but you can substitute
it with unit tests if you want.
Tim Hammerquist
[snippage]
> > From what I hear about Python, it is weakly-typed like Perl.
>
> Python is dynamically typed, and NOT like Perl!
Could someone elaborate on this a bit more? What is the difference
here? What makes Perl's typing _not_ dynamic? I've heard this
statement often in clpy, but do not understand the semantics of the
argument.
However, Perl's typing system is much more flexible (read:
context-sensitive) than Python's.
[snippage]
--
"supported" is an MS term that means the function exists. The fact
that it always fails means, that it is an exercise for the programmer.
-- Sarathy, p5p
Cameron Laird
June Kim <junaf...@moc.oohay> wrote:
>First of all, Python's dynamic typing nature can be a definite godsend when
>combined with proper disciplines -- such as UnitTesting,
>TestFirstProgramming
>and RefactorMercilessly from XP. XP is one of the new agile methdologies,
[excellent references]
.
.
>My link is broken, but I believe you can find the forum lead by Rob C.
>Martin
>(aka Uncle Bob) at http://forums.itworld.com/, which discussed about cons
>and
>pros of dynamic typing (mainly on pros) and how to take advantage from it.
<URL: http://www.itworld.com/AppDev/1246/ITF010425meyer/ >
<URL: http://www.itworld.com/AppDev/1032/ITF010402meyerpromo/
are apparently all that's left. I encourage readers to holler
at ITworld.com for making things so hard to find.
>
>It might look like a deficiency that Python doesn't have static typing at
>first, as it looked so to me, but you'll see it's rather liberating than
>constraning, moreover, with strong confidence on the code.
on the same subject here in clp just within the last year.
.
.
.
I don't think anyone has yet made the point explicitly in
this thread that compile-time validations have been a topic
of interest in Pythonia for quite some time. Several Con-
ference talks and an entire SIG <URL:
http://www.python.org/sigs/types-sig/ > touch on the subject.
--
Cameron Laird <cla...@NeoSoft.com>
Business: http://www.Phaseit.net
Personal: http://starbase.neosoft.com/~claird/home.html
Mitchell Morris
news:mailman.996258291...@python.org:
[snip]
> In most jobs I've used C/C++ or Java and for me the stronger the typing
> rules the more problems (and in turn bugs) it caused. I haven't sat
> down to really ponder why but I have noticed a few recurring problems.
>
> One is that stronger types introduce a lot more complexity. In Java I
> spend a lot of time creating new classes just to act as simple data
> structures. For example, if I need to store a list of points and color
> values I can't just make an array of (x,y,color) tuples. I need to
> create a class to hold them and then create instances and store them in
> an array or Vector. And darn it, if I want to pull them out of the
> Vector I have to cast them back to their class - even though both I and
> the objects in the Vector already know what they are.
I'll go out on a limb and say that C++ and Java don't really have strong
typing, they merely have an unreasonable facsimile thereof.
Allow me instead to repeat some advice that once flew across this newsgroup:
the timbot said (and I'm paraphrasing here) "If you want to see what strong
static typing is supposed to feel like, try Haskell or OCaml." I've tried
both now, and have discovered that they are both truly pleasant to work
with, and are refreshingly free of all the why-do-i-have-to-type-a-cast-
when-we-both-know-damn-well-what-this-is events that otherwise plague C++
and Java development.
Haskell: <http://www.haskell.org/>
OCaml: <http://www.ocaml.org/>
[snip]
> If nothing else, for large projects the Python versions are much
> smaller and simpler, and as a result are usually less buggy and easier
> to manage.
>
> -Dave
otherwise-in-complete-agreement-ly y'rs,
+Mitchell
Christopher Barber
> In most jobs I've used C/C++ or Java and for me the stronger the typing
> rules the more problems (and in turn bugs) it caused.
I am really surprised to hear this. My experience has been the opposite.
> For example, if I need to store a list of points and color
> values I can't just make an array of (x,y,color) tuples. I need to
> create a class to hold them and then create instances and store them in
> an array or Vector.
You seem to be complaining about the lack of tuples, not static type-checking
per-se.
> Strong typing seems to prevent a very small class of errors at great
> expense, and none of the errors it prevents should escape good testing
> (which you should have anyway).
I am all for unit testing, but I know from experience that it is *extremely*
difficult to comprehensively test a code unit, and virtually impossible if you
do not have a good code coverage tool. If the compiler can catch a type error
for me at compile-time, that is at least one less unit test I have to write.
I agree that static type declarations add complexity to a language, but there
is a significant body of developers that really appreciate what they can do
for you. It would really be nice if Python could add some sort of optional
type declarations. It should be technically feasible.
- Christopher
Chris Barker
> > > From what I hear about Python, it is weakly-typed like Perl.
> >
> > Python is dynamically typed, and NOT like Perl!
>
> Could someone elaborate on this a bit more? What is the difference
> here? What makes Perl's typing _not_ dynamic? I've heard this
> statement often in clpy, but do not understand the semantics of the
> argument.
>
> However, Perl's typing system is much more flexible (read:
> context-sensitive) than Python's.
I didn't mean to say that Perl was not dynamicly typed. What I meant to
say was:
a) Python is dynamically typed, as opposed to weakly typed
b) Python's typing is not like Perl's typing, as you just said yourself.
Whether or not Perl's typing is best described as "dynamic" or "weak" is
a question I will leave to others that are more familiar with Perl than
I.
-Chris
Tom
"Christopher Barber" <cba...@curl.com> wrote in message
news:psog0be...@jekyll.curl.com...
> <bru...@tbye.com> writes:
>
> > In most jobs I've used C/C++ or Java and for me the stronger the typing
> > rules the more problems (and in turn bugs) it caused.
>
> I am really surprised to hear this. My experience has been the opposite.
I agree with you. Strong typing can be very useful. Its better to have the
compiler catch an error than to catch it during unit testing.
The ideal language would support both strong and weak typing.
Tom.
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com
Hamish Lawson
> > Python is dynamically typed, and NOT like Perl!
>
> Could someone elaborate on this a bit more? What is the difference
> here? What makes Perl's typing _not_ dynamic? I've heard this
> statement often in clpy, but do not understand the semantics of the
> argument.
I think what was meant was:
Python is dynamically BUT strongly typed, and NOT weakly typed like Perl!
Hamish Lawson
bru...@tbye.com
> > For example, if I need to store a list of points and color
> > values I can't just make an array of (x,y,color) tuples. I need to
> > create a class to hold them and then create instances and store them in
> > an array or Vector.
>
> You seem to be complaining about the lack of tuples, not static type-checking
> per-se.
Hmmm... no, because in Java, for example, to store something in the
container (be it tuple or whatever), the container would have to be
declared as one that can contain my type of object, and to get my object
back out I'd have to explicitly state what it is. For example, if you
want to store the integer 5 in a Java Hashtable, you can't just stick it
in there, instead you do a "new Integer(5)" to create an Object
descendent that can be stored in Hashtable. Then to get it out:
int foo = ((Integer)myHashTable.get(someKey)).intValue(); // bleh
What's the value-add of strong typing here?
> do not have a good code coverage tool. If the compiler can catch a type error
> for me at compile-time, that is at least one less unit test I have to write.
How so? The type checking imposed by compilers like Java don't seem to
solve any real problems - all they do is free the compiler from blame:
Vector v = new Vector();
String s = "hi";
v.addElement(s);
Integer i = (Integer)v.elementAt(0);
The compiler made me stick in the cast to an Integer, but it didn't solve
any problems for me (the code is still wrong). The only result is that the
compiler can now say, "Hey, it's not my fault - you're the one who cast it
to an Integer!".
Others on here have pointed out that using C++/Java as an example of a
strongly-typed language is a mistake, so maybe my opinion is closer to
"the typing systems in C++ and Java tend to cause me more work and
problems" and not strong, static typing.
> for you. It would really be nice if Python could add some sort of optional
> type declarations. It should be technically feasible.
I don't think the technical feasability is the main thing keeping them
out. Can you help me better see how the value outweighs the cost? (I'm not
saying the value is 0, only that for me the cost has always *seemed* to be
greater than the benefit.
-Dave
Peter Milliken
both strong and weak typing." go by :-). As soon as you allow "both" (and
damned if I know how you would do that! :-)), then your typical C/C++
programmer (unfair there, I should say "programmer") will choose the loose
typing everytime, because he "knows" what he means, even if the compiler
can't work it out, doesn't he? :-) So all the benefits that you just
acknowledged for strong typing are lost :-).
I have used a very strongly typed language (Ada), I haven't tried Haskall or
Ocaml as one repondent mentioned, but I claim that with Ada, 95-98% of
"normal" bugs are out of my programs once I get the compiler to produce
executable code. The only bugs that are left are the purely logic bugs. Once
you have used a truly strongly typed language, then you would never consider
doing a large job (multiple programmers) in anything else. I love and use
Python, but anything more than a one man job would see me turfing it for a
better suited language. Sometimes I think that even the idea of using Python
may be a mistake because the number of hours I have spent chasing down code
bugs in my Python programs that would have been caught by a decent typed
language compiler are too numerous to count :-).
Peter
"Tom" <tom-...@home.com> wrote in message news:3b65a...@Usenet.com...
Steven D. Majewski
On Mon, 30 Jul 2001, Chris Barker wrote:
> > However, Perl's typing system is much more flexible (read:
> > context-sensitive) than Python's.
>
> I didn't mean to say that Perl was not dynamicly typed. What I meant to
> say was:
>
> a) Python is dynamically typed, as opposed to weakly typed
> b) Python's typing is not like Perl's typing, as you just said yourself.
>
> Whether or not Perl's typing is best described as "dynamic" or "weak" is
> a question I will leave to others that are more familiar with Perl than
> I.
I would say that some of Perl's automagic coercions (IMHO) make it
weakly typed.
But I suppose someone could argue that it's stronly typed -- it just
has a very convoluted type system. But I've never seen anyone try to
chart out that type system explicitly. ( For one example, that would
involve tieing strings and numbers together into some sort of sub-type
relationship, since you can mix them together ( 'print "1" + 2' ) )
-- Steve Majewski
Neil Hodgson
> Sorry Tom, Just can't let the statement "The ideal
> language would support both strong and weak
> typing." go by :-). As soon as you allow "both" (and
> damned if I know how you would do that! :-)), then
> your typical C/C++ programmer (unfair there, I
> should say "programmer") will choose the loose
> typing everytime, because he "knows" what he
> means, even if the compiler can't work it out,
> doesn't he? :-) So all the benefits that you just
> acknowledged for strong typing are lost :-).
The way I see a strong/weak language working well is with separate layers
that use one or the other technique. In my work I'd like to see the low
level data structure layer using strong typing both for correctness and
performance. The higher level policy and UI layers would be weakly typed for
development speed. Currently, this can be achieved by using C++ for the
lower layers and Python for the upper but this separates the code bases too
much. An ideal language would allow development of a feature to start as
weakly typed and then be migrated to strongly typed when and if required.
Neil
Christopher Barber
> > > For example, if I need to store a list of points and color
> > > values I can't just make an array of (x,y,color) tuples. I need to
> > > create a class to hold them and then create instances and store them in
> > > an array or Vector.
> >
> > You seem to be complaining about the lack of tuples, not static type-checking
> > per-se.
>
> Hmmm... no, because in Java, for example, to store something in the
> container (be it tuple or whatever), the container would have to be
> declared as one that can contain my type of object, and to get my object
> back out I'd have to explicitly state what it is.
Ok. You are complaining about lack of tuples AND parameterized container
types.
> want to store the integer 5 in a Java Hashtable, you can't just stick it
> in there, instead you do a "new Integer(5)" to create an Object
> descendent that can be stored in Hashtable. Then to get it out:
>
> int foo = ((Integer)myHashTable.get(someKey)).intValue(); // bleh
>
> What's the value-add of strong typing here?
Nothing. It only does you good if your type system is good enough. If you
have to resort to runtime casts then it does not buy you anything.
> > do not have a good code coverage tool. If the compiler can catch a type error
> > for me at compile-time, that is at least one less unit test I have to write.
>
> How so? The type checking imposed by compilers like Java don't seem to
> solve any real problems - all they do is free the compiler from blame:
>
> Vector v = new Vector();
> String s = "hi";
> v.addElement(s);
> Integer i = (Integer)v.elementAt(0);
>
> The compiler made me stick in the cast to an Integer, but it didn't solve
> any problems for me (the code is still wrong). The only result is that the
> compiler can now say, "Hey, it's not my fault - you're the one who cast it
> to an Integer!".
This is a bad example because the code is doing runtime, not compile-time
checking. I was talking about the advantages of compile-time checking.
Here is how this would be written in Curl:
let v:{Array-of String} = {new {Array-of String}}
let s:String = "hi"
{v.append s}
let i:int = v[0] || ERROR: can't assign a String to an int
Besides providing compile-time type checking, this allows the compiler to
generate much better code than it could otherwise do without the type
information.
> Others on here have pointed out that using C++/Java as an example of a
> strongly-typed language is a mistake, so maybe my opinion is closer to
> "the typing systems in C++ and Java tend to cause me more work and
> problems" and not strong, static typing.
Perhaps. I also would not paint C++ with the same brush as Java since it also
has parameterized types. C++ is weak in other ways in that you can use casts
to subvert the type system.
> > for you. It would really be nice if Python could add some sort of optional
> > type declarations. It should be technically feasible.
>
> I don't think the technical feasability is the main thing keeping them
> out. Can you help me better see how the value outweighs the cost? (I'm not
> saying the value is 0, only that for me the cost has always *seemed* to be
> greater than the benefit.
The cost must be paid by whoever designs and implements such a feature. Users
would not have to use it if they don't want to. At least one benefit would be
that anal programmers like myself would be more willing to consider using
Python for critical projects. Adding type declarations would also allow
significant code generation optimizations.
- Christopher
bru...@tbye.com
On Tue, 31 Jul 2001, Peter Milliken wrote:
> I have used a very strongly typed language (Ada), I haven't tried Haskall or
> Ocaml as one repondent mentioned, but I claim that with Ada, 95-98% of
> "normal" bugs are out of my programs once I get the compiler to produce
> executable code. The only bugs that are left are the purely logic bugs.
Only 2-5% of your bugs are purely logical!? While I'd agree that once your
compiler produces executable code the bugs left are of the logical
variety, that number seems incredibly low (or the number of bugs you
initially have is ridiculously high ;-) ). Back to the topic: how much of
this would you attribute to Ada's use of types versus its very thorough
compiler (that checks a wide range of problems, not all of which may be
type-related).
> Once you have used a truly strongly typed language, then you would
> never consider doing a large job (multiple programmers) in anything
> else. I love and use Python, but anything more than a one man job
> would see me turfing it for a better suited language. Sometimes I
> think that even the idea of using Python may be a mistake because the
> number of hours I have spent chasing down code bugs in my Python
> programs that would have been caught by a decent typed language
> compiler are too numerous to count :-).
Excellent! You seem to have what I was looking for from the initial
poster: a specific example. Can you post one please? I'd really like to
better understand your perspective. Thanks!
-Dave
bru...@tbye.com
> > The compiler made me stick in the cast to an Integer, but it didn't solve
> > any problems for me (the code is still wrong). The only result is that the
> > compiler can now say, "Hey, it's not my fault - you're the one who cast it
> > to an Integer!".
>
> This is a bad example because the code is doing runtime, not compile-time
> checking. I was talking about the advantages of compile-time checking.
?? Please review the example - that's compile-time checking there.
> Here is how this would be written in Curl:
>
> let v:{Array-of String} = {new {Array-of String}}
> let s:String = "hi"
> {v.append s}
> let i:int = v[0] || ERROR: can't assign a String to an int
But what benefit have I gained? The only reason it's an error to begin
with is because the language is forcing me to name the type. The fact that
the compiler can detect my breakage of the language-specific rules doesn't
really advance the notion that strict compile-time checking is beneficial.
What is a real world problem that would not have existed if Python had
strict compile-time checking, and how common is such a problem? (I'm not
saying those problems don't exist - I'm just desperate for some examples
so I can understand the other side of the issue).
I guess my thinking is something like this: _if_ the language forces the
programmer to be very involved with details like type information, then
some sort of strict, compile-time checking is a Good Thing. _If_, however,
the programmer doesn't have to deal with those issues (or at least not to
the same degree), then many of the related problems go away (or are
present to a much smaller degree), and therefore the ability to check for
those problems is much less beneficial. For example, Python handles most
of the details surrounding memory usage, and as a result Python programs
are generally free from corrupted/lost memory problems, and features to
detect such problems just aren't worth as much.
The few times I've had this sort of conversation I've kind of felt that
people's reasoning was like "I use language X. Language X without strict
compile-time checking would make big programs a mess. Therefore, big
Python programs would be a mess."
I've seen big Python programs that are messy, but not much evidence that
the messiness is related to type problems (but sloppy programmers). I've
also seen big Python programs that are far easier to maintain and modify,
and have far fewer bugs, than what their equivalents would be in
"stricter" languages, so now I'm fishing for examples to show me where the
reverse is true.
-Dave
Donn Cave
[ ... re demonstration that a type checking language can check types ...]
| But what benefit have I gained? The only reason it's an error to begin
| with is because the language is forcing me to name the type. The fact that
| the compiler can detect my breakage of the language-specific rules doesn't
| really advance the notion that strict compile-time checking is beneficial.
| What is a real world problem that would not have existed if Python had
| strict compile-time checking, and how common is such a problem? (I'm not
| saying those problems don't exist - I'm just desperate for some examples
| so I can understand the other side of the issue).
Well, one currently poignant example. In Python, as you can hardly
fail to notice these days, division with integer inputs produces an
integer output, silently losing any fractional part. That's held to
be a common problem.
It can happen because Python will let an integer (or anything else!)
walk into that division, and try to make the best of what happens.
If you a type checking compiler, the object would have to agree at
compile time with the programmer's notion of what would be an
appropriate input.
I don't know if that means Python should have type checking. Its
open typed design works awfully well with OOP, and it would be a
shame to strangle that freedom for the sake of a half-baked concept
of discipline. But some kind of type inference validation could be
applied from a lint utility - that already exists and is reportedly
useful, but I don't know if it actually covers types. Anyway, maybe
it's an area where a big investment could yield something real
interesting for industrial strength programming.
Donn Cave, do...@drizzle.com
Peter Hansen
>
> If you a type checking compiler, the object would have to agree at
> compile time with the programmer's notion of what would be an
> appropriate input.
Sometimes the programmer's notion of "appropriate input" is
a little restrictive. Sometimes it is better not to have type-checking
at compile time, but to do it at run time. When a library routine
is used for a purpose the original author wouldn't have predicted,
maybe. Run-time assertions could check that the inputs have the
required characteristics, but those characteristics might not
be restricted to the simple notion of "type". (For example,
maybe it is better to allow any object which will _act_ as an
integer, rather than just integers. If only integers are to
be allowed, Python easily supports checking for this, at run-time.)
> I don't know if that means Python should have type checking. Its
> open typed design works awfully well with OOP, and it would be a
> shame to strangle that freedom for the sake of a half-baked concept
> of discipline. But some kind of type inference validation could be
> applied from a lint utility - that already exists and is reportedly
> useful, but I don't know if it actually covers types.
The problem with an external checking utility such as lint, or
the proposed division scanning utility, is that it only works
in a limited set of cases (although perhaps the majority of cases).
What it can't do is deal with some cases which are possible
because of the dynamic nature of Python. It may be only at
run-time that one can determine which inputs are actually being
fed to a subroutine, for example.
This is where testing comes in. A compiler can only check
whether you have followed the rules to which it restricts you.
Testing, however, can verify that, whether you followed the
rules or not, the code is doing what you think it should.
Testing is both necessary and sufficient for Python code.
With statically typed languages, testing is still necessary,
though probably done less often, while compile-time checking
is _not_ sufficient, yet that's all most people use.
In the end, whether or not Python suffers because of a
lack of compile-time checking is hardly the issue. Anyone
relying on such simplistic means to provide them with
confidence in their code's correct operation is deluding
themselves. Python just doesn't try to delude you: if
you want safe code, test.
> Anyway, maybe it's an area where a big investment could yield
> something real interesting for industrial strength programming.
I suppose you could say my group does "industrial strength"
programming with Python, since we're in industry :-) and the
strength of the code -- without compile-time type checking --
is quite adequate for now.
Of all our problems, well under 5% are related to problems
caused by Python's syntax, lack of compile-time type checking,
division behaviour, and other such things. Adding static types
(i.e. _requiring_ them) would cost us significant productivity
with next to no practical gain (since they would catch a small
fraction of less than 1% of our problems).
(Note: these numbers are anecdotal, unscientific, and made up.
Hah, some engineer! :-)
--
----------------------
Peter Hansen, P.Eng.
pe...@engcorp.com
Ralf Muschall
> I guess my thinking is something like this: _if_ the language forces the
> programmer to be very involved with details like type information, then
> some sort of strict, compile-time checking is a Good Thing. _If_, however,
> the programmer doesn't have to deal with those issues (or at least not to
> the same degree), then many of the related problems go away (or are
> present to a much smaller degree), and therefore the ability to check for
> those problems is much less beneficial. For example, Python handles most
This is only valid for very simple type systems.
In a really statically typed language, your example would go as follows
(I don't speak curl, probably my stuff is no valid curl syntax):
> let v = new
# for now, leave it to the language what type v is
> let s = "hi"
# the language sees that the type of s must be String
> {v.append s}
# from this line the language sees that the type of v must have been
# Array of String, since anything else would not fit.
> let i:int = v[0] # ERROR: can't assign a String to an int
True - if you want to protect yourself against mistakes, add explicit
type information. This is one of the purposes it is intended for - the
other one is to allow the compiler to decide all types at compile
time, thus avoiding boxed data, checks etc. at runtime.
Without the type info in the last line, the compiler would create an
i of type int.
Without giving any type hints, two things can happen in a statically
typed language: (1) The compiler solves the constraints and finds all
types, (2) it can't since there are more solutions and give an error
message (or assumes a default).
E.g. in hugs you get 2/3 -> 0.6666666667 (there are no typecasts at
all, it interprets the "2" and "3" as Double at read time, since
integers cannot be divided and the defaults tell it to assume Double
rather than Rational). (2::Rational)/3 gives the fraction 2/3
(printed as "2 % 3").
Ralf
Jonathan Hogg
<bru...@tbye.com> wrote:
> Others on here have pointed out that using C++/Java as an example of a
> strongly-typed language is a mistake, so maybe my opinion is closer to
> "the typing systems in C++ and Java tend to cause me more work and
> problems" and not strong, static typing.
Bingo. This is Java's fault for not having parametric polymorphism
rather than a problem with strong static typing per se. In my opinion
it's unfair to lump C++ in with Java since C++ at least has templates to
do this sort of thing.
Maybe I just haven't been writing large enough programs in Python, but I
don't feel that I hit many errors where I think "damn! I wish the
compiler had picked this up for me". I'd be interested in other people's
opinion on this. (And I've programmed plenty in Haskell so I know a
strong static typing system when I see one.)
I can though, think of plenty of places where strong static typing would
defeat intentional exploitation of the dynamic typing. Maybe this is
poor programming on my part, but I can't even think how one would type a
function like say:
def f(zs): return reduce( lambda x, y: x + y, zs )
Python's dynamic typing has the most optimal type for this function:
"A function that takes something looking like a sequence of
things that you can call "+" on, and returns something that
you might get back by "+"ing a bunch of such things together."
I just don't ever see Python with a static type system. What's more I
don't think I'd want it. If you want static types then code in something
else. If you find Java/C++ too horrific, then I'd seriously recommend
O'Caml - though this is just from a cursory look, I did most of my FP in
Haskell but I don't think I could recommend it unless you're willing to
become a serious FP-head and want to get to grips with Monads ;-)
Jonathan
Michael Abbott
D4822F.083...@news.easynet.co.uk:
> I can though, think of plenty of places where strong static typing would
> defeat intentional exploitation of the dynamic typing. Maybe this is
> poor programming on my part, but I can't even think how one would type a
> function like say:
>
> def f(zs): return reduce( lambda x, y: x + y, zs )
>
> Python's dynamic typing has the most optimal type for this function:
>
> "A function that takes something looking like a sequence of
> things that you can call "+" on, and returns something that
> you might get back by "+"ing a bunch of such things together."
>
> I just don't ever see Python with a static type system. What's more I
> don't think I'd want it. If you want static types then code in something
> else. If you find Java/C++ too horrific, then I'd seriously recommend
> O'Caml - though this is just from a cursory look, I did most of my FP in
> Haskell but I don't think I could recommend it unless you're willing to
> become a serious FP-head and want to get to grips with Monads ;-)
Interesting example. I for one would like to see a strongly typed variant
of Python: if for nothing else, I'd like to be able to compile my code.
Giving your function a type really boils down to giving the function + a
type, doesn't it? In Haskell, I guess we could type f thus:
f :: Num a => [a] -> a
but this is perhaps a little unsatisfactory, since the type a is then
required to support all the other numeric operations as well.
It would be nicer to be able to, for example, define:
class SemiGroup a where
(+) :: a -> a -> a
f :: SemiGroup a => [a] -> a
but this seems both clunky and really a bit misleading (since we don't
actually require the + we're using to be associative).
It's probably a good start to look at the type of reduce. This is easy
enough in Haskell:
reduce :: (a -> a -> a) -> [a] -> a
(need to cook up a more Python friendly syntax for this).
Ok, so the real problem is this: what is the type of + ?!
In Haskell, + is definitely of type
(+) :: Num a => a -> a -> a
which simply means that it is a member of the class Num (which also
contains -, * and some other stuff; seems to be an ordered Ring) and is a
binary operation which returns a value of the same type as its arguments.
The Haskell type model is too restrictive for Python, particularly since it
is not possible for any global name to have more than one type (so, for
example, we can *only* use + to name a method of the Num class). However,
I do think that some of the principles of Haskell typing can be profitably
applied to Python.
Let me just observe this: statically typed Python would need to be a
different language from standard Python, though hopefully close enough to
interoperate and have the same name.
Can anyone comment on the current status of ideas for statically typing
Python? The SIG mailing list is only carrying spam at the moment, and
several of the SIG links are broken, so things don't look promising...
Alex Martelli
news:9k4l5c$o9...@news1.gtech.com...
...
> Once
> you have used a truly strongly typed language, then you would never
consider
> doing a large job (multiple programmers) in anything else. I love and use
False! I *have* used "truly strongly typed languages" (such as Haskell),
yet I DO regularly consider doing large multi-programmers tasks in many
other languages (typically a mixture of languages: C++ for crucial low
level components and accelerators, SQL for the storage and retrieval
parts for which SQL is so eminently suitable, Python for 80% to 90% of
the code, i.e., everything which doesn't require C++'s speed and
can't benefit from SQL's specialized nature).
As the world is chock full of people who HAVE used "truly strongly
typed languages" (Pascal and variants, Ada, Eiffel, Sather, Haskell,
ML and variants, and so on) and then have moved on to, not just
"considering", but actively choosing, very different languages for
very large multi-programmers tasks (C++, Java, Python, Smalltalk,
Lisp, Erlang, and so on), your assertion is clearly false and quite
misleading as you've stated it. Recast it in the first person
singular, and it may correctly be taken as a reflection on _your_
particular quirks and foibles:-).
Alex
Alex Martelli
news:sLl97.75231$Xr6.3...@news-server.bigpond.net.au...
...
> much. An ideal language would allow development of a feature to start as
> weakly typed and then be migrated to strongly typed when and if required.
I believe that Dylan is the only existing language that offers this
approach, in non-LISP-ish syntax style, and with both free and
commercial high-quality implementations widely available. See:
http://www.gwydiondylan.org/
http://www.fun-o.com/
The Dylan books are also quite good, though very few in number,
due to Dylan never really making any significant widespread impact.
Reflecting on Dylan's utter lack of widespread impact tends to be
maddening to True Believers in Dylan or in the basic technological
choices it embodies (key one being "support both weak and strong
typing", a second crucial one being a highly sophisticated macro
system, a third very important one being multiple dispatch). In
my experience, most hardboiled Dylanistas have had to buy into a
"technical merit is never rewarded" mindset to avoid a painful
cognitive dissonance arising from this. I don't agree with such
a mindset, so my take on things is quite different -- at the very
least, though, one would have to agree that allowing both weak
and strong typing, with smooth migration between the two, is
not a very important feature in a language (although it DOES
sound lovely), given how little good (none) it did to Dylan:-(.
My basic take on things? That trying to design "one language
to bind them all" is a doomed pursuit; that multi-language
systems are here to stay, and to grow; trying to be all things
to all people makes a language too rich and complex, and the
complexity itself weighs it down. Maybe a language can afford
optional type-annotations and so on (I truly hope so, since
I'd love to have them), maybe it can afford multi-dispatching
(again, I sure HOPE it can!), maybe it can afford sophisticated
macro systems (and here, I start to have my doubts!), and so on,
and so forth... but striving to cover *all* the bases conflicts
inevitably with the SIMPLICITY that should be a very important
meta-goal of any language's design.
Alex
Peter Hansen
>
> I for one would like to see a strongly typed variant
> of Python: if for nothing else, I'd like to be able to compile my code.
How would that let you compile your code? Lack of compile-time
type checking is hardly the only thing making it difficult to
develop a Python compiler.
More to the point, why would you want to compile your Python code?
If for packaging reasons, there are effective alternatives in
the form of, for example, py2exe.
If for performance, the better approach is perhaps to profile your
code, then rewrite the performance-critical portions as C
extensions. Then you have pretty much the best of both worlds.
> Let me just observe this: statically typed Python would need to be a
> different language from standard Python, though hopefully close enough to
> interoperate and have the same name.
It would be interesting, perhaps, to see how well such a beast
would be adopted by the programming community. I can just see
the "marketing war" triggered as both sides try to promote
their own version of Python as the One True Way to program...
Alex Martelli
news:jonathan-D4822F...@news.easynet.co.uk...
> In article <mailman.996524288...@python.org>,
> <bru...@tbye.com> wrote:
>
> > Others on here have pointed out that using C++/Java as an example of a
> > strongly-typed language is a mistake, so maybe my opinion is closer to
> > "the typing systems in C++ and Java tend to cause me more work and
> > problems" and not strong, static typing.
>
> Bingo. This is Java's fault for not having parametric polymorphism
> rather than a problem with strong static typing per se. In my opinion
> it's unfair to lump C++ in with Java since C++ at least has templates to
> do this sort of thing.
True. On the other hand, C++ has really no way to let
you specify "a sequence of things of ANY types at all", which
IS sometimes useful (e.g., Python's lists:-) and you get in
Java with Vector (almost everything in Java descends from
Object, and you can easily wrap into Object-derived equivalents
those few fundamental types that don't, such as integer).
Even for "a sequence of *pointers* to things of any types",
vector<void*> doesn't really work as well as Java's Vector,
since you get NO *runtime* checks of type correctness as
you pull pointers out and cast them back as needed (C++
does have dynamic_cast for runtime checking, but it does
not apply to types that have no virtual-table -- only
classes with virtual methods get effective use of the
dynamic_cast and its runtime checking).
So, summarizing: the (relative) strengths and weaknesses of
the ways in which Java and C++ let you escape (some of) the
constraints of strong typing present trade-offs (Java PLUS
generics, as in good old Pizza, or, I'm told, some future
forthcoming Java enhanced version, will definitely be a big
step forward, though).
> Maybe I just haven't been writing large enough programs in Python, but I
> don't feel that I hit many errors where I think "damn! I wish the
> compiler had picked this up for me". I'd be interested in other people's
> opinion on this. (And I've programmed plenty in Haskell so I know a
> strong static typing system when I see one.)
I've often wished I could use Haskell typeclasses in a
Python program, but haven't been bitten particularly hard
by lack of compile-time checking in Python. Such a lack
does mean that many errors are caught a BIT later than
they should be -- a delay of between about one and about
ten minutes in most cases (when I'm being good and coding
test-first -- when I'm *NOT*, I have far worse problems
in my code than the typo-level errors a compiler could
catch:-).
The single kind of issue where I've found the highest
cost for lack of type declarations in Python, in my
experience so far, is with documentation of library
modules. When a docstring says or implies argument f
must be a file, I know that's probably not true, it just
needs to be a file-like object, but, HOW file-like does
it need to be? Does it have to implement, e.g., .read(N)
for an integer N, or will just .read() without arguments
suffice? (and so on, and so forth). When a docstring
says or implies argument L must be a list, I know that's
likely not true, it may well suffice for L to be some
kind of sequence, BUT, how closely does it have to mimic
list interface? Ditto for "a function" (it can probably
be any callable, but ...) and so on.
That's why I'm SO keen for the PEPs that propose that
Python add interfaces and protocol-adaptation concepts
to the language. They won't be *compile-time* checked
and I don't really care -- but they *WILL* formalize
the polymorphism kind and amount required and supplied
by library modules. That will save me lots of code
reading or experimentation that I now need to make best
use of some library modules...!-).
> I can though, think of plenty of places where strong static typing would
> defeat intentional exploitation of the dynamic typing. Maybe this is
Sure, and that's what happens when some [expletive deleted]
module author codes
assert type(a)==type(b)
and other similar cases of type-testing -- they break my
beautiful signature-based polymorphism into smithereens.
I tend to think VERY unkind thoughts each time I see such
a typetest (or even the slightly less-hateful isinstance).
> poor programming on my part, but I can't even think how one would type a
> function like say:
>
> def f(zs): return reduce( lambda x, y: x + y, zs )
I'd type it
def f(zs): return reduce(operator.add, zs)
as I *particularly* despise using lambda where any halfway
decent alternative presents itself, but that's quite
tangential to your issue, of course:-).
> Python's dynamic typing has the most optimal type for this function:
>
> "A function that takes something looking like a sequence of
> things that you can call "+" on, and returns something that
> you might get back by "+"ing a bunch of such things together."
It's somewhat subtler (each application of '+' must produce an
object such that you can again '+' it with the NEXT item of zs),
but that's more a problem in theory than in practice.
> else. If you find Java/C++ too horrific, then I'd seriously recommend
> O'Caml - though this is just from a cursory look, I did most of my FP in
> Haskell but I don't think I could recommend it unless you're willing to
> become a serious FP-head and want to get to grips with Monads ;-)
For some reason O'Caml keeps feeling to me like a modern-day
C++ -- a grabbag of every interesting concept the authors can
possibly think of, with peculiar quirky syntax to tie it all
together, and a lot of emphasis on run-time efficiency. Any
comparison with the sheer elegance of Haskell (its whitespace-
using clean syntax, its lazy-semantics default, its beautiful
typeclass concept...) seems to be out of place... although I
do understand that O'Caml can make much-faster code than
Haskell, and I do confess that monads always did stretch the
limits of my conceptualization abilities...
Alex
Alex Martelli
news:mailman.99653849...@python.org...
>
> On Tue, 31 Jul 2001, Peter Milliken wrote:
>
> > I have used a very strongly typed language (Ada), I haven't tried
Haskall or
> > Ocaml as one repondent mentioned, but I claim that with Ada, 95-98% of
> > "normal" bugs are out of my programs once I get the compiler to produce
> > executable code. The only bugs that are left are the purely logic bugs.
>
> Only 2-5% of your bugs are purely logical!? While I'd agree that once your
> compiler produces executable code the bugs left are of the logical
> variety, that number seems incredibly low (or the number of bugs you
What do you mean by "logical" in this context? On some keyboards
I have the '+' and '-' symbols on the same key (one shifted, one
unshifted), so I sometimes have a typo where I use '+' where I
mean '-' or vice versa. No compiler I know catches this kind of
bug. Yet it's clearly a common typo, not anything having to do
with conceptualization. Fortunately, unit-testing catches this
(and most other typos) quite effectively:-).
Alex
Michael Abbott
news:3B66B3A2...@engcorp.com:
> Michael Abbott wrote:
>>
>> I for one would like to see a strongly typed variant of Python: if for
>> nothing else, I'd like to be able to compile my code.
>
> How would that let you compile your code? Lack of compile-time
> type checking is hardly the only thing making it difficult to
> develop a Python compiler.
Hmm. Well, it's interesting to look at the byte code interpreter source
and to consider just how much time and effort is spent in working out what
the types of operands are and therefore which particular operation to
invoke. You can't avoid that without static typing of some sort.
> More to the point, why would you want to compile your Python code?
>
> If for packaging reasons, there are effective alternatives in
> the form of, for example, py2exe.
>
> If for performance, the better approach is perhaps to profile your
> code, then rewrite the performance-critical portions as C
> extensions. Then you have pretty much the best of both worlds.
Possibly, possibly. Perhaps I'm old fasioned, and too concerned about
efficiency. I know that processor speeds have increased by a factor of
1,000 or so in the last couple of decades or so, but it gives me the creeps
to see just how much code is executed to implement such simple expressions
as 1+2 or f(x).
I'm concerned to write efficient and secure systems which handle high
volumes of data reliably. I believe that Python has three weaknesses, but
unfortunately these weaknesses are also at the heart of its strength:
1. No static type checking means that a very large class of errors
(including undeclared identifier access) cannot be caught until run time.
2. Efficient compilation is not possible, because operators and function
calls cannot be compiled to anything simpler than the existing byte code
interpreter's current interpretation.
3. There is no data hiding.
>> Let me just observe this: statically typed Python would need to be a
>> different language from standard Python, though hopefully close enough
>> to interoperate and have the same name.
>
> It would be interesting, perhaps, to see how well such a beast
> would be adopted by the programming community. I can just see
> the "marketing war" triggered as both sides try to promote
> their own version of Python as the One True Way to program...
>
I think it's a pretty safe bet that most people prefer dynamic types.
Writing statically typed code is harder. I doubt that statically typed
Python would be a competitor: I see it as more of a niche solution.
Also, it's pretty clear that the three weaknesses I've just complained
about are simultaneously part of the strength of Python.
1. No static type checking means that it is not necessary to work out a
type description (which may be very complex) and it is easy to modify
existing code to change its behaviour. It also means, often, that when a
type error does occur there can be useful run time information about the
cause of that error.
2. The edit and run cycle of Python is very slick and easy to use.
3. Of course, the lack of data hiding means that developers have more
access to the internals of libraries, and there is no arguments about
whether certain methods should be hidden or exposed.
Finally, it is possible to do some very clever things with the dynamically
adaptive type structure of Python and it will be very difficult to capture
many of these features in a strict static type system.
bru...@tbye.com
> > def f(zs): return reduce( lambda x, y: x + y, zs )
> >
> f :: Num a => [a] -> a
>
> class SemiGroup a where
> (+) :: a -> a -> a
>
> f :: SemiGroup a => [a] -> a
>
> reduce :: (a -> a -> a) -> [a] -> a
>
> (+) :: Num a => a -> a -> a
This goes back to my orignal point: when the language forces you to be so
explicit with the type it sometimes requires you to give it more
information than you really have, and so you have to stop and invent a
pigeonhole in which you can place this thing. I suppose that once you get
it right then it could provide a sort of safety net, but too often
"getting it right" is complex and error-prone (or maintenance-prone, if
your original definition was too restrictive and you need to go back and
modify it to be more broad).
-Dave
Terry Reedy
"Michael Abbott" <mic...@rcp.co.uk> wrote in message
news:Xns90EF9A33C302...@194.238.50.13...
> Perhaps I'm old fasioned, and too concerned about efficiency.
You are not the only one :-) Practice worrying less.
> I know that processor speeds have increased by a factor of
> 1,000 or so in the last couple of decades or so, but it gives me the
creeps
> to see just how much code is executed to implement such simple
expressions
> as 1+2 or f(x).
When I first realize that side effect of the neat concept
"Everything's a PyObject" I was shocked. Fortunately, I was too well
hooked otherwise to turn back. I still occasionally remind myself
that even at Python speeds, a billion additions are much cheaper than
a few decades ago.
Terry J. Reedy
Michael Abbott
li...@python.org:
> On Tue, 31 Jul 2001, Michael Abbott wrote:
>
>> > def f(zs): return reduce( lambda x, y: x + y, zs )
>> >
>> f :: Num a => [a] -> a
>>
>> class SemiGroup a where
>> (+) :: a -> a -> a
>>
>> f :: SemiGroup a => [a] -> a
>>
>> reduce :: (a -> a -> a) -> [a] -> a
>>
>> (+) :: Num a => a -> a -> a
>
> This goes back to my orignal point: when the language forces you to be so
> explicit with the type it sometimes requires you to give it more
> information than you really have, and so you have to stop and invent a
> pigeonhole in which you can place this thing.
Yes, I think this is the price you have to pay if you want this kind of
type safety.
> I suppose that once you get
> it right then it could provide a sort of safety net, but too often
> "getting it right" is complex and error-prone (or maintenance-prone, if
> your original definition was too restrictive and you need to go back and
> modify it to be more broad).
I would like the option of this kind of safety net, particularly after I've
finished designing things in detail. In particular, working out the types
in detail can in itself be a helpful design tool.
Skip Montanaro
Michael> Well, it's interesting to look at the byte code interpreter
Michael> source and to consider just how much time and effort is spent
Michael> in working out what the types of operands are and therefore
Michael> which particular operation to invoke. You can't avoid that
Michael> without static typing of some sort.
Not so fast, bucko. ;-)
The prototypical proof-by-existence is Self. Check out
http://www.sun.com/research/self/
Also, who said you had to compile your code before running it? Armin Rego
has been working on something called Psyco (Python Specializing Compiler)
that at runtime converts VM bytecodes into another -- partially typed --
virtual machine based upon the types of the inputs it sees.
--
Skip Montanaro (sk...@pobox.com)
http://www.mojam.com/
http://www.musi-cal.com/
Christian Tanzer
Jonathan Hogg <jona...@onegoodidea.com> wrote:
> I can though, think of plenty of places where strong static typing would
> defeat intentional exploitation of the dynamic typing. Maybe this is
> poor programming on my part, but I can't even think how one would type a
> function like say:
>
> def f(zs): return reduce( lambda x, y: x + y, zs )
Genericity (templates in C++) this is not a problem -- you get one
instance of `f` for each type `zs` you happen to apply `f` to.
Disclaimer: C++ templates don't allow different return types for such
functions, but that's a restriction of C++, not of genericity. IIRC,
Ada generics can be generic in the return type, too.
--
Christian Tanzer tan...@swing.co.at
Glasauergasse 32 Tel: +43 1 876 62 36
A-1130 Vienna, Austria Fax: +43 1 877 66 92
Chris Barker
> > I for one would like to see a strongly typed variant
> > of Python: if for nothing else, I'd like to be able to compile my code.
I would like to see OPTIONAL static typing in Python. There are places
where it makes a lot of sense, and places where it doesn't. Personally,
I would probably only use it in small computational functions that I
could efficiently compile.
> More to the point, why would you want to compile your Python code?
> If for performance, the better approach is perhaps to profile your
> code, then rewrite the performance-critical portions as C
> extensions. Then you have pretty much the best of both worlds.
No you don't. The best of both worlds would be to profile my code, and
then statically type those few performance critical portions. This would
be a whole lot less work than re-writing those parts in C.
For performances, sake another nice addition would be the introduction
of the "homogenous" sequence. Python already has some of these: strings,
the library array module, NumPy arrays. They allow efficient computation
for methods and functions that know about them (and these are all
written in C), such as the string methods, the re module, and, of
course, NumPy Ufuncs. Unfortunatly, Python itself has no concept of the
homogenous sequence. If it did then a lot of optimization could be done
inside map calls, or list comprehensions, etc. As an example, look at
the following trivial example:
a = range(100)
b = range(100)
c = [(x**2 + y**2) for x in a for y in b]
In this case, both a and b are homogenous lists of integers. When the
list comprehension is called, every single time (x**2 + y**2) is
evaluated, the interpreter must check the type of x and y and do the
appropriate operation. If the interpreter knew that a and b were
homogenous, it would only have to do that check once, which could speed
up things a LOT. Of course you could use NumPy for this example, but
there are others where NumPy's array operations are not enough.
Even without any static type declaration, the homogenous sequence could
be possible. It would be pretty easy for tuples, since they are
immutable. Even for lists, at every append, the type of the appended
object could be checked, and the homogenous flag turned off if the type
doesn't match.
Does anyone else think this could be a good idea???
-Chris
--
Christopher Barker,
Ph.D.
ChrisH...@home.net --- --- ---
http://members.home.net/barkerlohmann ---@@ -----@@ -----@@
------@@@ ------@@@ ------@@@
Oil Spill Modeling ------ @ ------ @ ------ @
Water Resources Engineering ------- --------- --------
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
Christopher Barber
> On 30 Jul 2001, Christopher Barber wrote:
> > This is a bad example because the code is doing runtime, not compile-time
> > checking. I was talking about the advantages of compile-time checking.
>
> ?? Please review the example - that's compile-time checking there.
No it isn't! The (Integer) cast is not performed until run-time.
> > Here is how this would be written in Curl:
> >
> > let v:{Array-of String} = {new {Array-of String}}
> > let s:String = "hi"
> > {v.append s}
> > let i:int = v[0] || ERROR: can't assign a String to an int
>
> But what benefit have I gained?
You have detected a bug at compile-time instead of run-time and you didn't
have to write a unit-test to find it.
> The only reason it's an error to begin
> with is because the language is forcing me to name the type.
The language does not force you to specify types, this would also be legal,
but you would not see the problem until you actually executed the code:
let v = {Array}
let s = "hi"
{v.append s}
let i = v[0] || runtime error
It is your choice. If type declarations were added to Python in a similar
fashion, then it would not have to impose any burden on developers who don't
want to use them.
> The fact that the compiler can detect my breakage of the
> language-specific rules doesn't really advance the notion that
> strict compile-time checking is beneficial.
Sure it does. You might not believe that it is worth the trouble, but it
clearly has some benefits.
Another definite benefit is that knowledge of compile-time types can allow the
compiler to generate much better code. We know this is true from our
experience on the Curl language. Obviously it would be a lot of work for
Python to actually deliver this benefit, but it
> What is a real world problem that would not have existed if Python had
> strict compile-time checking, and how common is such a problem?
Here is a real-world problem: a developer wants to use a language with
compile-time type checking and rejects Python because it doesn't have it.
> The few times I've had this sort of conversation I've kind of felt that
> people's reasoning was like "I use language X. Language X without strict
> compile-time checking would make big programs a mess. Therefore, big
> Python programs would be a mess."
Well, I actually used Python on a number of projects a couple of years ago and
did spend more time than I would have liked writing tests to detect type
consistency. I found myself writing comments next to variables indicating
their intended type. I would much have preferred turning those comments into
actual type declarations.
I don't think that lack of static type declarations makes Python at all messy.
It is still an interesting and useful language. I just think that it would be
better with the ability to use static typing when you want to.
- Christopher
Steve Holden
news:3B66E4FA...@home.net...
[ ... ]
> Even without any static type declaration, the homogenous sequence could
> be possible. It would be pretty easy for tuples, since they are
> immutable. Even for lists, at every append, the type of the appended
> object could be checked, and the homogenous flag turned off if the type
> doesn't match.
>
> Does anyone else think this could be a good idea???
>
do numerical constant folding, and adds two integers together every time it
uses (2+3) there is clearly a long road to hoe before we see progress.
optimization-is-boring-when-there-are-other-development-choices-ly y'rs -
steve
--
http://www.holdenweb.com/
Courageous
>I would probably only use it in small computational functions that I
>could efficiently compile.
By and large this is nothing a good optimizing JIT can't handle
without static typing information. QKS is getting a 10-100X improvement
in Python execution speeds without modifying the Python language.
http:\\www.smallscript.com. Execution times within 1-2X of C.
>For performances, sake another nice addition would be the introduction
>of the "homogenous" sequence.
This I can agree with.
>Does anyone else think this could be a good idea???
Sure. Pretty easy to implement yourself, of course, saving the first-class
language support. Although one can imagine something like:
var = []!
C//
Alex Martelli
news:mailman.996592198...@python.org...
"""
> poor programming on my part, but I can't even think how one would type a
> function like say:
>
> def f(zs): return reduce( lambda x, y: x + y, zs )
Genericity (templates in C++) this is not a problem -- you get one
instance of `f` for each type `zs` you happen to apply `f` to.
Disclaimer: C++ templates don't allow different return types for such
functions, but that's a restriction of C++, not of genericity. IIRC,
"""
Beg pardon?
template<class A, class B>
A myfunction(const B& b)
{
return b;
}
Maybe you mean that C++ doesn't *automatically INFER* A when
you just call myfunction(something)? But, you CAN partially
specify myfunction<int>(something) [or, of course, totally
specify myfunction<int,double>(something)], if you have a
standard-compliant C++ compiler.
Back to Python -- programming in Python has always felt to
me a *LOT* like generic programming (e.g. with C++ templates)
in so many ways... of course, I was "steeped" in C++ generics
well before I met Python, so that may help explain this:-).
But, really, the only significant difference seems to me to
be that C++ instantiates its generics at compile-time (less
flexibility, some more syntax cruft, better runtime speed)
while Python works in a fully dynamic way, at runtime (some
errors are diagnosed a bit later, speed is not so great,
syntax is very lightweight and flexibility is nonpareil).
They're still very subjectively similar programming mindsets...
*signature-based polymorphism* dominates in either case!
Alex (Brainbench MVP for C++)
Chris Barker
> By and large this is nothing a good optimizing JIT can't handle
> without static typing information. QKS is getting a 10-100X improvement
> in Python execution speeds without modifying the Python language.
> http:\\www.smallscript.com. Execution times within 1-2X of C.
I have a hard time believing that this can be done without some sort of
type declarations, or homogenous sequences, or something. How do you get
around the fact that every time you see a:
z = x+y
x and y could be anything?
I've seen the smallscript reference before, but the only mention of
Python I see on that page is a link back to python.org. Does anyone know
where we can find info about this miracualous JIT compiler?
> >For performances, sake another nice addition would be the introduction
> >of the "homogenous" sequence.
> Sure. Pretty easy to implement yourself, of course, saving the first-class
> language support. Although one can imagine something like:
Yes, and that's what NumPy is. However, without the first class language
support, it is pretty limited.
Steve Holden wrote:
> It's a good idea, but given that the compiler we use daily doesn't yet even
> do numerical constant folding, and adds two integers together every time it
> uses (2+3) there is clearly a long road to hoe before we see progress.
Good point, but if it was in the language, someone might make use of it.
If nothing else, you could use it in extension modules. Maybe it would
get used in Py2C. (is that project dead? it seems to be sleeping, but I
thought it was pretty promising)
Skip Montanaro
C//> QKS is getting a 10-100X improvement in Python execution speeds
C//> without modifying the Python language.
I keep seeing this reference, but can find nothing but an "under
construction" page about Python on their website. Where's the beef?
Peter Mayne
news:mailman.996540777...@python.org...
> On 30 Jul 2001, Christopher Barber wrote:
>
> What is a real world problem that would not have existed if Python had
> strict compile-time checking, and how common is such a problem? (I'm not
int/int -> int, float/float -> float
8-)
PJDM
--
Peter Mayne
IBM GSA (but the opinions are mine)
Canberra, ACT, Australia
This post is covered by Sturgeon's Law.
Alex Martelli
<aa...@panix.com> wrote in message
news:2001073114...@panix2.panix.com...
> Mind if I repost this to another newsgroup? (alt.polyamory if you're
> curious.)
Not at all, though I'll admit to curiosity about the relevance to
alt.polyamory (I did use to be a regular there, but that was a
LONG time ago -- and anyway, now that Elf posts to
comp.lang.python, there are surely more prominent ex-a.p'ers
than me on c.l.p...?-).
Alex
bru...@tbye.com
> > On 30 Jul 2001, Christopher Barber wrote:
> > > This is a bad example because the code is doing runtime, not compile-time
> > > checking. I was talking about the advantages of compile-time checking.
> >
> > ?? Please review the example - that's compile-time checking there.
>
> No it isn't! The (Integer) cast is not performed until run-time.
Thank goodness! I'll just remove that there and ... doh! it won't even
compile anymore. Hmm... there's actually a little of both there,
compile-time _and_ runtime checking.
> > > Here is how this would be written in Curl:
> > >
> > > let v:{Array-of String} = {new {Array-of String}}
> > > let s:String = "hi"
> > > {v.append s}
> > > let i:int = v[0] || ERROR: can't assign a String to an int
> >
> > But what benefit have I gained?
>
> You have detected a bug at compile-time instead of run-time and you didn't
> have to write a unit-test to find it.
Take off your Curl glasses for just a moment and think Pythonically - that
is an error only because of the language-specific constraints - in Python
it's neither an error nor a problem.
My point is that, because Python is handling many of the details,
there are groups of problems that just aren't an issue here. For example,
a thorough assembly compiler might look at this code:
mov ax, 10
xor cx, cx
mov ax, 12
and say, "Hold on there! You're already using that register!", and issue a
warning. This does not, however, imply that any other language would
greatly benefit from this type of checking (except maybe internally). A C
compiler takes care of register allocation, and with it disappear a host
of related problems. Don't you think that it's possible that Python is
taking care of many of the type-related problems for you?
> The language does not force you to specify types, this would also be legal,
> but you would not see the problem until you actually executed the code:
Again, it's a problem _created_ by the constraints of the language, and/or
the division of labor between the language and the programmer. It doesn't
apply here (since we're supposedly exploring the weaknesses that _Python_
has without strict compile-time type checking.
> > The fact that the compiler can detect my breakage of the
> > language-specific rules doesn't really advance the notion that
> > strict compile-time checking is beneficial.
>
> Sure it does. You might not believe that it is worth the trouble, but it
> clearly has some benefits.
And those benefits are? I'm still waiting for anyone to be specific about
these benefits. Someone finally brought up the
semi-automatic-integer-to-float promotion problem and maybe the potential
for some modest performance gains (thank you!), but that's about it. Each
person has to judge the costs vs benefits, but that seems like a pretty
paltry list of "plusses".
A few people have asserted that big Python programs would be unmanageable
without this compile-time checking, but it seemed like they were speaking
from other-language, and not Python, experience. Of the people who
frequent the list and have participated in large Python projects, nobody
has stepped forward and said, "we really had problems, and boy
compile-time checking would have solved problems X, Y, and Z.".
The whole reason I jumped on this thread was because I've never run into
those problems at all (in Python), and I don't think I can chalk it up to
luck or Good Clean Living. :) I was hoping someone could help me overcome
my ignorance in this area, but so far the discussion has only reinforced
this idea:
Python's features and overall simplicity tend to eliminate or
significantly reduce whole classes of problems, including many
type-related problems. _Some_ strictly-,statically-typed languages
introduce artifical complexities that cause more problems than they
solve.
> > What is a real world problem that would not have existed if Python had
> > strict compile-time checking, and how common is such a problem?
>
> Here is a real-world problem: a developer wants to use a language with
> compile-time type checking and rejects Python because it doesn't have it.
This is pretty consistent with the whole body of arguments we've heard so
far: this feature is beneficial in other languages, so if Python doesn't
have it, Python is obviously lacking. To me that reasoning ignores the
fact that maybe, just maybe, there exists more than one solution to the
problem.
As languages evolve we generally find it a bargain to exchange a little
CPU for features that relieve developers of work and at the same time
eliminate developers' bugs. By moving from assembly to C the number of
bugs per _assembly_ line of code dropped dramatically. Is it too much to
imagine that the same does not hold true when moving on up the language
scale?
> Well, I actually used Python on a number of projects a couple of years ago and
> did spend more time than I would have liked writing tests to detect type
> consistency.
Wonderful! Can you please post some examples? Was this proactive testing
or was it triggered by problems you were seeing? If was the result of
problems, what were they?
-Dave
Donn Cave
...
| And those benefits are? I'm still waiting for anyone to be specific about
| these benefits. Someone finally brought up the
| semi-automatic-integer-to-float promotion problem and maybe the potential
| for some modest performance gains (thank you!), but that's about it. Each
| person has to judge the costs vs benefits, but that seems like a pretty
| paltry list of "plusses".
You couldn't have expected to get a long list of horror stories that
would stand your hair on end and send you off to the nearest strongly
typed language's site. Most of us aren't chafing under the excessive
freedom of Python, it works for us just like as it does for you, or
we wouldn't be here.
But the observer is part of the equation. Maybe our colleagues are
sick and tired of our Python programs blowing up all the time with
run time errors, and you should have asked _them_ to post examples
to c.l.p.
Anyway, it's all about errors, right? Does this cover it -?
- I don't make all that many errors.
- I make errors, but I can fix them whenever they turn up.
- I catch errors with run time tests.
- I catch errors with compiler tools.
- I need to catch more errors than I currently do.
Most of us probably hit every one to some degree. I think the point
may be, suppose you need to catch more errors than you currently do.
Maybe you perceive this after deploying your software, or maybe your
next project calls for exceptionally high standards of reliability.
So for sure, you need to step up the run time tests, but is that
enough? It's really hard to test _everything_ - I don't know,
take exception handling, typically involves the exception data
and that data isn't very well formed. Like socket errors that
usually have two args but once in a while only one.
Type checking would catch some of that, stuff you could have missed
otherwise. It sure could be overrated, and especially when (as usual)
not well designed and integrated in the language, but that would be
like saying OOP is a dud - after evaluating it in C++.
On a different note, explicit type annotations arguably make a program
easier to understand.
Donn Cave, do...@drizzle.com
Terry Reedy
"Courageous" <jkra...@san.rr.com> wrote in message
news:u4qdmtcsd0uj65qgl...@4ax.com...
This site has about zero to do with Python. Main page include Python
as (planned) secondary language. Contents links Python back to PEP
index page (bizarre). 'Python for AOS' page is under construction.
Where did you find fabulous Python compilation results you keep
reporting?
Terry J. Reedy
Tim Peters
> http:\\www.smallscript.com. Execution times within 1-2X of C.
[Terry Reedy]
> This site has about zero to do with Python. Main page include Python
> as (planned) secondary language. Contents links Python back to PEP
> index page (bizarre). 'Python for AOS' page is under construction.
>
> Where did you find fabulous Python compilation results you keep
> reporting?
I believe it originally came from this cryptic tease:
Python is a big part for the execution engine, this is a new area.
The Smalltalk VM runs Python 10 to 100x faster.
at
http://www.smalltalkconsulting.com/html/SmalltalkSolutions2001%232.html
first referenced in this c.l.py thread:
http://groups.google.com/groups?ic=1&th=b121faa695c8b6fa
I haven't seen more than that single second-hand claim (except for Joe's
charming belief that if he repeats it often enough, Guido will get pissed
off enough to make it a reality now and for free <wink>).
every-jit-project-is-at-least-a-year-late-and-severely-underperforms-
expectations-for-at-least-a-year-after-that-ly y'rs - tim
Peter Hansen
>
> A few people have asserted that big Python programs would be unmanageable
> without this compile-time checking, but it seemed like they were speaking
> from other-language, and not Python, experience. Of the people who
> frequent the list and have participated in large Python projects, nobody
> has stepped forward and said, "we really had problems, and boy
> compile-time checking would have solved problems X, Y, and Z.".
I'll say it again: in over a year of Python development by a team now
numbering over ten, *none* of the problems that I can recall were in
any way caused by a lack of compile-time checking in Python.
Or, for that matter, any other limitation of Python... Python has that
wonderful characteristic of all the best technologies: it's invisible. :)
Michael Abbott
news:mailman.996593278...@python.org:
>
> Michael> Well, it's interesting to look at the byte code
> interpreter source and to consider just how much time and
> effort is spent in working out what the types of operands
> are and therefore which particular operation to invoke.
> You can't avoid that without static typing of some sort.
>
> Not so fast, bucko. ;-)
>
> The prototypical proof-by-existence is Self. Check out
>
> http://www.sun.com/research/self/
Thanks for the reference; quite an intruiging looking language there. I
don't really see how this affects my point, though. I can see a number of
published papers on optimisation techniques, but most of them seem to be
about working out a static type inference regime, anyway!
>
> Also, who said you had to compile your code before running it? Armin
> Rego has been working on something called Psyco (Python Specializing
> Compiler) that at runtime converts VM bytecodes into another --
> partially typed -- virtual machine based upon the types of the inputs
> it sees.
Hmm. Well, with the current interpreter cost, I'm sure there's plenty of
head-room for small improvements.
The only real reference I could find for Psyco is this:
http://mail.python.org/pipermail/python-dev/2001-June/015503.html
which seems recent enough to be definitive. I'm afraid that a factor of
two improvement on the existing byte code iterpreter doesn't seem very
startling to me.
Michael Abbott
news:4pA97.55816$EP6.12...@news1.rdc2.pa.home.com:
>
> "Michael Abbott" <mic...@rcp.co.uk> wrote in message
> news:Xns90EF9A33C302...@194.238.50.13...
>> Perhaps I'm old fasioned, and too concerned about efficiency.
>
> You are not the only one :-) Practice worrying less.
Ha! ;/
>
>> I know that processor speeds have increased by a factor of 1,000 or
>> so in the last couple of decades or so, but it gives me the creeps to
>> see just how much code is executed to implement such simple
>> expressions as 1+2 or f(x).
>
> When I first realize that side effect of the neat concept
> "Everything's a PyObject" I was shocked. Fortunately, I was too well
> hooked otherwise to turn back. I still occasionally remind myself
> that even at Python speeds, a billion additions are much cheaper than
> a few decades ago.
Oh, I most definitely agree that Python is a big step forwards. I'm just
looking for two more things: I'd like some of that processing power back,
please; and I'd love more early static checking.
Michael Abbott
news:u4qdmtcsd0uj65qgl...@4ax.com:
>
>>I would probably only use it in small computational functions that I
>>could efficiently compile.
>
> By and large this is nothing a good optimizing JIT can't handle
> without static typing information. QKS is getting a 10-100X improvement
> in Python execution speeds without modifying the Python language.
> http:\\www.smallscript.com. Execution times within 1-2X of C.
Sounds very impressive, almost too good to be true...? Do you have a more
specific reference, please?
Michael Abbott
news:9966417...@yabetcha.sttl.drizzle.com:
> On a different note, explicit type annotations arguably make a program
> easier to understand.
I absolutely agree with this (except for the "arguably" bit :)).
Christopher Barber
> On 31 Jul 2001, Christopher Barber wrote:
>
> > > On 30 Jul 2001, Christopher Barber wrote:
> > > > This is a bad example because the code is doing runtime, not
> > > > compile-time checking. I was talking about the advantages of
> > > > compile-time checking.
> > >
> > > ?? Please review the example - that's compile-time checking there.
> >
> > No it isn't! The (Integer) cast is not performed until run-time.
>
> Thank goodness! I'll just remove that there and ... doh! it won't even
> compile anymore. Hmm... there's actually a little of both there,
> compile-time _and_ runtime checking.
Yes, but the actual type error that we are interested in here is that the
program needed an integer but got a string instead. That particular type
error is not detected until runtime in the Java example you provided and that
is why it is not a relevant example. You are absolutely correct that using
statically typing does not ensure that all type errors will be detected at
compile time, but it does catch many.
> > > > Here is how this would be written in Curl:
> > > >
> > > > let v:{Array-of String} = {new {Array-of String}}
> > > > let s:String = "hi"
> > > > {v.append s}
> > > > let i:int = v[0] || ERROR: can't assign a String to an int
> > >
> > > But what benefit have I gained?
> >
> > You have detected a bug at compile-time instead of run-time and you didn't
> > have to write a unit-test to find it.
>
> Take off your Curl glasses for just a moment and think Pythonically - that
> is an error only because of the language-specific constraints - in Python
> it's neither an error nor a problem.
I used Curl as an example, not only because I know it well but because I know
that what could be done in Curl could also be added to Python.
As to the error, this would indeed be an error in Python as well but it would
happen later in the program when some code tries to use 'i' as an integer
instead of a string.
> Don't you think that it's possible that Python is
> taking care of many of the type-related problems for you?
Sure. By providing consistent dynamic typing and disallowing unsafe casts it
can be considered superior to C/C++
> > The language does not force you to specify types, this would also be legal,
> > but you would not see the problem until you actually executed the code:
>
> Again, it's a problem _created_ by the constraints of the language, and/or
> the division of labor between the language and the programmer. It doesn't
> apply here (since we're supposedly exploring the weaknesses that _Python_
> has without strict compile-time type checking.
I don't see why it doesn't apply. If I am forced to do more work because
Python doesn't do it for me, then why can't I consider that to be a weakness
in the language?
> > > The fact that the compiler can detect my breakage of the
> > > language-specific rules doesn't really advance the notion that
> > > strict compile-time checking is beneficial.
> >
> > Sure it does. You might not believe that it is worth the trouble, but it
> > clearly has some benefits.
>
> And those benefits are? I'm still waiting for anyone to be specific about
> these benefits.
Then you are not paying attention.
1) Static type declarations allow the compiler to catch some type errors
that would not otherwise be detected until run-time. There is no
question about this fact. You might not think the benefit is significant
enough to be worthwhile, but that is a different issue.
2) Static type declarations allow the programmer to clearly express
their intent. In this role they are only a shade better than comments.
3) Compile time type information is necessary to generate fast code. It
must either come from type declarations or must be inferred. Inferring
types in the absence of any static type declarations is difficult and
requires analysis of the entire program; this analysis is at best
O(n log n) but is more likely to be O(n^2) -- in other words it could
become too expensive for large programs.
> A few people have asserted that big Python programs would be unmanageable
> without this compile-time checking, but it seemed like they were speaking
> from other-language, and not Python, experience.
I was not one of those people. I have worked on a large project using almost
300k lines of Tcl code (a truly horrendous language) so I am quite sure that a
Python project of this side would be feasible. However, I would have concerns
about the performance and testability of such a large Python application.
> Python's features and overall simplicity tend to eliminate or
> significantly reduce whole classes of problems, including many
> type-related problems. _Some_ strictly-,statically-typed languages
> introduce artifical complexities that cause more problems than they
> solve.
I agree with this.
> > > What is a real world problem that would not have existed if Python had
> > > strict compile-time checking, and how common is such a problem?
> >
> > Here is a real-world problem: a developer wants to use a language with
> > compile-time type checking and rejects Python because it doesn't have it.
>
> This is pretty consistent with the whole body of arguments we've heard so
> far: this feature is beneficial in other languages, so if Python doesn't
> have it, Python is obviously lacking. To me that reasoning ignores the
> fact that maybe, just maybe, there exists more than one solution to the
> problem.
No, I was just trying to make a point that the Python audience is smaller than
it might otherwise be.
> As languages evolve we generally find it a bargain to exchange a little
> CPU for features that relieve developers of work and at the same time
> eliminate developers' bugs. By moving from assembly to C the number of
> bugs per _assembly_ line of code dropped dramatically. Is it too much to
> imagine that the same does not hold true when moving on up the language
> scale?
Not at all. The question is how many cycles are you willing to spend and when
are you willing to spend them (e.g. compile-time vs. run-time).
> > I actually used Python on a number of projects a couple of years ago and
> > did spend more time than I would have liked writing tests to detect type
> > consistency.
>
> Wonderful! Can you please post some examples? Was this proactive testing
> or was it triggered by problems you were seeing? If was the result of
> problems, what were they?
That was a couple of jobs ago and I don't have access to that code anymore.
One problem I remember having was that I was writing library code and had to
write explicit run-time type-checking code to check the types of incoming
arguments. It was not difficult to do, but it was extremely tedious. If I
could even have had the ability to put type declarations in function
signatures and have Python generate the run-time type checks I would have been
much happier.
I also found that I had to write longer unit tests for my Python code than for
my C++ code.
I am sorry that I cannot be more specific.
BTW, is there a code coverage measurement tool for Python yet?
- Christopher
Simon Brunning
> BTW, is there a code coverage measurement tool for Python yet?
<http://musi-cal.mojam.com/~skip/python/trace.py>.
Cheers,
Simon Brunning
TriSystems Ltd.
sbru...@trisystems.co.uk
-----------------------------------------------------------------------
The information in this email is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this email by anyone else
is unauthorised. If you are not the intended recipient, any disclosure,
copying, distribution, or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. TriSystems Ltd. cannot
accept liability for statements made which are clearly the senders own.
Skip Montanaro
>> BTW, is there a code coverage measurement tool for Python yet?
Simon> <http://musi-cal.mojam.com/~skip/python/trace.py>.
Actually, my trace.py module is now part of the Python distribution,
available at
.../Tools/scripts/trace.py
Skip
Alex Martelli
news:pso66c9...@jekyll.curl.com...
...
> > What is a real world problem that would not have existed if Python had
> > strict compile-time checking, and how common is such a problem?
>
> Here is a real-world problem: a developer wants to use a language with
> compile-time type checking and rejects Python because it doesn't have it.
Why is this "a problem"? Counterexample: a developer wants
to use a language with an 'e' in the language's name and rejects
Python because it has no 'e' in its name. Should we therefore
rename Python to 'Pethon' to "solve this problem"?
Each developer or development organization can freely choose
whatever constraint he/she/it/they prefer regarding their
programming tools. If this it a problem, it's *HIS* (her,
its, their) problem, not Python's. It would be silly and
self-defeating if Python tried to match such constraints just
because they are posed as constraints.
If it's worth for Python to have an 'e' in its name, it must
be because using that letter has intrinsic value within the
logic of Python, so, you should be arguing about that
intrinsic value, not about "marketing" advantages that
may stem from having it, if you want to speak "real world".
Marketing, by definition, isn't about the real world - it's
about successfully projecting a fantasy in the minds of
people with deep-enough pockets to make you rich:-).
> Well, I actually used Python on a number of projects a couple of years ago
and
> did spend more time than I would have liked writing tests to detect type
> consistency.
Probably no more time than I spend cursing [expletive deleted]'s who
write type-tests. They THINK they care if a isA b... I'd SO like to
rub their faces hard against all the collected works of General
Semanticists and make them foreswear the cursed "is of identity"!
"BEHAVES-LIKE-A" (i.e. IS_ADAPTABLE_TO_PROTOCOL) is what, at most,
one SHOULD be testing (if anything). Alas, the protocol adaptation
PEP is languishing. In 90 case of 100, though, "just try and catch
the exceptions if any" will do basically just as well.
> I found myself writing comments next to variables indicating
> their intended type. I would much have preferred turning those comments
into
> actual type declarations.
I fully agree that executable, formalized code is VASTLY superior
in all ways to comments. But the point is that one shouldn't be
asserting IS-A, even in comments, and that's what types ARE (as
opposed to protocols, aka interfaces: the confusion between the
two is, of course, rampant).
> I don't think that lack of static type declarations makes Python at all
messy.
> It is still an interesting and useful language. I just think that it
would be
> better with the ability to use static typing when you want to.
I suspect that the loss of dynamic-ity needed to *ensure* the
'typing' was STATIC would totally kill Python, disfiguring it
beyond all recognition. If a USEFUL compiler-recognizable
way to assert protocol-conformance existed, then theoretically
it could in SOME (but NOT all!) cases be hoisted up to compile
time (just like, e.g., in print 2+2, the addition COULD be
done at compile time -- although Python doesn't bother doing
so, there's no *theoretical* impediment to doing it). Just
like you could do now with
assert isinstance(foo, bar)
[except that you *SHOULDN'T*!] -- it's theoretically possible
(although NOT in all cases -- and in practice not done by
the Python compiler) to sometimes hoist such assertions up
to compiletime.
It's quite a different issue to dream about a way to assert
'anything that can ever be possibly bound to this variable
X must respect the following constraint' (it makes no
difference whether the constraint is isinstance, or if
it's something actually USEFUL). Conceptually, that would
require the assert equivalent to be inserted at each point
where X *might* be rebound (which might then be hoisted up
to compile-time in some cases). Seems awfully hard to me
in the general case.
Alex
Christian Tanzer
"Alex Martelli" <ale...@yahoo.com> wrote :
> "Christian Tanzer" <tan...@swing.co.at> wrote in message
> news:mailman.996592198...@python.org...
> """
> > poor programming on my part, but I can't even think how one would type a
> > function like say:
> >
> > def f(zs): return reduce( lambda x, y: x + y, zs )
>
> Genericity (templates in C++) this is not a problem -- you get one
> instance of `f` for each type `zs` you happen to apply `f` to.
>
> Disclaimer: C++ templates don't allow different return types for such
> functions, but that's a restriction of C++, not of genericity. IIRC,
> """
>
> Beg pardon?
>
> template<class A, class B>
> A myfunction(const B& b)
> {
> return b;
> }
>
> Maybe you mean that C++ doesn't *automatically INFER* A when
> you just call myfunction(something)? But, you CAN partially
> specify myfunction<int>(something) [or, of course, totally
> specify myfunction<int,double>(something)], if you have a
> standard-compliant C++ compiler.
You're right, I was mixing things up. The restriction I remembered is
related to overload resolution not to templates.
Shows that I'm slowly recovering from my past immersion in C++ <wink>.
The traces of obscure bits of knowledge fade. I should refrain from
making passing comments about C++ arcana in the future, I guess.
> Back to Python -- programming in Python has always felt to
> me a *LOT* like generic programming
Yup. Only, in Python it's very implicit.
The nice thing about Ada and Eiffel generics is that the requirements
on the generic parameters are spellt out explicitly. For most
application areas, Python still wins hands down, though.
Courageous
>I haven't seen more than that single second-hand claim (except for Joe's
>charming belief that if he repeats it often enough, Guido will get pissed
>off enough to make it a reality now and for free <wink>).
Oh, God. I've been found out. I think I better go run and hide now. :-)
C//
bru...@tbye.com
> > > You have detected a bug at compile-time instead of run-time and you didn't
> > > have to write a unit-test to find it.
> >
> > Take off your Curl glasses for just a moment and think Pythonically - that
> > is an error only because of the language-specific constraints - in Python
> > it's neither an error nor a problem.
[snip]
> As to the error, this would indeed be an error in Python as well but it would
> happen later in the program when some code tries to use 'i' as an integer
> instead of a string.
This is exactly why this compile-time type checking is ill-suited for
Python. The error is _not_ at the time of the "assignment", but later.
*If* the error occurs it will happen when the program attempts to use that
value in an incorrect way. The difference between the two is quite
significant. In Python there really isn't any assignment per se, but the
creation and deletion of references to values. Variables don't have types,
the values do, and attempting to check the types of the variables makes
far less sense.
Putting all that aside and assuming that the programmer goes on to try and
use an integer value as a string in some unsupported way, will that error
hide any longer than the very first time that piece of code is executed?
Probably not. And when it rears its ugly head, how will it manifest
itself? As an exception that includes the line number of the error. And
since there's no explicit compile step in Python, I probably would find
and fix that problem in Python before the Java version's compiler even
finished compiling! <g> The issue at hand is whether strict compile-time
type checking adds enough value to warrant their inclusion - I don't see
significant savings here.
Again, I think this goes back (at least in part) to programming in Python
while using a C++/Java/whatever mindset. For example, if you don't check
for null in a C program, your program may die in bizarre ways, so it's a
common and good practice to always check for null pointers. When you move
to Java you'll bring with you the same tendency to always check them, but
very often this doesn't actually add much value (if you check and raise an
exception on null, this isn't much different than what the VM will do on
its own if you don't perform the check).
> > > The language does not force you to specify types, this would also be legal,
> > > but you would not see the problem until you actually executed the code:
> >
> > Again, it's a problem _created_ by the constraints of the language, and/or
> > the division of labor between the language and the programmer. It doesn't
> > apply here (since we're supposedly exploring the weaknesses that _Python_
> > has without strict compile-time type checking.
>
> I don't see why it doesn't apply. If I am forced to do more work because
> Python doesn't do it for me, then why can't I consider that to be a weakness
> in the language?
Sure, *if* it forces you to do more work.
> question about this fact. You might not think the benefit is significant
> enough to be worthwhile, but that is a different issue.
Uh, sorta. Maybe it's time for this thread to die, but for a little while
I thought we were judging cost vs benefits. If so, then magnitude of
cost/benefit is of utmost importance. I'll readily agree (again) that in
some cases there can exist some benefit for this kind of type checking,
but it also seems that this benefit is almost never significant in Python.
> 2) Static type declarations allow the programmer to clearly express
> their intent. In this role they are only a shade better than comments.
Right. My assertion is that this causes more problems than it solves
because you seldom know your "intent" in that much detail, much less your
future intent. Seems like ultra strict type checking might work well in
cases where you have very precise requirements and once built, your
software won't be changing any time soon (a la space shuttle software).
Most programs don't fall into that category.
> One problem I remember having was that I was writing library code and had to
> write explicit run-time type-checking code to check the types of incoming
> arguments.
But why? Anyone using the library would catch on pretty quickly if they
were passing in bogus values. Also, that sort of type-checking tends to
diminish the usefulness of the code (you allow someone to give you a
dictionary but you prohibit an object that looks and acts like a
dictionary). Once again, I can't help but feel that this is more a
misapplication of other-language-Best-Practices that don't necessarily fit
with Python than response to specific problems one encounters in
Python.
> BTW, is there a code coverage measurement tool for Python yet?
Tools/scripts/trace.py
-Dave
Christopher Barber
> "Christopher Barber" <cba...@curl.com> wrote in message
> news:pso66c9...@jekyll.curl.com...
> > Well, I actually used Python on a number of projects a couple of years ago
> and
> > did spend more time than I would have liked writing tests to detect type
> > consistency.
>
> Probably no more time than I spend cursing [expletive deleted]'s who
> write type-tests. ...
> "BEHAVES-LIKE-A" (i.e. IS_ADAPTABLE_TO_PROTOCOL) is what, at most,
> one SHOULD be testing (if anything).
Sure, but it is often much easier to document that var x must be a foo than it
is to say that it must provide some arbitrary set of attributes. This after
all is one of the point of types: to create a shorthand for sets of properties
that you want to go together.
- Christopher
Christopher Barber
> On 1 Aug 2001, Christopher Barber wrote:
>> One problem I remember having was that I was writing library code and had to
>> write explicit run-time type-checking code to check the types of incoming
>> arguments.
>
> But why? ...
Maybe it just comes down to personal preference. I have written code in quite
a number of languages but have always preferred to declare types when I have
had the chance. I know that it has brought me benefits in my own development
work, and you are not going to convince me otherwise. I fully believe in
writing as comprehensive unit tests as you can, but I also know from
experience that it is virtually impossible to catch everything this way. I
would not want to force static type declarations down anyone else's throat
but it would be nice if they were available to those that wanted to use them.
You didn't respond to my comments on potential for performance gains. Can I
assume that I scored a point? ;-)
- Christopher
Steve Holden
a whooooole slew of properties, this itself can be a trap for the unwary.
Suppose, in your mission-critical program, you use an int to hold a value
which should only ever be between 32 and 213. The only way you have to
exercise that level of semantic control is to declare a class, and test that
no assignments violate the required constraint (unless you are using a
Pascal derivative, which allowed that kind of thing quite nicely, or you are
using program proof techniques, which are of little use to the
mathematically-deficient). So run-time value testing is required even if you
force the compiler to do type testing.
Type safety completely fails to live up to whatever promise it might have
unless programmers exercise adequate care to define types which do exactly
what they want. And most programmers are somewhat sloppy about that.
and-i'm-no-better-than-the-rest-ly y'rs - steve
PS: Alex, I'm sorry about the "if type()..."s!
--
http://www.holdenweb.com/
Jonathan Hogg
tan...@swing.co.at (Christian Tanzer) wrote:
> Jonathan Hogg <jona...@onegoodidea.com> wrote:
>
> > I can though, think of plenty of places where strong static typing would
> > defeat intentional exploitation of the dynamic typing. Maybe this is
> > poor programming on my part, but I can't even think how one would type a
> > function like say:
> >
> > def f(zs): return reduce( lambda x, y: x + y, zs )
>
> Genericity (templates in C++) this is not a problem -- you get one
> instance of `f` for each type `zs` you happen to apply `f` to.
>
> Disclaimer: C++ templates don't allow different return types for such
> functions, but that's a restriction of C++, not of genericity. IIRC,
> Ada generics can be generic in the return type, too.
Generics don't give meaningful types to functions. They simply delay the
type-checking to the instantiator. Specialising a function each time it
is used may be fine for native compilation, but not for type-checking or
for Python VM compilation. I'd argue that even doing that, you couldn't
come up with a workable type system for Python.
Exercise for the reader, think of types for the following:
class BadBoy:
def __radd__( self, other ):
return "Oh yeah that's a %s!" % `other`
pn = BadBoy()
zs = [ 1.3, 0, 5, pn, " Hoo hah." ]
heck = f(zs) # using my previous f()
You can argue that this is a shitty piece of code (and I agree!), but it
runs, and I don't want the compiler stopping me writing it.
The problem is that a dynamic language like Python creates types that
are effectively as complex as the language itself. You end up concerning
yourself so much with coming up with meaningful signatures, that you
write the program in the type and have to execute it to see if it
type-checks. That may be fine for proof systems like Alf, but not for
Python.
As Michael Abbot observed:
> statically typed Python would need to be a
> different language from standard Python
Now if you're just worried about speed and want to be give hints to the
compiler so it can unbox types and optimise dispatch, then that's a
whole different ball game. Do-able, but still tricky - as whenever it
was unsure the unifier would have to insert box/unbox checks just in
case and drop back to the standard dispatcher. You'd end up having to
litter your code with type hints and assertions.
Someone argued that typing makes code more readable, I'd wager that if
you took a standard Python program and annotated it to the level
necessary to get sensible optimisation/checking out of it, a beginner
would mistake it for perl ;)
Jonathan
Donn Cave
...
| Right. My assertion is that this causes more problems than it solves
| because you seldom know your "intent" in that much detail, much less your
| future intent. Seems like ultra strict type checking might work well in
| cases where you have very precise requirements and once built, your
| software won't be changing any time soon (a la space shuttle software).
| Most programs don't fall into that category.
OK, another example, happy to be able to accommodate your request.
From today's comp.lang.python, someone asks, how come my program
doesn't work? His case seems a ways away from space station control.
His point of failure looks like this,
arg = parse(data)
if arg[0] == 'privmsg':
...
elif arg[0] == 'ping':
...
And the run time error is a TypeError: unsubscriptable object.
He's baffled enough by this to post to comp.lang.python instead
of figure it out himself.
Here's his code:
-----
def parse(self, data):
if string.find(string.lower(data), 'privmsg') >= 0:
garbage,middle,msg = string.split(data,':')
sender,cmd,rcpt = string.split(middle)
arg=[string.lower(cmd),sender,rcpt,msg]
return arg
elif string.find(string.lower(data), 'ping') >= 0:
sender,cmd,pong = string.split(data[1:])
arg=[string.lower(cmd),sender,pong]
return arg
-----
And similar in OCaml:
-----
let ulfind str data =
try
let n = Str.search_forward (Str.regexp_case_fold str) data 0 in
true
with Not_found ->
false
let parse data =
if ulfind "privmsg" data then
let wl = Str.split (Str.regexp ":") data in
match wl with
garbage::middle::msg::[] ->
let wl = Str.split (Str.regexp "[ \t]+") middle in
begin
match wl with
sender::cmd::rcpt::[] ->
Some [(String.lowercase cmd); sender; rcpt; msg]
| _ -> None
end
| _ -> None
else if ulfind "ping" data then
let wl = Str.split (Str.regexp "[ \t]+") (Str.string_after data 1) in
match wl with
sender::cmd::pong::[] ->
Some [(String.lowercase cmd); sender; pong]
| _ -> None
else
None
let main () =
let data = begin print_string "parse me: "; read_line () end in
let argl = parse data in
match argl with
Some [] -> Printf.printf "Empty\n"
| None -> Printf.printf "unexpected input \"%s\"\n" data
| Some (cmd::argtl) -> Printf.printf "Command \"%s\"\n" cmd
;;
main ()
-----
It's probably a suboptimal expression of the idea, as I am very
new to OCaml. OCaml's type checking catches his error - if I
leave the last "else" out of the parse function, the compiler
notices the void return, in a context where void is not accounted
for.
I used a None/Some option for the return just to show how that
would work, but that one bit of explicit typing isn't really
necessary, an empty list or an exception would have worked fine.
That's a complete, working program.
Hope you find this useful.
Donn Cave, do...@u.washington.edu
bru...@tbye.com
> You didn't respond to my comments on potential for performance gains. Can I
> assume that I scored a point? ;-)
Sure! I agree that there's the potential for some gains, but the value of
that is so dependent on what you're doing so I stuck to things more
general. I'm all for performance improvements, but performance (in the
CPU sense) isn't what draws me to Python and keeps me here. For me, most
programs aren't performance critical, and those that are usually aren't
CPU-bound (but disk, memory, network, etc.), and the CPU-bound ones
usually have a small performance-critical core that can easily be moved to
C.
Also, before optimizing any code you should be fairly sure that the
potential payoff is worth the trouble/risk, and that the thing you're
optimizing is one of the biggest bottlenecks (fry the biggest fish). I
don't know either is true here.
I'm sure there's many benefits to strict compile-time type checking (for
example, it makes the language more appealing to developers like yourself
who want that feature). My interest has been in finding the most valuable,
most compelling reason for adding it to Python, something that would make
me better understand the fuss. ;-)
-Dave
Terry Reedy
"Donn Cave" <do...@u.washington.edu> wrote in message
news:9ka118$mj8$1...@nntp6.u.washington.edu...
> And the run time error is a TypeError: unsubscriptable object.
> He's baffled enough by this to post to comp.lang.python instead
> of figure it out himself.
What we need here are better error messages: a more explicit 'none
object is not subscriptable' might have steered him to the problem.
Terry J. Reedy
Andy Todd
<9k95f...@enews2.newsguy.com>:
>"Christopher Barber" <cba...@curl.com> wrote in message
>news:pso66c9...@jekyll.curl.com...
> ...
>> > What is a real world problem that would not have existed if Python
>> > had strict compile-time checking, and how common is such a problem?
>>
>> Here is a real-world problem: a developer wants to use a language with
>> compile-time type checking and rejects Python because it doesn't have
>> it.
>
>Why is this "a problem"? Counterexample: a developer wants
>to use a language with an 'e' in the language's name and rejects
>Python because it has no 'e' in its name. Should we therefore
>rename Python to 'Pethon' to "solve this problem"?
>
>Each developer or development organization can freely choose
>whatever constraint he/she/it/they prefer regarding their
>programming tools. If this it a problem, it's *HIS* (her,
>its, their) problem, not Python's. It would be silly and
>self-defeating if Python tried to match such constraints just
>because they are posed as constraints.
>
>If it's worth for Python to have an 'e' in its name, it must
>be because using that letter has intrinsic value within the
>logic of Python, so, you should be arguing about that
>intrinsic value, not about "marketing" advantages that
>may stem from having it, if you want to speak "real world".
Ladies and gentlemen, our quote of the week (so far), is;
>Marketing, by definition, isn't about the real world - it's
>about successfully projecting a fantasy in the minds of
>people with deep-enough pockets to make you rich:-).
>
<flame bait>
and yet, if no one claims to believe marketing why are Bill, Scott and
Larry so rich?
</flame bait>
At which point I believe it is pertinent for me to run away.
>
[snipping far too much interesting stuff about why static typing is a
crutch and you should really write your own code properly <wink> ]
>
>
>Alex
>
>
>
Regards,
Andy
--
Content free posts a speciality
bru...@tbye.com
> | Right. My assertion is that this causes more problems than it solves
> | because you seldom know your "intent" in that much detail, much less your
> | future intent. Seems like ultra strict type checking might work well in
> OK, another example, happy to be able to accommodate your request.
> >From today's comp.lang.python, someone asks, how come my program
> doesn't work? His case seems a ways away from space station control.
>
...
> And the run time error is a TypeError: unsubscriptable object.
> He's baffled enough by this to post to comp.lang.python instead
> of figure it out himself.
[ snipped 10 lines of Python ]
[ snipped 30 - 40 lines of type-checking OCaml ]
> Hope you find this useful.
Very! Thank you. Here's what it shows me:
- A Python newbie like the original poster is going to get a heck of a lot
further than an OCaml newbie trying to accomplish the same thing in a
strict type-checking way (I wonder how many posts he would have sent to
c.l.ocaml?). A veteren Pythonista won't have too many chances for causing
that bug because the Python version will be so short and simple.
- The cost of finding the Python bug (type-related or not) is probably
quite lower than the cost of finding a bug in the same OCaml code. If
nothing there's 66 to 75% fewer lines of code to sift through so you'll
narrow the search a lot quicker (I realize you're new to OCaml, but the
poster is new to Python).
- If the function later needs to be modified to be more flexible with
different types or even just a little extra functionality, the Python one
is gonna be a breeze. I'd feel confident about a quick change followed by
some quick testing. I'd feel much more nervous about modifying the other
one.
- After a little experience using Python, the cost of finding that bug
will drop dramatically (how long would it take you, Donn, to figure it
out? 30 seconds maybe?) as will the likelihood of that bug occurring, but
the cost of creating the OCaml type-checking version will remain more or
less constant - you'll always have to write about that much code. So
you'll fight your way through the learning curve only to find that by
doing so what you learned just went down in value. ;-)
-Dave
Donn Cave
| On 1 Aug 2001, Donn Cave wrote:
|
| > | Right. My assertion is that this causes more problems than it solves
| > | because you seldom know your "intent" in that much detail, much less your
| > | future intent. Seems like ultra strict type checking might work well in
|
| > OK, another example, happy to be able to accommodate your request.
| > >From today's comp.lang.python, someone asks, how come my program
| > doesn't work? His case seems a ways away from space station control.
| >
| ...
|
| > And the run time error is a TypeError: unsubscriptable object.
| > He's baffled enough by this to post to comp.lang.python instead
| > of figure it out himself.
|
| [ snipped 10 lines of Python ]
| [ snipped 30 - 40 lines of type-checking OCaml ]
|
| > Hope you find this useful.
|
| Very! Thank you. Here's what it shows me:
|
| - A Python newbie like the original poster is going to get a heck of a lot
| further than an OCaml newbie trying to accomplish the same thing in a
| strict type-checking way (I wonder how many posts he would have sent to
| c.l.ocaml?). A veteren Pythonista won't have too many chances for causing
| that bug because the Python version will be so short and simple.
A Perl version could be shorter yet, maybe half the lines. So even
fewer bugs?
| - The cost of finding the Python bug (type-related or not) is probably
| quite lower than the cost of finding a bug in the same OCaml code. If
| nothing there's 66 to 75% fewer lines of code to sift through so you'll
| narrow the search a lot quicker (I realize you're new to OCaml, but the
| poster is new to Python).
The compiler sifted through the lines of code.
| - If the function later needs to be modified to be more flexible with
| different types or even just a little extra functionality, the Python one
| is gonna be a breeze. I'd feel confident about a quick change followed by
| some quick testing. I'd feel much more nervous about modifying the other
| one.
Fine! You'll overcome your nervousness, sit down and start coding
in those new types or whatever. Maybe later, you'd do it with more
confidence. But confident or not, OCaml won't let you go until you
have accounted for those new types, wherever necessary in the program.
OCaml doesn't let your confidence become a liability.
| - After a little experience using Python, the cost of finding that bug
| will drop dramatically (how long would it take you, Donn, to figure it
| out? 30 seconds maybe?) as will the likelihood of that bug occurring, but
| the cost of creating the OCaml type-checking version will remain more or
| less constant - you'll always have to write about that much code. So
| you'll fight your way through the learning curve only to find that by
| doing so what you learned just went down in value. ;-)
Perversely missing the point. For sure, once the bug came to my
attention, it took much less than 30 seconds to find it. But should
we count the time it took my client to get me a traceback? The time
it the solution took getting to the client, properly installed?
If I spent extra hours wrestling with a few lines of code, and the
reward is a modest improvement in reliability, it might well be worth
it even if I don't do space shuttle gigs. If I spent the same time
staring at my Python code, that bug might or might not jump out at
me. Or I could test the bejeezus out of it, which would surely take
much longer, and the magic inputs to get to that bug might or might
not come up.
I don't mean to pitch OCaml here, this is about types and Python.
OCaml just happens to be the most serious type checking language
I sort of barely know, and maybe it's a poor example because the
parts that were really about type checking were sort of lost in
the noise. (Yet that was actually part of the point - type
checking per se doesn't need to add wads of declarations.)
But your criticisms seem a little misguided. If your point is
that Python is good for quick hacks, and OCaml isn't, fine. I'll
buy that. But what about type checking and serious programming?
I guess you don't want any more examples of problems type checking
could solve?
Donn Cave, do...@drizzle.com
Alex Martelli
...
"""
> Back to Python -- programming in Python has always felt to
> me a *LOT* like generic programming
Yup. Only, in Python it's very implicit.
"""
...just about as in C++, except more so:-).
"""
The nice thing about Ada and Eiffel generics is that the requirements
on the generic parameters are spellt out explicitly. For most
"""
The problem with Ada and Eiffel generics is that the requirements
on the generic parameters are to be spelled out explicitly. Type
inference (as in Haskell) or signature-based polymorphism (as in
C++ templates and in Python) is smooth indeed (although the ability
to state out loud what you think the compiler will infer, if it
could be done without really adding any significant complexity,
would be a nice extra).
The issue from my POV is: what defines "a type", except the
signatures of that subset of its interfaces that a given generic
component is actually using? Presumably the programming-as-
contracting preconditions/postconditions/invariants on those
interfaces -- but those are *runtime*-checked, and even that
hardly catches a significant portion of the "type semantics".
Consider the issue of: what predicate-parameter P can I pass to
a generic such as C++'s std::sort. Of course, I have some
signature constraints -- P(x,y) must be callable for certain
x and y and return something usable as a boolean value; C++
catches errors about this at compile-time, while Python, if
I try calling mylist.sort(unsuitableP) where unsuitableP is
not callable with two parameters that are items of mylist,
will only catch it at run-time. Having the check happen at
compile-time gives me the error diagnostics slightly earlier
(5 to 10 minutes, if I use a sensible testing policy) AND
makes it easy for C++ to generate stupendous code (if P is
inline or can be inlined).
But that barely scratches the surface! The REAL conditions
that define the type of P are:
not P(x,x) for any x
P(x, y) implies not P(y, x) for any x, y
P(x, y) and P(y, z) implies P(x, z) for any x, y, z
Note the universal-quantifier 'any' liberally used here:
it's not something quaint and peculiar, it's TYPICAL of
real type-constraints, making them uncheckable in the
general case, even at runtime (much less compiletime).
THIS is the kind of thing I'd really like to express in a
compiler-usable way! THIS is the kind of thing that would
make my programs MUCH clearer to human readers, and really
get mileage out of type-checking. What, if anything, the
compiler is able to do with such a hypothetically formalized
expression of my real design intentions is a minor issue.
If *sometimes* a runtime check (or, joy!, even a compile
time one, saving me several precious extra minutes) can be
inferred from such type-constraints (maybe only in some
kind of debug-mode, if, as likely, checks would burden
the program), then great. If *sometimes* the compiler is
able to optimize away some computation by *relying* on my
assertions, then super (maybe only in some kind of optimize
mode, if, as likely, it would slow down compilation a lot).
But existing languages appear to me to have things
backwards. They *start* from what they'll be able to
check, and/or able to exploit for optimization, and
use THAT criterion to decide what subset of my design
intentions I'm allowed (or, more often, required) to
formally express in the language.
That's like the drunkard looking for his keys under
the streetlamp, even though he knows he's dropped them
at the other end of the road, because it's too dark
over there...:-). A language should let me say what
I mean, and do of that what it can. Compilers and
supporting tools do evolve, old applications tend to
live forever; basing what a language lets me express
(or forces me to express) on today's compiler and
tool technology is back-asswards.
Given the incomplete and unsatisfactory state of the
way various languages let me (or rather force me to)
express my design intentions, I'd rather do away with
all the complexity, and just use Python (or C++
templates). Give me an URL to a usable language that
lets me express the most basic typing constraints,
such as symmetry or antisymmetry, reflexivity or
antireflexivity, and transitivity, as above, and I'll
be more than happy to give it a try. But Ada, Eiffel,
C++ (what it DOES do for typechecking), Java, and,
sigh, even Haskell, don't really cut the mustard today.
Therefore, Python's *simplicity* wins the day, as
you say...:
"""
application areas, Python still wins hands down, though.
"""
I agree, but I hope you can now better see WHY I do
so fully agree. Complication must bring a payoff that
offsets its undeniable costs. Today's type-checking,
as applied in the above-listed languages, doesn't, in
terms of useful earlier-detection of errors -- it's
little use to find out about an error 10 minutes
earlier, if it takes me 15 more minutes to write the
code snippet because of extra language complication.
Most interesting checks happen at runtime if at all,
and in fact most important properties can't even be
stated formally in the languages, because of a lack
of existential and universal quantifiers.
Today's typechecks DO help hugely in terms of
optimization, even though Self, the Scheme compiler
stalin, and good Common Lisp compilers all prove
that static typing is nowhere near a NEED to get
such optimizations, that type inference even in
a hugely dynamic language can get there too. But
C++'s generics are an excellent example of the way
lack of type-constraint declaration in no way
inhibits type-based optimization -- given signature
based polymorphism, type-inference in any actual
instantiation of a generic IS pretty trivial:-).
Alex
Alex Martelli
...
> > > did spend more time than I would have liked writing tests to detect
type
> > > consistency.
> >
> > Probably no more time than I spend cursing [expletive deleted]'s who
> > write type-tests. ...
> > "BEHAVES-LIKE-A" (i.e. IS_ADAPTABLE_TO_PROTOCOL) is what, at most,
> > one SHOULD be testing (if anything).
>
> Sure, but it is often much easier to document that var x must be a foo
than it
> is to say that it must provide some arbitrary set of attributes. This
after
Which, again, boils down to looking for your keys under
the streetlamp, because it's too dark over there where
you actually dropped the keys...?-)
> all is one of the point of types: to create a shorthand for sets of
properties
> that you want to go together.
That's what I'd call a PROTOCOL (or, but the term carries
connotations that might be unwarranted, an interface), and
the very reason I'd *LOVE* the protocol-adaptation PEP to
become a reality.
You see, one doesn't normally need just a boolean yes/no
answer to the question "'IS' wenk a pleep?" -- rather, one
needs a 'handle' for wenk that DOES satisfy protocol pleep,
or the information that no such 'handle' can be obtained
(wenk is not *adaptable to* protocol pleep).
That's how one does it in Java -- with the goofy syntax
(pleep)wenk
returning a pleep or raising an exception; in C++, with
the syntax dynamic_cast<pleep*>(wenk) [or pleep& if you
want an exception in case wenk can't peel]; and, best of
all object-models I know, in COM (syntax depends on the
language, as COM is a multi-language object-model, but
semantics rely on calling a specified protocol adaptation
method of wenk called QueryInterface).
All of these approaches have one serious defect from
my POV, and another one from yours. From yours: they
can't really be turned into *compile-time* checks! A
function receiving argument wenk has no way to tell at
compile time whether wenk is adaptable to pleep in
the general case (except, in theory, by examining all
the source code for the program: but even then one
cannot normally tell without solving the halting
problem:-). In order to ENSURE all checks pass at
compile-time reliably, a language must impose far
too many complications and constraints. In fact,
neither Java nor Eiffel really do ensure that (they
have loopholes due to covariance -- Java doesn't
explicitly support covariance, but its arrays in
fact do, and so you can get typeerrors at runtime),
let alone C++ with its unconstrained pointers.
Personally, I don't much mind what can be done at
compile time and what can't -- if *sometimes* the
compiler is able to hoist some check or computation
up to compile time, so much the better, but I don't
want to chafe under unreasonable restrictions to
help the compiler writers do that (and still fail
to do that everywhere anyway, again see Eiffel and
Java's covariance typing-issues:-).
So, the defect of Java's casts, C++'s dynamic_cast,
and even COM's most powerful QueryInterface from MY
viewpoint: they ALL focus on object-identity! You're
supposed to be able to 'cast back' (or QI back) to
the original interface-pointer (or other expressions
of object-identity). This obsessive focus on identity
would be interpreted as a mental illness in many
contexts (at least by therapists of schools harking
back to Korzibski's General Semantics)...;-). Of
course, identity DOES matter in a few of these cases
(COM focuses on it for specific technical reasons
having to do with remoting, for example), but most
definitely NOT for everyday programming issues. If
you look at COM's vast array of 'service' interfaces,
you'll see it has grown one to address exactly this
need -- IServiceProvider basically duplicates the
builtin QueryInterface *except* that it can return
a pointer *without* identity-constraints. I would
far rather have unconstrained protocol adaptation as
the normal modus operandi -- if and when one does
need to worry about identity, in a few rare and
limit cases such as remoting and persistence, hey,
that's what id() is for.
All of which makes the protocol-adapdation PEP a
real jewel in my eyes... except for the little
detail that it doesn't seem that anybody who really
matters cares much about it, so it just stays
parked there. Oh well...:-(.
Alex
Alex Martelli
news:f8Z97.12190$9i1.1...@e420r-atl1.usenetserver.com...
...
> Suppose, in your mission-critical program, you use an int to hold a value
> which should only ever be between 32 and 213. The only way you have to
> exercise that level of semantic control is to declare a class, and test
that
> no assignments violate the required constraint (unless you are using a
> Pascal derivative, which allowed that kind of thing quite nicely, or you
are
...but handles it *at runtime* in the general case, of course (even
though a few special-cases can be hoisted up to compile-time). Not
to mention the peculiar implication of having INTEGER ranges as a
built-in language feature, and REAL ranges nowhere to be seen -- isn't
it just as important to be able to state that a REAL (in Pascal terms:
float, in Python) variable x ensures A<=x<B, as it is when x is
an INTEGER...?!
> using program proof techniques, which are of little use to the
> mathematically-deficient). So run-time value testing is required even if
you
> force the compiler to do type testing.
Yes, by far most interesting things can't be compile-time tested.
> Type safety completely fails to live up to whatever promise it might have
> unless programmers exercise adequate care to define types which do exactly
> what they want. And most programmers are somewhat sloppy about that.
And most languages (all that I know) just give programmers NO way
to make the type-assertions that really matter ("this function is
antisymmetric", and so on -- all requiring existential and/or
universal quantifiers...).
> and-i'm-no-better-than-the-rest-ly y'rs - steve
>
> PS: Alex, I'm sorry about the "if type()..."s!
You're forgiven, Steve, as long as you undertake to sin no more...!-)
Alex
David M. Cooke
Or perhaps better, a compile-time error that 'parse' has an implicit
'return None' at the end. Of course, adding that would mean that this:
def three_plus_1_problem(n):
while 1:
if n == 1:
return 1
if x % 2 == 0:
n = n / 2
else:
n = 3*n + 1
would also be an error; you'd have to throw an extraneous return at
the end. Personally, I could live with that for the extra sanity checking.
--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
Michael Abbott
Alex, I'd like to say more in response to your very interesting message
below, but it's difficult to know where to begin. I agree wholeheartedly
with your sentiments, but I'm wondering if you're aware of just how
difficult what you're describing is to achieve!?
For example, there is a type system known as Coq (http://coq.inria.fr/)
which uses a type system similar in spirit to what you describe (but Coq is
oriented towards the automation of formal mathematical proofs). Coq is
based upon dependent types, and demonstrates that type checking of the sort
you are talking about is, in principle, at least as difficult as solving an
arbitrary mathematical problem! Also, it has to be said that expressing
mathematics in Coq is *very* hard work (formalising the "fundamental
theorem of algebra", ie, the result that polynomials over C have roots, was
worth several PhDs recently).
I also remember a recent language in a similar spirit (don't have the
reference to hand) where a two-level language was developed: one level for
expressing programs, the other level for expressing assertions about
programs (and building proofs of those assertions, if I recall correctly).
I'm of the opinion that work on dependent types is the way forwards, but
there are some big (gigantic?) obstacles to be overcome first...
"Alex Martelli" <ale...@yahoo.com> wrote in
news:9kb8m...@enews1.newsguy.com:
Alex Martelli
> [posted and mailed]
>
> Alex, I'd like to say more in response to your very interesting message
> below, but it's difficult to know where to begin. I agree wholeheartedly
> with your sentiments, but I'm wondering if you're aware of just how
> difficult what you're describing is to achieve!?
I suspect we may be talking at cross-purposes -- I don't request
a language that will in fact absolutely CHECK assertions containing
universal or existential quantifiers, nor one that will always be
able to take advantage of them to further optimization or other
type-inference goals: just one that will let me *express* such
assertions simply, which, per se, isn't all that hard.
Surely it wouldn't be the first time a language let me express
design-intent aspects without committing any implementation of
the language to doing anything in particular about them! Don't
you recall the 'register' keyword in C, for example? Nowadays
compilers do their own register allocation -- but I still get to
express my design intent "and, I believe this thing here should
live in a register" to my heart's content. Not much, as far
as expression design intentions goes, but, a start. Similarly,
I recall BSD variants where my programs were ALLOWED to give
hints to the operating systems about their intentions wrt some
files or memory areas -- "I think I'll just read this file
right through" or "I'll jump up and down all over this file
in random-access so I don't think I would bother doing any
prefetch if I were you", and so on, and so forth. Sometimes,
when reading such programs, such expressions of design intent
were illuminating, in ways that comments and other forms of
documentation rarely are. And, hey, maybe SOME versions of the
system did/will take advantage of my design-hints, so, why not?
We may have gotten too focused on the imperative and declarative
to exploit the value of *design-intent hints*. The assertions
I'd like to be able to make aren't just hints, but even if they
work like no more than that, I still think they'd be a pre-req
before one can state a language has types in a meaningful sense.
Eventually, 'checking modes' where some of the assertions were
partially checked (maybe just at runtime, maybe just during
some kinds of very thorough tests...), and 'optimizing modes'
where some of the assertions were relied upon for possible
optimizations (maybe just when compiling with "optimize all
you can please" flags), might emerge. Automatic deductions
and inferences would be a potential further step. But I think
the key step is the first one -- ALLOWING me to express those
design-intent constraints! I won't pay a dime extra to be
able to say something pretty useless such as "I intend X to
be a list" (particularly because it's *NOT* what I _really_
should intend -- 'a mutable sequence', MAYBE:-); I might be
much better disposed could I express "I intend X to be a
mutable sequence with an odd number of items such that at
all times items with an even positive index are strictly
greater than either of their neighbors" -- now *THAT* is
what I'd call "typing". (No, I wouldn't insist on THAT
being checked at compile-time... even checking it at RUN
time might be expensive enough that I'd be quite content
with only checking partially, and/or only when using certain
flags, if at all -- but my point is that, THIS is the kind
of things I find myself wishing I *COULD* usefully express,
*NOT* 'X is a list', 'Y is a complex', 'Z is a tuple of
float'... so, I think that for languages to claim they're
doing 'typing checks' for me, when such trivialities is all
they can or do check, violates truth in advertising!-).
Alex
Chris Barker
That would help, as would any number of debugging techniques that any
Python programmer with any experience would know. However, these would
only work if, in fact the author checked his code with invalid data. It
could be a long time before an error like that was found. Frankly, not
thinking to handle the case of invalid data would be likely to bite you
in any language, and I imagine that it is rare that type checking would
find it very often.
-Chris
--
Christopher Barker,
Ph.D.
ChrisH...@home.net --- --- ---
http://members.home.net/barkerlohmann ---@@ -----@@ -----@@
------@@@ ------@@@ ------@@@
Oil Spill Modeling ------ @ ------ @ ------ @
Water Resources Engineering ------- --------- --------
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
Skip Montanaro
David> Or perhaps better, a compile-time error that 'parse' has an
David> implicit 'return None' at the end. Of course, adding that would
David> mean that this:
David> def three_plus_1_problem(n):
David> while 1:
David> if n == 1:
David> return 1
David> if x % 2 == 0:
David> n = n / 2
David> else:
David> n = 3*n + 1
David> would also be an error; you'd have to throw an extraneous return
David> at the end. Personally, I could live with that for the extra
David> sanity checking.
And you thought the division change would break a lot of code! There's
nothing wrong with execution falling off the end of the function. Python
was designed that way. It's when there are explicit returns and the
implicit return is reachable that you need to warn about a potential
problem.
A fairly small amount of analysis can detect that the implicit return is
never reached (I did such a thing for a peephole optimizer). In that case,
all I did was toss out any unreachable code block. It would be sufficient
to detect that the last return is reachable and emit a warning if there was
another return executed in the body of the function. This would be okay:
def foo(a):
if a == 0:
return 1
elif a == 1:
return 0
else:
return a*5
because that final implicit return is unreachable. This would emit a
warning though:
def foo(a):
if a == 0:
return 1
elif a == 1:
return 0
--
Skip Montanaro (sk...@pobox.com)
http://www.mojam.com/
http://www.musi-cal.com/
Christian Tanzer
"Alex Martelli" <ale...@yahoo.com> wrote:
> "Christian Tanzer" <tan...@swing.co.at> wrote in message
> news:mailman.996682083...@python.org...
> ...
> """
> > Back to Python -- programming in Python has always felt to
> > me a *LOT* like generic programming
>
> Yup. Only, in Python it's very implicit.
> """
> ...just about as in C++, except more so:-).
>
>
> """
> The nice thing about Ada and Eiffel generics is that the requirements
> on the generic parameters are spellt out explicitly. For most
> """
> The problem with Ada and Eiffel generics is that the requirements
> on the generic parameters are to be spelled out explicitly.
The explicit spelling of the requirements has two consequences:
- It improves type checking. I agree that this doesn't cover a whole
lot of ground (although Eiffel does a better job here than any other
language, IMHO).
I used to care a lot about static type checking but don't see this
as a big issue anymore.
- It improves reusability. In C++ or Python, you have to look at the
source code of the generic component to know what the requirements
are. In Ada or Eiffel, a look at the interface suffices. This is a
big advantage over the implicit requirements.
In Python that isn't so bad, because most often you can try it out
in the interpreter just as fast as looking up the interface. In C++,
trying it out and having the compiler barf out less than enlightened
error messages is a royal pain.
Anyway, I'd love to have a type inference engine for Python code.
> But C++'s generics are an excellent example of the way lack of
> type-constraint declaration in no way inhibits type-based optimization
> -- given signature based polymorphism, type-inference in any actual
> instantiation of a generic IS pretty trivial:-).
C++ is very weak in optimizing template instantiations when compared
to Ada. That's probably not a feature of the language but a
shortcoming of the implementations. OTOH, after ten years templates
should be really trivial for the C++ compiler implementers. When I
looked last time (1998), the compilers were still pretty brain dead in
that regard, though.
bru...@tbye.com
> | [ snipped 10 lines of Python ]
> | [ snipped 30 - 40 lines of type-checking OCaml ]
> |
> | c.l.ocaml?). A veteren Pythonista won't have too many chances for causing
> | that bug because the Python version will be so short and simple.
>
> A Perl version could be shorter yet, maybe half the lines. So even
> fewer bugs?
Two points: (1) yes, in general there is a correlation between lines of
code and number of bugs. (Don't bother posting a complex one-liner as
counter-evidence - this is a general and not an all-encompassing rule).
(2) I said short *and* simple.
Your strict type-checking snipped of code was the perfect example of this
- there were a ton of places where I, the developer, could have screwed up
in writing that mess. The complexity of the Python one was so much lower
there's just not as much that I could do wrong. Yes, that type checking
probably did prevent those type-related bugs, but the added complexity
opens the door for a slew of different bugs.
> | - If the function later needs to be modified to be more flexible with
> | different types or even just a little extra functionality, the Python one
> | is gonna be a breeze. I'd feel confident about a quick change followed by
> | some quick testing. I'd feel much more nervous about modifying the other
> | one.
>
> Fine! You'll overcome your nervousness, sit down and start coding
> in those new types or whatever. Maybe later, you'd do it with more
> confidence. But confident or not, OCaml won't let you go until you
> have accounted for those new types, wherever necessary in the program.
> OCaml doesn't let your confidence become a liability.
?? The point is that, not only is the initial cost of writing the
type-checking one dramatically higher, maintenance on the code is also
much, much higher. When you come back to the function after not looking at
it for a few months, that's a lot of code to make sense of. Fiddling with
it has a high risk of introducing new bugs (not necessarily type-related
bugs) because of the much higher complexity.
This shouldn't strike you as odd. If you have an assembly function and a C
function that do the same thing and you need to change them, which one
will you most likely break? If you have a complex implementation of a
function and a simple, straightforward one, and you change them both,
which one will most likely break? 99 times out of 100 (or more), the short
and simple functions are harder to break. It's just a fact of life that
the more balls you have to keep in the air at one time, the more likely
you are to drop one.
> | - After a little experience using Python, the cost of finding that bug
> | will drop dramatically (how long would it take you, Donn, to figure it
> | out? 30 seconds maybe?) as will the likelihood of that bug occurring, but
> | the cost of creating the OCaml type-checking version will remain more or
> | less constant - you'll always have to write about that much code. So
> | you'll fight your way through the learning curve only to find that by
> | doing so what you learned just went down in value. ;-)
>
> Perversely missing the point. For sure, once the bug came to my
> attention, it took much less than 30 seconds to find it. But should
> we count the time it took my client to get me a traceback?
This is a bit of an exaggeration, but I'll play along. First of all, look
at this specific bug. How in the world did you manage to ship that? Did
you test at all? This wasn't exactly hard to reproduce or find. Now look
at it in the general sense. Yes, you should count the time it took your
client to give you traceback, install a solution, etc, but it should count
for each and every bug, not just this one. I can guarantee one of two
things (and maybe both): either the complex type-checking one has more
bugs in it due to the added complexity OR it took so long to write it
correctly that before you were a third of the way done your evil twin
underbid you, delivered a working Python version, fixed that one bug,
delivered a new version with better features, delivered another version to
suit the changing needs of the business, delivered another version with
new features, and got showered with money.
I mean, just *look* at that strict type-checking version of code. Wow! All
that complexity for such a simple piece of functionality. What in the
world would it look like if you needed to do something complex? I mean,
yes, you may have solved those darn type-related bugs, but how many more
bugs did you introduce in the process?
> it the solution took getting to the client, properly installed?
> If I spent extra hours wrestling with a few lines of code, and the
> reward is a modest improvement in reliability, it might well be worth
> it even if I don't do space shuttle gigs.
Yes, it might be, and you do have to judge for yourself. In general,
though, wrestling with code is a Bad Thing, especially if something so
simple is causing the wrestle. Your code may end up reliable in the
type-checking sense but be less reliable overall, sort of a
complexity-induced flakiness.
> me. Or I could test the bejeezus out of it, which would surely take
> much longer,
Thorough testing should happen on both versions, and the type-safe version
would definitely take a lot longer to test thoroughly. Every time I look
at it I just go, "wow."
> and the magic inputs to get to that bug might or might
> not come up.
Huh? There's nothing magic about thorough testing.
> But your criticisms seem a little misguided. If your point is
> that Python is good for quick hacks, and OCaml isn't, fine. I'll
> buy that. But what about type checking and serious programming?
What in the world? Sorry if that's what you thought I was saying, because
that's not it at all. I _am_ saying that, because more complexity and more
lines of code tend to mean more bugs, the type-safe version nearly ruins
my chances of success in a reasonable time frame.
> I guess you don't want any more examples of problems type checking
> could solve?
Why would that be the case? Because the examples so far have reinforced
the notion that today's type-checking systems can fix those problems but
only at a tremendous cost (increased development time/effort, increased
testing time, increased maintenance time)?
Donn Cave
| Terry Reedy wrote:
|> "Donn Cave" <do...@u.washington.edu> wrote in message
|> news:9ka118$mj8$1...@nntp6.u.washington.edu...
|>> And the run time error is a TypeError: unsubscriptable object.
|>> He's baffled enough by this to post to comp.lang.python instead
|>> of figure it out himself.
|>
|> What we need here are better error messages: a more explicit 'none
|> object is not subscriptable' might have steered him to the problem.
|
| That would help, as would any number of debugging techniques that any
| Python programmer with any experience would know. However, these would
| only work if, in fact the author checked his code with invalid data. It
| could be a long time before an error like that was found. Frankly, not
| thinking to handle the case of invalid data would be likely to bite you
| in any language, and I imagine that it is rare that type checking would
| find it very often.
Not sure what that last clause meant (rare that ... very often?) but
I guess this depends on the relationship between "thinking to handle
the case of invalid data" and type consistency.
If you decide to ignore his probable intention (did he think about
invalid inputs or not?), then the flaw in his program really was a
inconsistency between the function that could return None and caller
who failed to account for that. Either in isolation is arguably fine
(maybe None is the right thing to return for an unmatched case), the
problem is that they don't match, an inconsistency that could have
been caught.
Of course a program can be written that will be perfectly type
consistent but still vulnerable to unexpected inputs. But on
the other hand, you can use the enforced consistency of a type
checking language to help avoid accidents of that nature.
From this point of view the language is not just the rules, but
the idioms that apply those rules as a programming tool. If you
use type checking in this more active sense, I bet you can avoid
a significant number of unexpected input problems.
Donn Cave, do...@u.washington.edu
Donn Cave
| On Thu, 2 Aug 2001, Donn Cave wrote:
|> I guess you don't want any more examples of problems type checking
|> could solve?
|
| Why would that be the case? Because the examples so far have reinforced
| the notion that today's type-checking systems can fix those problems but
| only at a tremendous cost (increased development time/effort, increased
| testing time, increased maintenance time)?
Because we have demonstrated to your satisfaction that there are
indeed common problems that type checking could solve?
The rest is only details. You seem to have assessed that tremendous
cost by looking at my novice solution in a language you don't know at
all, and then I have some questions about your logic (type checking
needs more testing?) But now that we are agreed that there is a domain
of problems to be solved, we can move on to consider solutions in that
domain. Or not, but anyway that's progress.
Donn Cave, do...@u.washington.edu
John S. Yates, Jr
I have been programming for 30 years, starting with papertape
on a PDP-8 and a KSR 33. That means I spent a long time
coding in assembler. In the late 1970s I joined DEC (Digital
Equipment Corp) and had the joy of "graduating" to a "high
level" systems programming language: BLISS. The important
point for this discussion is that BLISS is (was?) type-free.
Being released from managing registers and from generating
labels and branches meant I could code faster and tackle
projects I would never have contemplated in assembler. In
this case the project was a brand new optimizer and code
generator for a VAX Pascal compiler.
While I was building that backend I was also absorbing the
concepts of "strong" type checking. Regularly I found myself
getting to the end of a debugging session only to discover
that "Gee! If I had been coding in Pascal this error would
have been caught at compile time". Such bugs frequently
involved argument mismatches, values chosen from the wrong
(simulated) enumeration, etc.
This impression was born out at the end of the project when,
with a now fully functioning Pascal compiler in hand, I set
out to reimplement the project's bug tracking system. I found
that I regularly did major clean-up on the system with great
confidence knowing that once I got through an error-free
compile the system would work as well as it did before the
clean-up. In the BLISS work I had never felt that confidence.
Python is not as bad as BLISS since values at run-time carry
their types and these are checked. But the difference feels
very much a kin the the difference between Pascal or Ada and
old, no prototypes, no true booleans, all enumerated values
are really just ints, K&R C.
Once I get my Python code running I find I am loath to change
it. The system does not help me find every site that needs to
be updated. Only runtime execution can do that. Which places
a greater value on unit testing and coverage analysis. And the
investment I make in those areas detracts from the rate of code
evolution.
I find that Python supports rapid prototyping but not necessarily
rapid evolution and solidification.
/john
--
John Yates
40 Pine Street
Needham, MA 02492
781 444-2899
Greg Ewing
>
> Not
> to mention the peculiar implication of having INTEGER ranges as a
> built-in language feature, and REAL ranges nowhere to be seen
I think that's because various other language features
made use of the existence of ordinal subrange types,
such as array indexes and set types. There is nowhere
else in the language that real subranges are used,
so presumably Mr. Wirth didn't consider there was
enough justification for building them in.
Also, given the imprecision of comparing floats,
it could be difficult to pin down whether a given
value should be considered legal for a given
subrange in some cases. Discrete subranges are
much easier to write precise statements about
in language definitions.
--
Greg Ewing, Computer Science Dept, University of Canterbury,
Christchurch, New Zealand
To get my email address, please visit my web page:
http://www.cosc.canterbury.ac.nz/~greg
Michael Abbott
> "Michael Abbott" <mic...@rcp.co.uk> wrote in message
> news:Xns90F19B5D3770...@194.238.50.13...
>>
>> I agree
>> wholeheartedly with your sentiments, but I'm wondering if you're aware
>> of just how difficult what you're describing is to achieve!?
>
> I suspect we may be talking at cross-purposes -- I don't request
> a language that will in fact absolutely CHECK assertions containing
> universal or existential quantifiers, nor one that will always be
> able to take advantage of them to further optimization or other
> type-inference goals: just one that will let me *express* such
> assertions simply, which, per se, isn't all that hard.
Um. But I don't see the point. If the compilers going to ignore
everything you write, then why not just put your assertions into comments?
It's what we all have to do already!
>
> Surely it wouldn't be the first time a language let me express
> design-intent aspects without committing any implementation of
> the language to doing anything in particular about them! Don't
> you recall the 'register' keyword in C, for example? Nowadays
> compilers do their own register allocation -- but I still get to
> express my design intent "and, I believe this thing here should
> live in a register" to my heart's content.
I always thought that compiler register allocation made the register
keywork pointless. Surely using this keyword is just noise now.
> Not much, as far
> as expression design intentions goes, but, a start. Similarly,
> I recall BSD variants where my programs were ALLOWED to give
> hints to the operating systems about their intentions wrt some
> files or memory areas -- "I think I'll just read this file
> right through" or "I'll jump up and down all over this file
> in random-access so I don't think I would bother doing any
> prefetch if I were you", and so on, and so forth. Sometimes,
> when reading such programs, such expressions of design intent
> were illuminating, in ways that comments and other forms of
> documentation rarely are. And, hey, maybe SOME versions of the
> system did/will take advantage of my design-hints, so, why not?
Seems to me that there'd only be any point in putting such hints into your
program if there was a chance that they'd have some concrete effect.
Otherwise you might as well just write good comments, surely?
>
> We may have gotten too focused on the imperative and declarative
> to exploit the value of *design-intent hints*. The assertions
> I'd like to be able to make aren't just hints, but even if they
> work like no more than that, I still think they'd be a pre-req
> before one can state a language has types in a meaningful sense.
>
> Eventually, 'checking modes' where some of the assertions were
> partially checked (maybe just at runtime, maybe just during
> some kinds of very thorough tests...), and 'optimizing modes'
> where some of the assertions were relied upon for possible
> optimizations (maybe just when compiling with "optimize all
> you can please" flags), might emerge.
Interesting idea. So, our "high level abstract assertions" can be used in
several ways:
1. To supplement comments
2. As a pious hope that the complier will check and validate them (fat
chance!)
3. As run time debug consistency checks
4. To generate automated thorough tests
5. As a stepping stone to the future
Hmm. Well, for 1 I think we should write comments, and it seems we're
agreed that 2 is implasible. Of course, it sounds like you'd like an
assertion language that is not fully amenable to 3 (after all, checking
quantifiers can't necessarily be done in finite time), but I suppose we
could expect that a compiler could make a stab at generating sensible
tests. Automatically generated tests is an interesting idea as well, and
would be very handy: the abstract assertions could be used to guide the
generation of tests.
> Automatic deductions
> and inferences would be a potential further step. But I think
> the key step is the first one -- ALLOWING me to express those
> design-intent constraints! I won't pay a dime extra to be
> able to say something pretty useless such as "I intend X to
> be a list" (particularly because it's *NOT* what I _really_
> should intend -- 'a mutable sequence', MAYBE:-); I might be
> much better disposed could I express "I intend X to be a
> mutable sequence with an odd number of items such that at
> all times items with an even positive index are strictly
> greater than either of their neighbors" -- now *THAT* is
> what I'd call "typing".
But what do you gain by writing all this in a formal language that's going
to be quietly ignored by all your tools? Why not just write good comments?
> (No, I wouldn't insist on THAT
> being checked at compile-time... even checking it at RUN
> time might be expensive enough that I'd be quite content
> with only checking partially, and/or only when using certain
> flags, if at all -- but my point is that, THIS is the kind
> of things I find myself wishing I *COULD* usefully express,
> *NOT* 'X is a list', 'Y is a complex', 'Z is a tuple of
> float'... so, I think that for languages to claim they're
> doing 'typing checks' for me, when such trivialities is all
> they can or do check, violates truth in advertising!-).
Oh, that's unfair! After all, there is always a trade-off in type checking
between the expressive power of the language being checked and how much
checking is actually performed.
Alex Martelli
...
> > I suspect we may be talking at cross-purposes -- I don't request
> > a language that will in fact absolutely CHECK assertions containing
> > universal or existential quantifiers, nor one that will always be
> > able to take advantage of them to further optimization or other
> > type-inference goals: just one that will let me *express* such
> > assertions simply, which, per se, isn't all that hard.
>
> Um. But I don't see the point. If the compilers going to ignore
> everything you write, then why not just put your assertions into comments?
> It's what we all have to do already!
When expressing design intentions in a formal language, I get
a *CHANCE* that they will be checked, at least partially, or
used for optimization purposes and other inferences. It's the
same difference there is between
# here x will always be greater than y
and
assert x>y
the latter is a formal way to express the former, which makes it:
-- totally unambiguous (comments are in natural language,
thus ambiguous more often than not)
-- potentially checkable -- some compiler might turn this into
if not (x>y): raise AssertionError
at least with the right flags
-- potentially usable for inferences -- again with the right
flags a compiler MIGHT be able to use this alleged fact
to get higher-performing code (e.g. eliminate a following
"if x<y" block)
The first point always holds and it's a key one. Second and third
are just possibilities (with current Python technology, for assert,
the second does hold, the third one doesn't), but the point is:
once the construct is in the language, and therefore in those
application programs that might live for the next 20 or 40 years,
compiler technology may improve and actually take advantage of
what today are just 'potentials', aka possibilities.
So I want to be able to write, say (I'm not hung on syntax):
def sort(sequence, predicate):
forany x: not predicate(x,x)
forany x,y: truth(predicate(x,y)) != truth(predicate(y,x))
or whatever -- whether a current compiler is able to insert SOME
partial checks (compile- or run-time), or take some inferences from
these formal assertions, is of SECONDARY importance. They still
stand as formal-language, potentially-(partially)-checkable,
potentially-usable-for-optimization, unambiguous expressions
of my design intent, so their existence enhances the value of
the program containing them. And if they didn't add substantial
complication, they'd be worth having in the language.
> I always thought that compiler register allocation made the register
> keywork pointless. Surely using this keyword is just noise now.
It doesn't tell the reader of the program much that is useful, no.
> Seems to me that there'd only be any point in putting such hints into your
> program if there was a chance that they'd have some concrete effect.
> Otherwise you might as well just write good comments, surely?
See it the other way 'round: you might as well write things out
in formalized language -- they'll be at least as good as the best
comments you could possibly write (no ambiguity risk), PLUS,
if an application lives for 40 years who can rule out a future
compiler may be able to make use of them?
> Interesting idea. So, our "high level abstract assertions" can be used in
> several ways:
> 1. To supplement comments
> 2. As a pious hope that the complier will check and validate them (fat
> chance!)
I think you mean 'fat chance' _at compile-time_. Well, maybe.
> 3. As run time debug consistency checks
> 4. To generate automated thorough tests
> 5. As a stepping stone to the future
>
> Hmm. Well, for 1 I think we should write comments,
I think otherwise, because natural language is inherently ambiguous.
> > greater than either of their neighbors" -- now *THAT* is
> > what I'd call "typing".
>
> But what do you gain by writing all this in a formal language that's going
> to be quietly ignored by all your tools? Why not just write good
comments?
Because comments are in natural language and natural language
is inherently ambiguous. Sorry for repeating this answer, but
then you keep repeating the question:-).
> Oh, that's unfair! After all, there is always a trade-off in type
checking
> between the expressive power of the language being checked and how much
> checking is actually performed.
I think the sweet spot of the tradeoff (for all purposes except
perhaps code speed, and Self, stalin &c make that disputable)
is a VERY expressive language -- one which can express a lot,
including stuff for which the checking is not normally performed.
The only trade-off that's embarassing is the one that comes if
one takes for granted that the compiler MUST make use NOW of
EVERYTHING it lets you express -- then, it had better be darn
selective on what it lets you express. But the 'taking for
granted' is IMHO totally unwarranted.
Alex
Scott Langley
The SmallScript.net web site says that the first public beta will now
appear this month, August 2001.
And, selectively quoting from this message by David Simmons:
"...the AOS Platform has been developed and tested for dynamic (and
dynamically typed) languages and is geared as an adaptive virtual
machine jitting model...
As to Python specifically, I want to stress that work is currently
focused on delivering the .NET, PPC, and x86 platform versions...
Once the other pieces are in place, more attention will be focused on
work in Python and PHP...
As an aside, the execution performance numbers for
SmallScript/Smalltalk are directly indicative of the level of
performance one could expect for Python or PHP on the same platform."
--
Scott Langley
s...@scottlangley.com
http://www.scottlangley.com
"Tim Peters" <tim...@home.com> wrote in message news:<mailman.996644762...@python.org>...
> [Courageous]
> > http:\\www.smallscript.com. Execution times within 1-2X of C.
>
> [Terry Reedy]
> > This site has about zero to do with Python. Main page include Python
> > as (planned) secondary language. Contents links Python back to PEP
> > index page (bizarre). 'Python for AOS' page is under construction.
> >
> > Where did you find fabulous Python compilation results you keep
> > reporting?
>
> I believe it originally came from this cryptic tease:
>
> Python is a big part for the execution engine, this is a new area.
> The Smalltalk VM runs Python 10 to 100x faster.
>
> at
>
> http://www.smalltalkconsulting.com/html/SmalltalkSolutions2001%232.html
>
> first referenced in this c.l.py thread:
>
> http://groups.google.com/groups?ic=1&th=b121faa695c8b6fa
>
> I haven't seen more than that single second-hand claim (except for Joe's
> charming belief that if he repeats it often enough, Guido will get pissed
> off enough to make it a reality now and for free <wink>).
>
> every-jit-project-is-at-least-a-year-late-and-severely-underperforms-
> expectations-for-at-least-a-year-after-that-ly y'rs - tim
John Schmitt
> -----Original Message-----
> From: bru...@tbye.com [mailto:bru...@tbye.com]
> Sent: Wednesday, August 01, 2001 6:56 PM
> To: pytho...@python.org
> Subject: Re: Typing system vs. Java
>
>
> On 1 Aug 2001, Donn Cave wrote:
>
> > | Right. My assertion is that this causes more problems
> than it solves
> > | because you seldom know your "intent" in that much
> detail, much less your
> > | future intent. Seems like ultra strict type checking
> might work well in
>
> > OK, another example, happy to be able to accommodate your request.
> > >From today's comp.lang.python, someone asks, how come my program
> > doesn't work? His case seems a ways away from space
> station control.
> >
> ...
>
> > He's baffled enough by this to post to comp.lang.python instead
> > of figure it out himself.
>
> [ snipped 30 - 40 lines of type-checking OCaml ]
>
>
> Very! Thank you. Here's what it shows me:
>
> - A Python newbie like the original poster is going to get a
> heck of a lot
> further than an OCaml newbie trying to accomplish the same thing in a
> strict type-checking way (I wonder how many posts he would
> have sent to
> for causing
> that bug because the Python version will be so short and simple.
A Python newbie will probably get more done simply because it's easier to
learn Python than ML, yeah, probably true.
>
> - The cost of finding the Python bug (type-related or not) is probably
> quite lower than the cost of finding a bug in the same OCaml code. If
> nothing there's 66 to 75% fewer lines of code to sift through
> so you'll
> narrow the search a lot quicker (I realize you're new to
> OCaml, but the
> poster is new to Python).
I'm following this conversation with great interest and I think I'm grokking
everything up to here.
As for finding this bug, well, isn't the point of ML that *the compiler
finds the bug* way before it ever gets to a customer? I guess that assumes
that you've expressed your types correctly. If you had not expressed your
types correctly, that would be a bug related to the semantics of their
meaning (a 'logic error' as described in an earlier post). That is, you
knew exactly what they types mean and expressed exactly that.
>
> - If the function later needs to be modified to be more flexible with
> different types or even just a little extra functionality,
> the Python one
> is gonna be a breeze. I'd feel confident about a quick change
> followed by
> some quick testing. I'd feel much more nervous about
> modifying the other
> one.
I'm not sure if you're being fair here. I've only done a small amount of ML
(in school, some years ago) but your question is pretty hand-wavey and
contrived to make your point. I much prefer coding in Python, but I
wouldn't make the claim you just made there. How about becoming fluent in
ML and then doing a detailed analysis? :-)
>
> - After a little experience using Python, the cost of finding that bug
> will drop dramatically (how long would it take you, Donn, to figure it
> out? 30 seconds maybe?) as will the likelihood of that bug
> occurring, but
> the cost of creating the OCaml type-checking version will
> remain more or
> less constant - you'll always have to write about that much code. So
> you'll fight your way through the learning curve only to find that by
> doing so what you learned just went down in value. ;-)
Obviously the barrier to entry for ML is quite a bit higher than Python,
(are we still talking about newbies?) but the compiler will find the bug for
you if it was written ML'ically (as opposed Pythonically?) unless I am much
mistaken.
[...]
I withhold judgement of whether strong, static typing will help or hurt
productivity. I'm inclined to think that The Python Way might turn out to
be more productive for the majority of programmers and for lots of
programming problems. I don't know how you would go about demonstrating
that, and I would certainly want to be very proficient in both systems
before I felt qualified to make an assertion either way.
I don't think C and C++ (and Java, I guess) are good examples of strong,
static typing and so not really useful if you want to have a discussion
about strong, static typing.
Here's a nice article you might like to read that talks more about this
whole subject. Be warned, the article is from a guy who wants to find a way
to add some of this stuff (typing a la ML) to Perl. It's not a
Python-centric article.
http://perl.plover.com/yak/typing/typing.html
John
bru...@tbye.com
> > - The cost of finding the Python bug (type-related or not) is probably
> > quite lower than the cost of finding a bug in the same OCaml code. If
> > nothing there's 66 to 75% fewer lines of code to sift through
> > so you'll
> > narrow the search a lot quicker (I realize you're new to
> > OCaml, but the
> > poster is new to Python).
>
> I'm following this conversation with great interest and I think I'm grokking
> everything up to here.
>
> As for finding this bug, well, isn't the point of ML that *the compiler
> finds the bug* way before it ever gets to a customer? I guess that assumes
> that you've expressed your types correctly. If you had not expressed your
> types correctly, that would be a bug related to the semantics of their
> meaning (a 'logic error' as described in an earlier post). That is, you
> knew exactly what they types mean and expressed exactly that.
Well, expressing your types correctly does not mean your program has no
bugs. I guess the debate has at least these two main parts: just what
percentage of bugs would be eliminated by correct and detailed types, and
does the effort to correctly specify them yield enough benefit to make it
worth it?
My take is that, especially in Python, type-related bugs make up an
extremely small segment of all bugs, and the examples I've seen of
specifying all that type information dramatically increase the complexity
of the source code, even to the point of possibly _introducing_ bugs.
> > - If the function later needs to be modified to be more flexible
> with > different types or even just a little extra functionality, >
> the Python one > is gonna be a breeze. I'd feel confident about a
> quick change > followed by > some quick testing. I'd feel much more
> nervous about > modifying the other > one.>
> I'm not sure if you're being fair here. I've only done a small amount of ML
> (in school, some years ago) but your question is pretty hand-wavey and
> contrived to make your point. I much prefer coding in Python, but I
> wouldn't make the claim you just made there. How about becoming fluent in
> ML and then doing a detailed analysis? :-)
That comment was based _solely_ on the added size and complexity of the
type-checking version. While I obviously can't guarantee that in all cases
my assertion stands ( like I said before, I'm no ML expert) I do know that
these rules generally hold, and that's the basis of my statement:
- more lines of code tends to mean more bugs (all bugs are human error,
all humans are error-prone, the more opportunities you create for error,
the more errors occur)
- more complex code is buggier than straightforward code (consider code
that is so obvious that it is self-documenting vs. code so complex that
the ratio of comments to code statements is > 1)
- in addition to being buggier, more code and more complexity increases
the cost of: development, maintenance, and transfer of ownership if for no
other reason than the fact that your brain has to do that much more
processing
These rules are pretty universal (when applied to humans); they don't just
apply to software development. The more details I have to manage, the more
likely I am to screw something up.
> - After a little experience using Python, the cost of finding that bug
> > will drop dramatically (how long would it take you, Donn, to figure
> it out? 30 seconds maybe?) as will the likelihood of that bug
> occurring, but the cost of creating the OCaml type-checking version
> will remain more or less constant - you'll always have to write
> about that much code. So you'll fight your way through the learning
> curve only to find that by doing so what you learned just went down
> in value. ;-)
>
> Obviously the barrier to entry for ML is quite a bit higher than Python,
> (are we still talking about newbies?)
I didn't think we were originally, but the example posted came from a
newbie question. ;-)
> but the compiler will find the bug for
> you if it was written ML'ically (as opposed Pythonically?) unless I am much
> mistaken.
I would agree that it could find type-related bugs, and perhaps the
stricter type model expands the realm of those bugs so that more are
caught, but I don't think that any ML makes the grandiose claim that all
bugs fall into that category.
Also, it goes back to the question of just how much that costs you. You
and your business have a non-zero tolerance level of bugs (unless you're
doing some sort of formal validation of all your code). Why? Because
there's other factors like time-to-market, budgets, and so on. (For
personal programming, there's the time-to-bordem-with-this-idea factor)
How bad is it if your customer finds a bug, anyway? This discussion so far
has assumed that when your customer finds a defect it's deadly to your
business. This isn't always the case. Extreme Programming, for example, is
all about releasing early and often. It's obviously not always possible or
correct to use XP, but when you do then both you and your customers
_expect_ to find bugs, and it's okay: they find a bug, you add it to your
internal tests, and they get the fix in a few days or weeks. That being
the case, the work to add strict type checking better have a huge payoff
or you're wasting valuable development time (because you're slowing down
the development pace to build in checks for specific types of bugs, but as
soon as you release you get higher quality testing that covers all types
of bugs).
-Dave
Courageous
>My take is that, especially in Python, type-related bugs make up an
>extremely small segment of all bugs, and the examples I've seen of
>specifying all that type information dramatically increase the complexity
>of the source code, even to the point of possibly _introducing_ bugs.
I feel alot of synergy for the above sentiment and believe it is quite
insightful. Of late, I've been thinking alot about lexical, grammatical,
and semantic complexity as it applies to computer programming and
the various different correctness memes which float around in the
community. I've concluded that comprehensible tersness trumps all
other measures of merit.
I believe that there is a sort of Occam's Razor of Coding, as it were,
where given two different programs which execute identically, the one
which is shorter, simpler, and easier to understand is the better
program, notwithstanding any principles it may or may not adhere to.
Various different rules which have crept up over the years such as
"don't use global variables," and "practice strong encapsulation,"
and so forth are all only distractions and are useless to the degree
that in any particular context they don't satisfy Sir Occam's relentless
blade.
Joe Kraska
San Diego
CA
xauau
> [...] I've concluded that comprehensible tersness trumps all
> other measures of merit.
Sounds good to me, Joe.
> I believe that there is a sort of Occam's Razor of Coding, as it were,
> where given two different programs which execute identically, the one
> which is shorter, simpler, and easier to understand is the better
> program, notwithstanding any principles it may or may not adhere to.
>
> Various different rules which have crept up over the years such as
> "don't use global variables," and "practice strong encapsulation,"
> and so forth are all only distractions and are useless to the degree
> that in any particular context they don't satisfy Sir Occam's relentless
> blade.
I've also been thinking similar thoughts for some time. I regularly
see people going to extreme lengths to protect themselves from
themselves. It often fails because it adds unnecessary bulk and
complexity in exchange for the false sense of security that following
a method brings with it.
Take "information hiding" as one small example. In theory it's
reasonable that _some_ data should be encapsulated and accessed only
through strictly defined interfaces. In practice, though, it's often a
case of interring trivial functionality deep inside a perilously
constructed wall of cruft.
I've seen lots of C++ and Java programs that really should have been
very simple and clean. They end up being complicated because the
majority of the program consists of small pieces of simple
functionality trying desperately to contact each other through nearly
impenetrable walls of "protection". When you're not in the mood to
laugh at it, it's a truly pitiful spectacle.
--
And there's a dreadful law here -- it was made
by mistake, but there it is -- that if anyone asks
for machinery they have to have it
and keep on using it.
Donn Cave
(quoting Dave Brueck>
The hard problems are really in large systems, though, aren't they?
How do you apply this razor to IIS, for example, or SunOS?
This afternoon I finally forced myself to get back to work on a large
piece of software that I want to modify. It's a network server that
performs a fairly simple function and isn't really all that big, but
it's inordinately difficult to work with. I could show you some
excerpts, you'd laugh, but I'm sure the author would argue that his
code is shorter and simpler, and for him I suppose perhaps even
easier to understand.
But the worst part is just that it's big and you have to follow
the flow of control all over the place to understand what interacts
with what, and to some extent you have to retain that understanding
while writing your part. And this is "Hello, world", compared to
real industrial strength big software.
Can you make that stuff manageable by keeping things short and simple?
Well, maybe yes, but it isn't enough to just meet some definition of
short and simple. Classic software engineering models like structured,
object oriented, functional programming all add some modular discipline
to code at the expense of straightforward brevity, and you need it to
survive. The enemy is not so much the small scale complexity of a
few lines of code, but the complexity of interactions in a large
system. Make that short and simple, and you're cooking - and you're
probably doing pretty well with your common software engineering
models.
I'm here because I need to reduce complexity, my head doesn't do
too good with the complicated stuff. And I make mistakes, a lot.
Python helps. It handles complicated stuff like managing storage
so I can deal with objects at a more abstract level. It handles
mistakes with exceptions, so I don't have to account for all the
things I don't expect. Those two, anyway, make for shorter and
simpler code most of the time, but you know there's a lot more
there than just how many lines of code. For one thing, in my
opinion those two features alone make a big difference for the
implementation of library functions, they reduce the complexity
of interfaces in a very important way.
If you look at a functional language with strong static type
checking, it's doing the same thing from a different direction -
reducing unnecessary side effects and so forth. But it also
benefits from a model that allows the compiler to infer some
degree of correctness in the relationships in your program.
It can't do your job better than you, but you can use it to
do your job better.
Donn Cave, do...@drizzle.com
Jonathan Hogg
Courageous <jkra...@san.rr.com> wrote:
> I feel alot of synergy for the above sentiment and believe it is quite
> insightful. Of late, I've been thinking alot about lexical, grammatical,
> and semantic complexity as it applies to computer programming and
> the various different correctness memes which float around in the
> community. I've concluded that comprehensible tersness trumps all
> other measures of merit.
Top point. Since I think this thread originally arose out of a
comparison to Java (I think, seems a long time ago now...) I've got a
Real World (tm) example that might be interesting.
I'm working on a system at the moment that I prototyped in Python and am
now re-coding in Java - unfortunately the latter language is the only
one I can use officially. The most trouble I've had has been in
translating the following pattern:
def something( self, y ):
for x1, y1 in self.foo.something( y ):
for x2, y2 in self.bar.something( y1 ):
yield (x1, x2), y2
The original was pretty much this much code. Problems I've had
translating this to Java:
* I've had to create some additional classes to encapsulate things that
I previously just used tuples/lists for (as in the code above).
* I have to explicitly type all of my variable declarations (more on
this in a tick).
* There is no standard iterator pattern (for x in xs) so I have to
explicitly ask for an iterator and do the hasNext() and next() calls on
it.
* There is no simple way of doing generators (Way to go Python 2.2) so I
have to construct the entire list and return it.
So my code has ended up something like:
public Vector something( String y )
{
Vector results = new Vector();
Vector foos = this.foo.something( y );
for ( Iterator i = foos.iterator(); i.hasNext(); )
{
Thingy xy1 = (Thingy) i.next();
Vector bars = this.bar.something( xy1.getY() );
for ( Iterator j = bars.iterator(); j.hasNext(); )
{
Thingy xy2 = (Thingy) j.next();
Vector xs = new Vector();
xs.add( xy1.getX() );
xs.add( xy2.getX() );
results.add( new Thingy(xs, xy2.getY()) );
}
}
return results;
}
Writing this out again (abstracting on the fly to protect the guilty)
I'm pretty sure that the Python code would compile, but I'm not sure the
Java code would. I generally have found with moving code from Python to
Java that most of my type errors are not logical errors, but me writing
the wrong type declaration.
I was chatting to a guy at work the other day who doesn't know Python,
but had heard of it and seen some code before. He said, "Python, isn't
that that 4GL language". At first I started going on about how it wasn't
a 4GL, but then stopped. The thought struck me that Python is a 4GL. The
difference becomes obvious in comparison to a 3rd generation language
like Java. Python gives me much higher level constructs to program with.
Since I can write more powerful programs in smaller, more readable
chunks, I think I write less buggy code. If we have to reduce the
simplicity/expressiveness of the language in order to add type-checking,
then I think the tradeoff is not worth it.
I'd argue that even the protocols/interfaces idea falls into this
category. Since the majority of protocol/interface errors fall out in
runtime testing very quickly, the additional effort that would be
required to specify the interfaces and the hugely difficult compilation
problem of enforcing them, just doesn't add up for me.
Jonathan
Alex Martelli
news:5k1nmts15l7nsio00...@4ax.com...
...
> I believe that there is a sort of Occam's Razor of Coding, as it were,
> where given two different programs which execute identically, the one
> which is shorter, simpler, and easier to understand is the better
> program, notwithstanding any principles it may or may not adhere to.
Great insight, except I think you've forgotten "easier to test, modify,
refactor". As user-stories change all the time, we need to be able
to change our programs -- that is were the simplicity & matter most,
actually -- and test to ensure our changes don't break things.
> Various different rules which have crept up over the years such as
> "don't use global variables," and "practice strong encapsulation,"
> and so forth are all only distractions and are useless to the degree
> that in any particular context they don't satisfy Sir Occam's relentless
> blade.
Except that they often promote ease of testing, modification and
refactoring. That is what gives them value.
Alex
Alex Martelli
news:jonathan-08E8BC...@news.easynet.co.uk...
...
> Since I can write more powerful programs in smaller, more readable
> chunks, I think I write less buggy code. If we have to reduce the
> simplicity/expressiveness of the language in order to add type-checking,
> then I think the tradeoff is not worth it.
>
> I'd argue that even the protocols/interfaces idea falls into this
> category. Since the majority of protocol/interface errors fall out in
> runtime testing very quickly, the additional effort that would be
> required to specify the interfaces and the hugely difficult compilation
> problem of enforcing them, just doesn't add up for me.
I agree with most of what you're saying but I think you have
the wrong viewpoint about the protocol-adaptation PEP. It
most assuredly doesn't reduce the expressiveness of Python,
and the "simplicity reduction" is minimal and in an excellent
cause. Consider...:
Say that function A receives as an argument object x. Now, A
needs to use x according to the "sequential binary file-reading
protocol". How do you handle that in Python today? If you're
lucky, A's author will mention that x must be a file-like object.
More likely A's author will falsely say that x must be a file (you
need to read code and experiment to prove that A's author
was lying). Hardly ever will A's author specify what _subset_
of a file's functionality x must supply -- is it .read (does x need
to support .read(N), or just .read()?), .readline, .readlines,
.xreadlines, ...? Does x.close() have to work? What about
x.tell and x.seek -- do THOSE need to work?
And what happens if client code starts with the data in a
form that isn't a file? Presumably client code imports
cStringIO and explicitly wraps (adapts) the string-form
data for A's consumption -- but often A's author supplies
two functions, A(fileobj) and As(stringobj) too. And what
if x is neither a file nor a string but some other kind of
object yet -- who's responsible for knowing how x gets
adapted to the file protocol...?
Having a *standard* way to request adapt-to-protocol
does away with most of these issues; it _promotes_
simplicity for both the author of A and the author of
client code. A's author asks for adaptation of argument
x to the protocol A needs; A's client passes any object
as x that can be adapted to the protocol; the adaptation
framework does the rest -- whether the 'adaptation' is
a no-operation (x itself already supports the protocol),
a standard wrapping (cStringIO to wrap a string object
into file protocol), or something specified by the author
of *x* (somebody else again from the authors of A and
of A's client code) by registering suitable hooks with the
adaptation framework, or by somebody else again.
That's not "type checking" as an overhead in development
for the purpose of getting some errors diagnosed a bit
earlier or getting some extra runtime performance: it's
something else altogether -- it's a provision of a standard
way for multiple authors of reusable components and
client-code of those components to resolve 'impedance
mismatches' (protocol differences) that inevitably arise.
Alex
Courageous
>> Various different rules which have crept up over the years such as
>> "don't use global variables," and "practice strong encapsulation,"
>> and so forth are all only distractions and are useless to the degree
>> that in any particular context they don't satisfy Sir Occam's relentless
>> blade.
>
>Except that they often promote ease of testing, modification and
>refactoring. That is what gives them value.
Sure. There's a bit of give and take on this, however. Take, for example,
the oft maligned public variable:
While in context there is a good argument to be made for separating
interface from implementation -- and this is especially appropriate if
you are producing an API -- sometimes the public attribute is itself
the interface. If you changed the name of your attribute, you would be
changing the name of your setters and getters, and you'd be flying
through all of your files with sed substitutions in any case. I do this
all the time, myself, to maintain consistency.
Note that I'm not arguing against interface and implementation separation;
I am simplying pointing out that the rule "always have getters and setters"
isn't necessarily correct all of the time. IMO, this is particularly true in
situations where you have a package of objects which are mutually
manipulatory, but don't otherwise expose a public API.
C//
Courageous
>Since I can write more powerful programs in smaller, more readable
>chunks, I think I write less buggy code. If we have to reduce the
>simplicity/expressiveness of the language in order to add type-checking,
>then I think the tradeoff is not worth it.
I emphatically agree. Syntactic type declaration is considerably out of
place with Python.
Although I was thinking about about something which might be useful.
What if Python were to support a type consistency check on containers?
>>> mylist = [1,2,"alpha"]!
Type Consistency Error:
>>>
Note the exclamation point.
C//
Chris Tavares
Doesn't the array module already do this?
>>> a = array.array( 'i', [1, 2, "hello world"])
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: an integer is required
>>>
-Chris
Courageous
>> >>> mylist = [1,2,"alpha"]!
>> Type Consistency Error:
>> >>>
>Doesn't the array module already do this?
For numeric types with known type codes, yes. More
generally, no. For example, a common use of container
objects is to put instances of objects into them. A consitent
list would simply make certain that all objects in the list
were of the same type, perhaps using isinstance or some
such.
C//