Clarity vs. code reuse/generality
kj
I'm will be teaching a programming class to novices, and I've run
into a clear conflict between two of the principles I'd like to
teach: code clarity vs. code reuse. I'd love your opinion about
it.
The context is the concept of a binary search. In one of their
homeworks, my students will have two occasions to use a binary
search. This seemed like a perfect opportunity to illustrate the
idea of abstracting commonalities of code into a re-usable function.
So I thought that I'd code a helper function, called _binary_search,
that took five parameters: a lower limit, an upper limit, a
one-parameter function, a target value, and a tolerance (epsilon).
It returns the value of the parameter for which the value of the
passed function is within the tolerance of the target value.
This seemed straightforward enough, until I realized that, to be
useful to my students in their homework, this _binary_search function
had to handle the case in which the passed function was monotonically
decreasing in the specified interval...
The implementation is still very simple, but maybe not very clear,
particularly to programming novices (docstring omitted):
def _binary_search(lo, hi, func, target, epsilon):
assert lo < hi
assert epsilon > 0
sense = cmp(func(hi), func(lo))
if sense == 0:
return None
target_plus = sense * target + epsilon
target_minus = sense * target - epsilon
while True:
param = (lo + hi) * 0.5
value = sense * func(param)
if value > target_plus:
hi = param
elif value < target_minus:
lo = param
else:
return param
if lo == hi:
return None
My question is: is the business with sense and cmp too "clever"?
Here's the rub: the code above is more general (hence more reusable)
by virtue of this trick with the sense parameter, but it is also
a bit harder to understand.
This not an unusual situation. I find that the processing of
abstracting out common logic often results in code that is harder
to read, at least for the uninitiated...
I'd love to know your opinions on this.
TIA!
kj
Steven D'Aprano
> ... I find that the processing of
> abstracting out common logic often results in code that is harder to
> read ...
Yes. There is often a conflict between increasing abstraction and ease of
understanding.
[...]
> The implementation is still very simple, but maybe not very clear,
> particularly to programming novices (docstring omitted):
>
> def _binary_search(lo, hi, func, target, epsilon):
> assert lo < hi
> assert epsilon > 0
You're using assert for data sanitation. That is a very bad habit for you
to be teaching novices. Your code will fail to behave as expected
whenever the caller runs Python with the -O switch.
[...]
> My question is: is the business with sense and cmp too "clever"?
For production code? Maybe -- you would need to profile the code, and
compare it to a less general, more simple-minded function, and see which
performs better. If there is an excessive performance penalty, that's a
good sign that perhaps you're being too clever, too general, or too
abstract -- or all three.
Too clever for novices? That depends on how inexperienced they are -- are
they new to Python or new to programming in general? Are they children or
adults? Do they have a PhD degree or are they still at primary school? It
depends on their experience and their intelligence -- dare I say it, some
people will *never* get programming, let alone code-reuse. It also
depends on how you lead up to it -- are you dropping them straight into
the abstract code, or giving them two pieces of nearly the same code and
showing them how to pull out the common functionality?
--
Steven
Alan G Isaac
1. Don't use assertions to test argument values!
2.
from scipy.optimize import bisect
def _binary_search(lo, hi, func, target, epsilon):
def f(x): return func(x) - target
return bisect(f, lo, high, xtol=epsilon)
3. If you don't want to use SciPy (why?), have them
implement http://en.wikipedia.org/wiki/Bisection_method#Pseudo-code
to produce their own `bisect` function.
hth,
Alan Isaac
Aahz
>
>This seemed straightforward enough, until I realized that, to be
>useful to my students in their homework, this _binary_search function
>had to handle the case in which the passed function was monotonically
>decreasing in the specified interval...
>
> assert lo < hi
> assert epsilon > 0
> sense = cmp(func(hi), func(lo))
> if sense == 0:
> return None
> target_plus = sense * target + epsilon
> target_minus = sense * target - epsilon
> while True:
> param = (lo + hi) * 0.5
> value = sense * func(param)
> if value > target_plus:
> hi = param
> elif value < target_minus:
> lo = param
> else:
> return param
>
> if lo == hi:
> return None
>
>My question is: is the business with sense and cmp too "clever"?
First of all, cmp() is gone in Python 3, unfortunately, so I'd avoid
using it. Second, assuming I understand your code correctly, I'd change
"sense" to "direction" or "order".
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"as long as we like the same operating system, things are cool." --piranha
Bearophile
> I'm will be teaching a programming class to novices, and I've run
> into a clear conflict between two of the principles I'd like to
> teach: code clarity vs. code reuse.
They are both important principles, but clarity is usually more
important because short code that can't be read can't be fixed and
modified, while long code that can be read is alive still.
> So I thought that I'd code a helper function, called _binary_search,
> that took five parameters: a lower limit, an upper limit, a
> one-parameter function, a target value, and a tolerance (epsilon).
Five arguments are a bit too much, even for non-novices. 2-3 arguments
are more than enough for novices.
Where are doctests'? Novices have to learn to think of tests as part
of the normal code :-)
I think the main problem in all this discussion is that generally to
learn to program you have to write code yourself, so your students
have to invent and write their own binary search. Reading your code
they are going to learn much less. Creating a binary search from
almost scratch can turn out to be too much hard for them, it means you
have to ask them to invent something simpler :-)
Programming is (among other things) problem solving, so they have to
learn to solve not easy problems from the beginning. Showing them
famous algorithms (and very good code to copy from) can be useful, but
it's less important than learning basic problem solving skills, and
much less important than learning why solving problems has to became
their purpose and pleasure :-)
Bye,
bearophile
Bruno Desthuilliers
> I'm will be teaching a programming class to novices, and I've run
> into a clear conflict between two of the principles I'd like to
> teach: code clarity vs. code reuse. I'd love your opinion about
> it.
(snip - others already commented on this code)
> Here's the rub: the code above is more general (hence more reusable)
> by virtue of this trick with the sense parameter, but it is also
> a bit harder to understand.
Perhaps better naming (s/sense/direction/g ?) and a small comment could
help ?
> This not an unusual situation. I find that the processing of
> abstracting out common logic often results in code that is harder
> to read, at least for the uninitiated...
IOW : the notion of "clarity" depends on the context (including the
reader). Anyway, there are algorithms (or implementations of...) that
are definitly and inherently on the 'hard to read' side - well,
complexity is something you have to learn to live with, period. The key
is to try and make it as simple and readable *as possible*.
Also, factoring out common code - or even slightly complex code - often
makes _client_ code (much) more readable.
Jean-Michel Pichavant
> I'm will be teaching a programming class to novices, and I've run
> into a clear conflict between two of the principles I'd like to
> teach: code clarity vs. code reuse. I'd love your opinion about
> it.
>
> sense = cmp(func(hi), func(lo))
> if sense == 0:
> return None
# assuming func is monotonous
if func(high) > func(low):
direction = 1 # aka sign of the func derivative
elif func(low) > func(high):
direction = -1
else:
return None
Avoid using cmp, this will prevent the "what the hell is cmp doing ?" ,
unless you want to teach your students how to search for inline python
documentation.
Some other list members have suggested to improve the variable naming. I
couldn't agree more, in your case, I think clarity can be achieved as
well with the abstraction (these notions does not necessarily collide).
Here's a link on my how-to-name bible :
http://tottinge.blogsome.com/meaningfulnames/
Jean-Michel
Lie Ryan
> I'm will be teaching a programming class to novices, and I've run
> into a clear conflict between two of the principles I'd like to
> teach: code clarity vs. code reuse. I'd love your opinion about
> it.
Sometimes when the decision between clarity and generality becomes too
hard; you might fare better to save the code, go out for a walk to
forget the code, and start a blank text editor. Being in a fresh mind,
you may found an alternative approach, e.g.:
from __future__ import division
def binary_search(lo, hi, func, target, epsilon):
# reverses hi and lo if monotonically decreasing
lo, hi = (lo, hi) if func(hi) > func(lo) else (hi, lo)
param = (lo + hi) / 2
# loop while not precise enough
while abs(func(param) - target) > epsilon:
param = (lo + hi) / 2
if target < func(param):
hi = param
else:
lo = param
return param
kj
>1. Don't use assertions to test argument values!
Out of curiosity, where does this come from?
Thanks,
kj
kj
>First of all, cmp() is gone in Python 3, unfortunately, so I'd avoid
>using it.
Good to know.
>Second, assuming I understand your code correctly, I'd change
>"sense" to "direction" or "order".
Thanks,
kj
Paul Rubin
> sense = cmp(func(hi), func(lo))
> if sense == 0:
> return None
> target_plus = sense * target + epsilon
> target_minus = sense * target - epsilon
The code looks confusing to me and in some sense incorrect. Suppose
func(hi)==func(lo)==target. In this case the solver gives up
and returns None even though it already has found a root.
Also, the stuff with the sense parameter, and target_minus and
target_plus looks messy. I do like to use cmp. I'd just write
something like (untested):
def _binary_search(lo, hi, func, target, epsilon):
y_hi, y_lo = func(hi), func(lo)
while True:
x_new = (lo + hi) * 0.5
y_new = func(x_new)
if abs(y_new - target) < epsilon:
return x_new
elif cmp(y_new, target) == cmp(y_hi, target):
hi = x_new
else:
lo = x_new
if lo == hi:
return None
This uses an extra couple function calls in that trivial case, of
course.
Alan G Isaac
>> 1. Don't use assertions to test argument values!
On 7/3/2009 12:19 PM kj apparently wrote:
> Out of curiosity, where does this come from?
http://docs.python.org/reference/simple_stmts.html#grammar-token-assert_stmt
"The current code generator emits no code for an assert statement when optimization is requested at compile time."
Alan Isaac
Steven D'Aprano
Assertions are disabled when you run Python with the -O (optimise) flag.
If you rely on assertions to verify data, then any time the user runs
python with -O your code will be running without error checking.
assert should be used to verify program logic, not to sanitize data.
--
Steven
Charles Yeomans
I wouldn't describe the use of an assert statement in the original
code as data sanitizing, but rather as a precondition. And with that
description, the use of an assert statement that might be compiled
away is not unreasonable; indeed, it certainly is so in the context of
design by contract.
Charles Yeomans
Terry Reedy
In other words, assertions should never happen, whereas your assertions
could easily happen with 'bad' input. An example of program logic
assertion would be 'assert <loop-invariant>' at the top of the loop.
That would be useful for development, but would not be needed for
production use of tested code.
tjr
Pablo Torres N.
> The context is the concept of a binary search. In one of their
> homeworks, my students will have two occasions to use a binary
> search. This seemed like a perfect opportunity to illustrate the
> idea of abstracting commonalities of code into a re-usable function.
> So I thought that I'd code a helper function, called _binary_search,
> that took five parameters: a lower limit, an upper limit, a
> one-parameter function, a target value, and a tolerance (epsilon).
> It returns the value of the parameter for which the value of the
> passed function is within the tolerance of the target value.
Five params for a novice function is overkill. I'd drop the epsilon
thing and, assuming you are searching in a list, tuple or something
similar, make the lower limit default to the first index and the upper
default to the last one.
In fact, I'd let them to realize that a function is convenient, and
base some of the grading in whether they wrote it or not. Just a
thought.
--
Pablo Torres N.
kj
>kj <no.e...@please.post> writes:
>> sense = cmp(func(hi), func(lo))
>> if sense == 0:
>> return None
>> target_plus = sense * target + epsilon
>> target_minus = sense * target - epsilon
>> ...
>The code looks confusing to me and in some sense incorrect. Suppose
>func(hi)==func(lo)==target.
In hindsight, I too think that it is incorrect, but for a different
reason. I've rewritten it like this:
sense = cmp(func(hi), func(lo))
assert sense != 0, "func is not strictly monotonic in [lo, hi]"
I regard the very special case of func(hi)==func(lo)==target as
pathological (analogous to the fact that a stopped watch is "exactly
right" twice a day), and not one I care to support.
Thanks for your feedback!
kj
kj
Sorry, this doesn't say anything like "don't use assertions to test
argument values". I'm aware of the fact that assertions are silenced
under certain circumstances. But where does the principle 1. in
your first reply come from? I've never seen it before, and would
like to understand the reasoning behind it.
kj
Pablo Torres N.
>>http://docs.python.org/reference/simple_stmts.html#grammar-token-assert_stmt
>>"The current code generator emits no code for an assert statement when optimization is requested at compile time."
>
> Sorry, this doesn't say anything like "don't use assertions to test
> argument values". I'm aware of the fact that assertions are silenced
> under certain circumstances. But where does the principle 1. in
> your first reply come from? I've never seen it before, and would
> like to understand the reasoning behind it.
>
> kj
> http://mail.python.org/mailman/listinfo/python-list
>
But...if no code is generated for assertions on some occasions, then the
parameters would go unchecked, potentially breaking your code in said
occasions.
--
Pablo Torres N.
kj
>On Sat, Jul 4, 2009 at 10:05, kj<no.e...@please.post> wrote:
>>>http://docs.python.org/reference/simple_stmts.html#grammar-token-assert_s=
>tmt
>>>"The current code generator emits no code for an assert statement when op=
>timization is requested at compile time."
>>
>> Sorry, this doesn't say anything like "don't use assertions to test
>> argument values". =C2=A0I'm aware of the fact that assertions are silence=
>d
>> under certain circumstances. =C2=A0But where does the principle 1. in
>> your first reply come from? =C2=A0I've never seen it before, and would
>> like to understand the reasoning behind it.
>>
>> kj
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>But...if no code is generated for assertions on some occasions, then the
>parameters would go unchecked, potentially breaking your code in said
>occasions.
This implies that code that uses *any* assert statement (other than
perhaps the trivial and meaningless ones like "assert True") is
liable to break, because whatever it is that these assert statements
are checking "on some occasions, ... would go unchecked, potentially
breaking your code."
I'm beginning to think that the original "precept" was simply "cargo
cult," i.e. one of those rules that are parroted without being
fully understood.
As I wrote in my original post, the function I posted was an internal
("helper") function, not to be used outside of the file. This fact
was further (ahem) underscored by the leading underscore in the
function's name. Under these circumstances, the assert statements
seem to me perfectly in keeping with the intended use for them.
kj
Paul Rubin
bisection search usually just requires the function to be continuous
and to have its value cross the target somewhere between the endpoints,
not be monotonic.
> I regard the very special case of func(hi)==func(lo)==target as
> pathological (analogous to the fact that a stopped watch is "exactly
> right" twice a day), and not one I care to support.
I do think you should support that case, under the "do 'nothing'
gracefully" principle. Having your equation solver crash if the
equation is already solved is like having your sorting function crash
if the input is already sorted. E.g. the function should do something
reasonable if you end up calling it twice.
Paul Rubin
> This implies that code that uses *any* assert statement (other than
> perhaps the trivial and meaningless ones like "assert True") is
> liable to break, because whatever it is that these assert statements
> are checking "on some occasions, ... would go unchecked, potentially
> breaking your code."
Yes, that implication is absolutely valid. The purpose of assert
statements is to debug the code, by checking for conditions that are
supposed to be impossible. Unless the program is broken (i.e. the
impossible happened), no assert statement should ever trigger.
Invalid input data is not considered impossible and doesn't imply a
broken program, so assert statements are not the appropriate way to
check for it. I like to use a function like
def check(condition, msg="data error"):
if not condition: raise ValueError, msg
...
check (x >= 0, "invalid x") # raises ValueError if x is negative
y = sqrt(x)
instead.
MRAB
> kj <no.e...@please.post> writes:
>> This implies that code that uses *any* assert statement (other than
>> perhaps the trivial and meaningless ones like "assert True") is
>> liable to break, because whatever it is that these assert statements
>> are checking "on some occasions, ... would go unchecked, potentially
>> breaking your code."
>
> Yes, that implication is absolutely valid. The purpose of assert
> statements is to debug the code, by checking for conditions that are
> supposed to be impossible. Unless the program is broken (i.e. the
> impossible happened), no assert statement should ever trigger.
>
True if the program is bug-free, no matter what the user might throw at
it; it's not the same as validation.
Alan G Isaac
> I'm beginning to think that the original "precept" was simply "cargo
> cult," i.e. one of those rules that are parroted without being
> fully understood.
Adopting such a view is of course an alternative to
attempting to understand, but perhaps less useful.
Alan Isaac
PS For additional explanation:
http://java.sun.com/j2se/1.4.2/docs/guide/lang/assert.html
Scott David Daniels
> Invalid input data is not considered impossible and doesn't imply a
> broken program, so assert statements are not the appropriate way to
> check for it. I like to use a function like
>
> def check(condition, msg="data error"):
> if not condition: raise ValueError, msg
>
> ...
> check (x >= 0, "invalid x") # raises ValueError if x is negative
> y = sqrt(x)
And I curse such uses, since I don't get to see the troublesome value,
or why it is troublesome. In the above case, y = sqrt(x) at least
raises ValueError('math domain error'), which is more information than
you are providing.
How about:
...
if x >= 0: raise ValueError('x = %r not allowed (negative)?' % x)
...
--Scott David Daniels
Scott....@Acm.Org
Paul Rubin
> And I curse such uses, since I don't get to see the troublesome value,
> or why it is troublesome. In the above case, y = sqrt(x) at least
> raises ValueError('math domain error'), which is more information than
> you are providing.
>
> How about:
Better still in these situations is to throw to pdb.
But yes, you can put a formatted string in a check message.
kj
>kj <no.e...@please.post> writes:
>> sense = cmp(func(hi), func(lo))
>> assert sense != 0, "func is not strictly monotonic in [lo, hi]"
>bisection search usually just requires the function to be continuous
>and to have its value cross the target somewhere between the endpoints,
>not be monotonic.
Try the algorithm I posted with lo = -pi/4, hi = 2*pi, func = cos,
target = -1, and see what you get...
>> I regard the very special case of func(hi)==func(lo)==target as
>> pathological (analogous to the fact that a stopped watch is "exactly
>> right" twice a day), and not one I care to support.
>I do think you should support that case, under the "do 'nothing'
>gracefully" principle.
You keep missing the point that this is an *internal* *helper*
*convenience* function, meant to abstract away common logic from
a handful of places and thus eliminate some code repetition within
a module. It is *not* a library function intended to be called
from elsewhere. So talk of "supporting" anything is besides the
point. Any internal use of this function that applies it to a
non-strictly-monotonic function is, by assumption, an error.
kj
kj
>kj <no.e...@please.post> writes:
>> This implies that code that uses *any* assert statement (other than
>> perhaps the trivial and meaningless ones like "assert True") is
>> liable to break, because whatever it is that these assert statements
>> are checking "on some occasions, ... would go unchecked, potentially
>> breaking your code."
>Yes, that implication is absolutely valid. The purpose of assert
>statements is to debug the code, by checking for conditions that are
>supposed to be impossible.
Precisely. As I've stated elsewhere, this is an internal helper
function, to be called only a few times under very well-specified
conditions. The assert statements checks that these conditions
are as intended. I.e. they are checks against the module writer's
programming errors.
kj
>Paul Rubin wrote:
>> kj <no.e...@please.post> writes:
>>> This implies that code that uses *any* assert statement (other than
>>> perhaps the trivial and meaningless ones like "assert True") is
>>> liable to break, because whatever it is that these assert statements
>>> are checking "on some occasions, ... would go unchecked, potentially
>>> breaking your code."
>>
>> Yes, that implication is absolutely valid. The purpose of assert
>> statements is to debug the code, by checking for conditions that are
>> supposed to be impossible. Unless the program is broken (i.e. the
>> impossible happened), no assert statement should ever trigger.
>>
>Technically these are known as "invariants". An assertion will always be
>True if the program is bug-free, no matter what the user might throw at
>it; it's not the same as validation.
What *user* are you talking about??? I've stated a bazillion times
that this function is meant to be called only from within this
module.
Simon Forman
> In <mailman.2611.1246721197.8015.python-l...@python.org> "Pablo Torres N." <tn.pa...@gmail.com> writes:
>
>
>
> >On Sat, Jul 4, 2009 at 10:05, kj<no.em...@please.post> wrote:
> >>>http://docs.python.org/reference/simple_stmts.html#grammar-token-asse...
Assertions are for you, the programmer, to check that your
understanding of the code is correct (or not, i.e. if the assertion
fails.)
It's unfortunately not uncommon to see code where the assert statement
is used as an integral part of the processing, or, in other words,
where running the code with '-O' will cause changes in the way the it
runs.
In the code you posted, the assert statements seem to be checking that
other code/algorithms aren't (will never) pass that function "out of
bounds" arguments. Perfectly valid IMO. Since it's an internal
helper function you can (should) know that client code will /always/
call it with valid values, the asserts simply make your intuition or
knowledge explicit in a way that will show up dramatically if you're
wrong about that (i.e. if there's some problem elsewhere in the code
feeding that function.)
Assertions should never fail, which is why you can (or should be able
to without breaking your code) remove them entirely and know that your
program will run identically. That's why folks are quick to jump on
cases where assertions are used as control-flow constructs. (In your
function, however, they aren't.)
In a sense, assertions /are/ meaningless, except to the programmer.
FWIW, you can use "if __debug__:" and put anything you like in the
statement suite and the whole if statement goes away when run with '-
O'. Sort of a "super-assert".
Regards,
~Simon
Paul Rubin
> >bisection search usually just requires the function to be continuous
> >and to have its value cross the target somewhere between the endpoints,
> >not be monotonic.
>
> Try the algorithm I posted with lo = -pi/4, hi = 2*pi, func = cos,
> target = -1, and see what you get...
Sorry, my comment was poorly phrase. Bisection search usually is
phrased in terms of solving f(x)=0 where f is continuous and if the
initial endpoints are a and b, then f(a) and f(b) have opposite sign.
> You keep missing the point that this is an *internal* *helper*
> *convenience* function, ... Any internal use of this function that
> applies it to a non-strictly-monotonic function is, by assumption,
> an error.
In that case there are better algorithms you can use. But didn't this
thread start out asking what version of the code was best suited for
presentation to students? In that case, the function's preconditions
should be stated explicitly and not be narrower than necessary.
Steven D'Aprano
>>Technically these are known as "invariants". An assertion will always be
>>True if the program is bug-free, no matter what the user might throw at
>>it; it's not the same as validation.
>
> What *user* are you talking about??? I've stated a bazillion times that
> this function is meant to be called only from within this module.
In that case, you would be the user, and like any user, you might
accidentally call the function with invalid data.
I believe this discussion started because you are presenting this as code
for novices. In that case, it is absolutely important that you start off
by teaching them the Right Way to do things. As a general rule, the Right
Way is to do an explicit test and raise rather than use assert.
In production code, "internal use only" code is a grey area where assert
is sometimes justified (although I'll point out that in practice, code
written for internal use only has a habit of being called by others). But
you're not writing production code, you're writing *teaching code*, where
even more important than code correctness is user education.
--
Steven
Pablo Torres N.
> Do you really want to inflict novices with the subtleties of when
> "assert" is a proper structure to use?
I'll second that. This very thread is proof that assertions are polemic, so
maybe you (kj) should just save your student's sanity and stick to good old
conditionals :-)
As for your initial question, I think, thirty four emails after, that yes, your
function is a bit too clever and you should sacrifice some generality in order
to make it more readable.
--
Pablo Torres N.
wwwayne
wrote:
>On 7/3/2009 10:05 AM kj apparently wrote:
=== 8< ===
>2.
>from scipy.optimize import bisect
>def _binary_search(lo, hi, func, target, epsilon):
> def f(x): return func(x) - target
> return bisect(f, lo, high, xtol=epsilon)
>
>3. If you don't want to use SciPy (why?), have them
>implement http://en.wikipedia.org/wiki/Bisection_method#Pseudo-code
>to produce their own `bisect` function.
Of course this isn't really pseudo-code, it's VB code with quite poor
comments:
'Bisection Method
'Start loop
Do While (abs(right - left) > 2*epsilon)
'Calculate midpoint of domain
midpoint = (right + left) / 2
'Find f(midpoint)
If ((f(left) * f(midpoint)) > 0) Then
'Throw away left half
left = midpoint
Else
'Throw away right half
right = midpoint
End If
Loop
Return (right + left) / 2
and even just throwing away the VB code and leaving the comments does
not give a good algorithm:
'Bisection Method
'Start loop
'Calculate midpoint of domain
'Find f(midpoint)
'Throw away left half
'Throw away right half
A much better approach to teaching introductory programming in any
language at almost any level is to incorporate some "top down problem
solving", including writing a method of solution (algorithm) in some
reasonably well-defined pseudo-code that can be easily elaborated and
translated into one's target language (and, peferably, some
reasonable-sized subset of related languages). This pseudo-code should
then become comments (or equivalent) for the students to convert to
real code, as in:
Algorithm bisect (f, left, right, epsilon):
# Bisection Method to find a root of a real continuous function f(x):
# Assume f(x) changes sign between f(left) and f(right) and
# we want a value not further than epsilon from a real root.
# Begin with the domain left...right.
# While the absolute value of (left - right) exceeds 2*epsilon:
# Calculate the midpoint, mid, of the domain.
# If the product of f(left) and f(mid) is positive:
# Set left to mid;
# Otherwise:
# Set right to mid.
# Return the midpoint of left...right.
===
And adapting this approach to kj's case is straightforward.
Of course, what consitutes a suitable vocabulary and syntax for an
algorithm pseudo-code language depends upon the target language(s),
the tastes of the instructor, and the point in the lesson or course.
My choice is for python+COBOL (as above) initially, soon incorporating
the usual arithmetic and relational operators (and how soon and how
many at once depends upon the level of the students: for an
introductory university/college course in Computer Science or
equivalent, where everyone should have a reasonable background in
mathemtics notation as a prerequisite, this should be very soon and
quite fast), arrays and subscripting, etc.
But if we were to write this algorithm or kj's in python-like
pseudo-code it would already *be* python codeor very close to
it--which is why we should teach intorductory programming in python.
Very soon students would be writing algorithms that required very
little elaboration to be programs.
But without including suitable problem solving and psudo-code
algorithm writing there will soon come a time or an example where
students are trying to think in code instead of in their natural
language and don't have the experience and repertoire to be able to do
that well.
I hope that's not too pedantic or AR?
wayne
>hth,
>Alan Isaac
David Smith
First, let me say *I got the point*. I use asserts, but only in unit
testing where I want to test the result of some action for correctness.
In the course of programming product code, I personally don't think
they should ever be used exactly for the reasons everyone else is
pointing out. They can be disabled with the -O option and that changes
the program's behavior in ways that could break in production.
If you insist on teaching the assert statement, teach it in the context
of writing unit testing code. Its an extremely valuable skill.
--David
Martin Vilcans
> I'm will be teaching a programming class to novices, and I've run
> into a clear conflict between two of the principles I'd like to
> teach: code clarity vs. code reuse. I'd love your opinion about
> it.
In general, code clarity is more important than reusability.
Unfortunately, many novice programmers have the opposite impression. I
have seen too much convoluted code written by beginners who try to
make the code generic. Writing simple, clear, to-the-point code is
hard enough as it is, even when not aiming at making it reusable.
If in the future you see an opportunity to reuse the code, then and
only then is the time to make it generic.
YAGNI is a wonderful principle.
Andre Engels
> On Fri, Jul 3, 2009 at 4:05 PM, kj<no.e...@please.post> wrote:
>> I'm will be teaching a programming class to novices, and I've run
>> into a clear conflict between two of the principles I'd like to
>> teach: code clarity vs. code reuse. I'd love your opinion about
>> it.
>
> In general, code clarity is more important than reusability.
> Unfortunately, many novice programmers have the opposite impression. I
> have seen too much convoluted code written by beginners who try to
> make the code generic. Writing simple, clear, to-the-point code is
> hard enough as it is, even when not aiming at making it reusable.
>
> If in the future you see an opportunity to reuse the code, then and
> only then is the time to make it generic.
Not just that, when you actually get to that point, making simple and
clear code generic is often easier than making
complicated-and-supposedly-generic code that little bit more generic
that you need.
--
André Engels, andre...@gmail.com
Scott David Daniels
First, a quote which took me a bit to find:
Thomas William Körner paraphrasing Polya and Svego
in A Companion to Analysis:
Recalling that 'once is a trick, twice is a method,
thrice is a theorem, and four times a theory,' we
seek to codify this insight.
Let us apply this insight:
Suppose in writing code, we pretty much go with that.
A method is something you notice, a theorem is a function, and
a theory is a generalized function.
Even though we like DRY ("don't repeat yourself") as a maxim, let
it go the first time and wait until you see the pattern (a possible
function). I'd go with a function first, a pair of functions, and
only then look to abstracting the function.
--Scott David Daniels
Scott....@Acm.Org
Jean-Michel Pichavant
> I've rewritten it like this:
>
> sense = cmp(func(hi), func(lo))
> assert sense != 0, "func is not strictly monotonic in [lo, hi]"
>
>
> kj
>
As already said before, unlike other languages, sense in english does
**not** mean direction. You should rewrite this part using a better
name. Wrong informations are far worse than no information at all.
JM
Tim Rowe
> Precisely. As I've stated elsewhere, this is an internal helper
> function, to be called only a few times under very well-specified
> conditions. The assert statements checks that these conditions
> are as intended. I.e. they are checks against the module writer's
> programming errors.
Good for you. I'm convinced that you have used the assertion
appropriately, and the fact that so many here are unable to see that
looks to me like a good case for teaching the right use of assertions.
For what it's worth, I read assertions at the beginning of a procedure
as part of the specification of the procedure, and I use them there in
order to document the procedure. An assertion in that position is for
me a statement to the user of the procedure "it's your responsibility
to make sure that you never call this procedure in such a way as to
violate these conditions". They're part of a contract, as somebody
(maybe you) pointed out.
As somebody who works in the safety-critical domain, it's refreshing
to see somebody teaching students to think about the circumstances in
which a procedure can legitimately be called. The hostility you've
received to that idea is saddening, and indicative of why there's so
much buggy software out there.
--
Tim Rowe
David Niergarth
the professor started us off writing a program to factor polynomials;
he probably also incorporated binary search into an assignment. But
people don't generally use Python to implement binary search or factor
polynomials so maybe you should start with a problem more germane to
typical novice users (and less algorithm-y). Wouldn't starting them
off with string processing or simple calculations be a practical way
to get comfortable with the language?
--David
On Jul 3, 9:05 am, kj <no.em...@please.post> wrote:
> I'm will be teaching a programming class to novices, and I've run
> into a clear conflict between two of the principles I'd like to
> teach: code clarity vs. code reuse. I'd love your opinion about
> it.
>
> The context is the concept of a binary search. In one of their
> homeworks, my students will have two occasions to use a binary
> search. This seemed like a perfect opportunity to illustrate the
> idea of abstracting commonalities of code into a re-usable function.
> So I thought that I'd code a helper function, called _binary_search,
> that took five parameters: a lower limit, an upper limit, a
> one-parameter function, a target value, and a tolerance (epsilon).
> It returns the value of the parameter for which the value of the
> passed function is within the tolerance of the target value.
>
> This seemed straightforward enough, until I realized that, to be
> useful to my students in their homework, this _binary_search function
> had to handle the case in which the passed function was monotonically
> decreasing in the specified interval...
>
> The implementation is still very simple, but maybe not very clear,
> particularly to programming novices (docstring omitted):
>
> def _binary_search(lo, hi, func, target, epsilon):
> assert lo < hi
> assert epsilon > 0
> sense = cmp(func(hi), func(lo))
> if sense == 0:
> return None
> target_plus = sense * target + epsilon
> target_minus = sense * target - epsilon
> while True:
> param = (lo + hi) * 0.5
> value = sense * func(param)
> if value > target_plus:
> hi = param
> elif value < target_minus:
> lo = param
> else:
> return param
>
> if lo == hi:
> return None
>
> My question is: is the business with sense and cmp too "clever"?
>
> Here's the rub: the code above is more general (hence more reusable)
> by virtue of this trick with the sense parameter, but it is also
> a bit harder to understand.
>
> This not an unusual situation. I find that the processing of
> abstracting out common logic often results in code that is harder
> to read, at least for the uninitiated...
>
> I'd love to know your opinions on this.
>
> TIA!
>
> kj
Steven D'Aprano
Absolutely.
From Webster's Dictionary:
8. (Geom.) One of two opposite directions in which a line,
surface, or volume, may be supposed to be described by the
motion of a point, line, or surface.
[1913 Webster]
And from WordNet:
2: the meaning of a word or expression; the way in which a word
or expression or situation can be interpreted
Both meanings are relevant to the way KJ is using the word. Please take
your own advice and stop giving wrong information. As a native English
speaker, I had no difficulty understanding the meaning of "sense" in the
sense intended by KJ.
--
Steven
Steven D'Aprano
LOL.
Maybe the reason for "so much buggy software" is that people
inappropriately use assert, thus changing the behaviour of code depending
on whether it is run with the -O flag or not.
I don't know what "hostility" you're seeing. The only hostility I'm
seeing is from the OP, which is bizarre considering that he asked for
advice and we gave it. What I see is a bunch of people concerned that the
OP is teaching novices a bad habit, namely, using assert for error
checking. He claims -- angrily and over and over again -- that in his
code, the assertions should never fail. Great. Good for him for knowing
when to use assert. But are the novices going to learn that lesson, or
will they simply learn "use assert for error checking"?
--
Steven
Aahz
Steven D'Aprano <st...@REMOVE-THIS-cybersource.com.au> wrote:
>On Mon, 06 Jul 2009 14:32:10 +0200, Jean-Michel Pichavant wrote:
>> kj wrote:
>>>
>>> sense = cmp(func(hi), func(lo))
>>> assert sense != 0, "func is not strictly monotonic in [lo, hi]"
>>
>> **not** mean direction. You should rewrite this part using a better
>> name. Wrong informations are far worse than no information at all.
>
>Absolutely.
>
>From Webster's Dictionary:
>
> 8. (Geom.) One of two opposite directions in which a line,
> surface, or volume, may be supposed to be described by the
> motion of a point, line, or surface.
> [1913 Webster]
>
>
>And from WordNet:
>
> 2: the meaning of a word or expression; the way in which a word
> or expression or situation can be interpreted
>
>Both meanings are relevant to the way KJ is using the word. Please take
>your own advice and stop giving wrong information. As a native English
>speaker, I had no difficulty understanding the meaning of "sense" in the
>sense intended by KJ.
As another native English speaker, I agree with Jean-Michel; this is the
first time I've seen "sense" used to mean direction.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"as long as we like the same operating system, things are cool." --piranha
Steven D'Aprano
Just goes to show you learn something new all the time.
http://www.merriam-webster.com/dictionary/sense
7: one of two opposite directions especially of motion (as
of a point, line, or surface)
http://dictionary.reference.com/browse/sense
18. Mathematics. one of two opposite directions in which
a vector may point.
Paraphrasing the Collins Dictionary of Mathematics:
The sense of a vector is the sign of the measure, contrasted with the
magnitude. Thus the vectors AB and BA have the same direction but
opposite sense. Sense is also used to distinguish clockwise and anti-
clockwise.
Sense is, if you like, a "signed direction". "Towards north" (say) as
opposed to "along the north-south axis".
--
Steven
Lie Ryan
When people are fighting over things like `sense`, although sense may
not be strictly wrong dictionary-wise, it smells of something burning...
Steven D'Aprano
> When people are fighting over things like `sense`, although sense may
> not be strictly wrong dictionary-wise, it smells of something burning...
That would be my patience.
I can't believe the direction this discussion has taken. Anybody sensible
would be saying "Oh wow, I've just learned a new meaning to the word,
that's great, I'm now less ignorant than I was a minute ago". But oh no,
we mustn't use a standard meaning to a word, heaven forbid we disturb
people's ignorance by teaching them something new.
It's as simple as this: using `sense` as a variable name to record the
sense of a function is not a code smell, any more than using `flag` to
record a flag would be, or `sign` to record the sign of an object. If you
don't know the appropriate meanings of the words sense, flag or sign,
learn them, don't dumb down my language.
--
Steven
Asun Friere
cybersource.com.au> wrote:
[snip]
> Sense is, if you like, a "signed direction".
Or to put it another way, in the graphical representation of a vector,
'direction' is the line, 'sense' is the arrowhead.
Lie Ryan
> On Tue, 07 Jul 2009 05:13:28 +0000, Lie Ryan wrote:
>
>> When people are fighting over things like `sense`, although sense may
>> not be strictly wrong dictionary-wise, it smells of something burning...
>
> That would be my patience.
>
> I can't believe the direction this discussion has taken.
Me neither.
> Anybody sensible
> would be saying "Oh wow, I've just learned a new meaning to the word,
> that's great, I'm now less ignorant than I was a minute ago". But oh no,
> we mustn't use a standard meaning to a word, heaven forbid we disturb
> people's ignorance by teaching them something new.
A meaning of a word is meaningless if nobody apart the writer
understands it. The purpose of code is 1) to communicate with the
computer, 2) to communicate with fellow programmer. The second point is
especially important if the code are written for pedantic purpose.
Teaching is largely one-way communication and often students that does
not understand about a slight point could not or would not communicate
their feelings because they think it is too silly. If the use of word is
criticized on a two-way communication channel (e.g. newsgroup), it
should raise a question of whether the word should be described first or
whether a synonym would be more suitable for the purpose. Most of these
do not apply on practical, non-pedantic purpose though, since in
non-pedantic settings you are expected to know and use the jargons
however (in)sensible they may be at first sight.
> It's as simple as this: using `sense` as a variable name to record the
> sense of a function is not a code smell, any more than using `flag` to
> record a flag would be, or `sign` to record the sign of an object.
Nobody said code smell... linguistic smell is more appropriate.
Jean-Michel Pichavant
Can't we just calm down ? I'm really sorry my ignorance started this
thread, and my apologies go to Kj who's obviously more fluent in english
than me.
I've never used sense in that way before, nor I've seen used by others
until now. However Kj is right, and my dictionary seems wrong
(wordreference.com). I've searched through others dictionaries and find
out this is actually applicable to functions. My bad.
JM
Aahz
Jean-Michel Pichavant <jeanm...@sequans.com> wrote:
>
>Can't we just calm down ? I'm really sorry my ignorance started this
>thread, and my apologies go to Kj who's obviously more fluent in english
>than me.
>I've never used sense in that way before, nor I've seen used by others
>until now. However Kj is right, and my dictionary seems wrong
>(wordreference.com). I've searched through others dictionaries and find
>out this is actually applicable to functions. My bad.
You were not the first person to point out that "sense" is a poor
variable name. Unless KJ is specifically using this code in the context
of a math class, I still think that "sense" is completely inappropriate.
Your English fluency is just fine.
Simon Forman
Pichavant<jeanm...@sequans.com> wrote:
> and my apologies go to Kj who's obviously more fluent in english than me.
> I've never used sense in that way before, nor I've seen used by others until
> now. However Kj is right, and my dictionary seems wrong (wordreference.com).
> I've searched through others dictionaries and find out this is actually
> applicable to functions. My bad.
>
Well met, sir.
pdpi
But he doesn't.
He asserts:
assert lo < hi
but then compares:
sense = cmp(func(hi), func(lo))
sense can't ever be anything other than 1. I actually find it amusing
that this threat got to 50 posts of raving discussion about assertions
without anyone spotting that.
Personally, I think the code is an unreadable mess, but that's mostly
because of all the micro optimizations, not the generality of it.
Here's my unoptimized, but still equally generic, version:
def _binary_search(lo, hi, func, target, epsilon):
sense = cmp(func(hi), func(lo))
if sense == 0:
return None
guess = (lo + hi) / 2.
while abs(func(guess) - target) > epsilon:
guess = (lo + hi) / 2.
if func(guess) > target:
hi = guess
elif func(guess) < target:
lo = guess
elif lo == hi:
return None
return guess
This is a newbie course, right? A while True loop might be faster, but
it's all sorts of wrong for teaching newbies. Likewise, calculating a
midpoint as mid = (hi + lo) * .5 is an aberration in a beginner
environment. You want your students asking why you're calculating an
average, not asking why you're multiplying by 0.5. In the same vein, I
have no words for your target_plus/target_minus cleverness.
The right way to go about it, IMO, is to give them my version, let
them digest it, make all the necessary changes to it to turn it into
your (faster) version. Present benchmarks for both, then let the
readability/performance trade-off sink in. What you achieve with this
exercise is that, instead of making your students think "I have to
read that crud!?", they'll respect that ugly code is often the result
of eking out every last drop of performance from a program as you
possibly can.
Paul Rubin
> Personally, I think the code is an unreadable mess, but that's mostly
> because of all the micro optimizations, not the generality of it.
> Here's my unoptimized, but still equally generic, version:
That version doesn't use "sense" inside the binary search, i.e. it
relies on the function being monotonically increasing.
Andre Engels
> He asserts:
> assert lo < hi
> but then compares:
> sense = cmp(func(hi), func(lo))
>
> sense can't ever be anything other than 1.
It can - there is no necessity that func is monotonically increasing.
--
André Engels, andre...@gmail.com
pdpi
> On Tue, Jul 7, 2009 at 8:01 PM, pdpi<pdpinhe...@gmail.com> wrote:
> > He asserts:
> > assert lo < hi
> > but then compares:
> > sense = cmp(func(hi), func(lo))
>
> > sense can't ever be anything other than 1.
>
> It can - there is no necessity that func is monotonically increasing.
>
> --
Yes, I realized that as I was walking home.
In other news, some of you may be interested in my seminar in advanced
foot-in-mouth placement.
pdpi
You're right, make that:
def _binary_search(lo, hi, func, target, epsilon):
sense = cmp(func(hi), func(lo))
if sense == 0:
return None
guess = (lo + hi) / 2.
while abs(func(guess) - target) > epsilon:
guess = (lo + hi) / 2.
if sense * func(guess) > target:
hi = guess
elif sense * func(guess) < target:
lo = guess
elif lo == hi:
return None
return guess
Seems I had a serious brain cramp while posting that...
pdpi
Actually, scratch that.
def _binary_search(lo, hi, func, target, epsilon):
sense = cmp(func(hi), func(lo))
if sense == 0:
return None
guess = (lo + hi) / 2.
while abs(func(guess) - target) > epsilon:
guess = (lo + hi) / 2.
if sense * func(guess) > sense * target:
hi = guess
elif sense * func(guess) < sense * target:
Dave Angel
>
> sense can't ever be anything other than 1. I actually find it amusing
> that this threat got to 50 posts of raving discussion about assertions
> without anyone spotting that.
>
>
other. If the function is monotonically decreasing, then by definition
lo<hi would guarantee that func(lo)>= func(hi), which would yield a
sense of 0 or -1.
Trivial example of monotonically decreasing:
def func(inp):
return 53.0 - inp
> Personally, I think the code is an unreadable mess, but that's mostly
> because of all the micro optimizations, not the generality of it.
> Here's my unoptimized, but still equally generic, version:
>
> def _binary_search(lo, hi, func, target, epsilon):
> sense =mp(func(hi), func(lo))
> if sense =0:
> return None
> guess =lo + hi) / 2.
> while abs(func(guess) - target) > epsilon:
> guess =lo + hi) / 2.
> if func(guess) > target:
> hi =uess
> elif func(guess) < target:
> lo =uess
> elif lo =hi:
> return None
> return guess
>
>
And of course your clarified function will fail if the func is
monotonically decreasing.
I still think that rather than using sense in the loop, one should
simply swap lo and hi, and continue.
> This is a newbie course, right? A while True loop might be faster, but
> it's all sorts of wrong for teaching newbies. Likewise, calculating a
> midpoint as mid =hi + lo) * .5 is an aberration in a beginner
> environment. You want your students asking why you're calculating an
> average, not asking why you're multiplying by 0.5. In the same vein, I
> have no words for your target_plus/target_minus cleverness.
>
> The right way to go about it, IMO, is to give them my version, let
> them digest it, make all the necessary changes to it to turn it into
> your (faster) version. Present benchmarks for both, then let the
> readability/performance trade-off sink in. What you achieve with this
> exercise is that, instead of making your students think "I have to
> read that crud!?", they'll respect that ugly code is often the result
> of eking out every last drop of performance from a program as you
> possibly can.
>
>
(untested)
def _binary_search(lo, hi, func, target, epsilon):
""" lo, hi are floats representing the desired range of input values to func()
func() is a function that takes a float argument, and returns a float result
target is the desired result value of func()
epsilon is the allowable error compared to target. If set too small, this function will fail by returning None
precondition: func is monotonic over the range of floating point inputs from lo to hi """
return a float value between lo and hi (inclusive) which yields approximately target
if func(lo) > func(hi):
lo, hi = hi, lo
if not (func(lo) <= target <= func(hi)):
return None #function doesn't have target value within the input range
guess = lo
while abs(func(guess) - target) > epsilon:
guess = (lo + hi) / 2.
if func(guess) > target:
hi = guess
elif func(guess) < target:
lo = guess
elif lo == hi:
return None #function is too steep to have a value within epsilon
return guess
The reason for the "too steep" condition is that with a finite
resolution for 'guess' epsilon could be enough smaller than target as
to make a result impossible. For example, if the slope is 45 degrees,
this happens when epsilon is about 2**51 smaller.
pdpi
> And of course your clarified function will fail if the func is
> monotonically decreasing.
yeah, I eventually realized that and corrected it... And the assert()/
cmp() confusion too. I blame lack of sleep.
The whole sign/sense thing left a really bad taste in my mouth,
though, and the swapping lo and hi suggestion of yours seems like the
neatest solution presented.
Paul Rubin
> while abs(func(guess) - target) > epsilon:
> guess = (lo + hi) / 2.
> if sense * func(guess) > sense * target:
> hi = guess
> elif sense * func(guess) < sense * target:
> lo = guess
> elif lo == hi:
> return None
> return guess
That is completely confusing. I get the, er, impression that "sense"
is supposed to change during the loop, and it takes much head
scratching to tell whether what you have there is right or not. Also,
it calls func 3 times on each loop, which could be extremely slow.
You don't know what func does, so eliminating 2/3 of the calls to it
is not a micro-optimization--it could be a major time saving.
Yet another version:
def _binary_search(x0, x1, func, target, epsilon):
y0,y1 = func(x0), func(x1)
while abs(y1 - target) > epsilon:
if x0 == x1 or cmp(y0,target) == cmp(y1,target):
return None
xn = (x0 + x1) / 2.
yn = func(xn)
if cmp(yn,target) == cmp(y0,target):
x0,y0 = xn,yn
else:
x1,y1 = xn,yn
return x1
Gabriel Genellina
<jeanm...@sequans.com> escribi�:
> I've never used sense in that way before, nor I've seen used by others
> until now. However Kj is right, and my dictionary seems wrong
> (wordreference.com). I've searched through others dictionaries and find
> out this is actually applicable to functions. My bad.
Using a common word with its common meaning is important too in order to
understand the code. It's hard enough for students to grasp the algorithm
itself, why make it artificially harder by using strange variable names.
Some years ago I had to endure using an in-house framework with names like
bring_XXX and fix_XXX instead of the usual names get_XXX and set_XXX (that
was C++, emulating properties; I'm not sure of the actual verbs used,
perhaps "obtain" and "establish", but certainly not get/set/put). Add some
undecipherable comments in spanglish, profuse usage of macros that alter
the lexical appearance of the language, and even reading code was a
torture.
--
Gabriel Genellina
pdpi
micro-optimization was perhaps too harsh, but it is an optimization
that's not obvious for the newbie, and one best learnt from experience
rather than having it spoon fed. I'll restate: you should start with
the cleanest code possible and mangle it for performance. Then again,
that implicitly assumes that f(x) is more readable than y, which is
really a matter of taste...
Tim Rowe
> Maybe the reason for "so much buggy software" is that people
> inappropriately use assert, thus changing the behaviour of code depending
> on whether it is run with the -O flag or not.
I've done my share of code review and process audits, and assertions
seem *far* to rare for that distinction.
> I don't know what "hostility" you're seeing. The only hostility I'm
> seeing is from the OP, which is bizarre considering that he asked for
> advice and we gave it. What I see is a bunch of people concerned that the
> OP is teaching novices a bad habit, namely, using assert for error
> checking. He claims -- angrily and over and over again -- that in his
> code, the assertions should never fail. Great. Good for him for knowing
> when to use assert. But are the novices going to learn that lesson, or
> will they simply learn "use assert for error checking"?
They are rather more likely to learn if they are taught, aren't they?
Or do you think it's better to keep them in the dark? "There are these
things called assertions -- work them out for yourselves".
I am convinced that the time to teach programmers to consider under
what circumstances a routine can be called and who is responsible for
ensuring that those conditions are met is as soon as they hit the
concept of calling routines. And assertions provide an excellent way
of doing that, fostering good habits for the rest of their careers.
Any hostility from the OP seems to be a response to the persistent
refusal to accept his assurances that he is not using the assertions
for run-time error checking, nor teaching the students to do that,
--
Tim Rowe
kj
>On Fri, Jul 3, 2009 at 4:05 PM, kj<no.e...@please.post> wrote:
>> I'm will be teaching a programming class to novices, and I've run
>> into a clear conflict between two of the principles I'd like to
>> teach: code clarity vs. code reuse. =A0I'd love your opinion about
>> it.
>In general, code clarity is more important than reusability.
>Unfortunately, many novice programmers have the opposite impression. I
>have seen too much convoluted code written by beginners who try to
>make the code generic. Writing simple, clear, to-the-point code is
>hard enough as it is, even when not aiming at making it reusable.
>If in the future you see an opportunity to reuse the code, then and
>only then is the time to make it generic.
>YAGNI is a wonderful principle.
Thanks!
kynn
kj
>First, a quote which took me a bit to find:
> Thomas William Körner paraphrasing Polya and Svego
> in A Companion to Analysis:
> Recalling that 'once is a trick, twice is a method,
> thrice is a theorem, and four times a theory,' we
> seek to codify this insight.
Good stuff.
>Let us apply this insight:
> Suppose in writing code, we pretty much go with that.
>A method is something you notice, a theorem is a function, and
>a theory is a generalized function.
>Even though we like DRY ("don't repeat yourself") as a maxim, let
>it go the first time and wait until you see the pattern (a possible
>function). I'd go with a function first, a pair of functions, and
>only then look to abstracting the function.
Thanks!
kynn
kj
>2009/7/4 kj <no.e...@please.post>:
>> Precisely. =A0As I've stated elsewhere, this is an internal helper
>> function, to be called only a few times under very well-specified
>> conditions. =A0The assert statements checks that these conditions
>> are as intended. =A0I.e. they are checks against the module writer's
>> programming errors.
>Good for you. I'm convinced that you have used the assertion
>appropriately, and the fact that so many here are unable to see that
>looks to me like a good case for teaching the right use of assertions.
>For what it's worth, I read assertions at the beginning of a procedure
>as part of the specification of the procedure, and I use them there in
>order to document the procedure. An assertion in that position is for
>me a statement to the user of the procedure "it's your responsibility
>to make sure that you never call this procedure in such a way as to
>violate these conditions". They're part of a contract, as somebody
>(maybe you) pointed out.
>As somebody who works in the safety-critical domain, it's refreshing
>to see somebody teaching students to think about the circumstances in
>which a procedure can legitimately be called. The hostility you've
>received to that idea is saddening, and indicative of why there's so
>much buggy software out there.
Thanks for the encouragement.
When I teach programming, the students are scientists. For the
stuff they do, correctness of the code trumps everything else. I
teach them to view assertions as way of translating their assumptions
into code. And by this I mean not only assumptions about the
correctness of their code (the typical scope of assertions), but
also, more broadly, assumptions about the data that they are dealing
with (which often comes from external sources with abysmal quality
control).
My scientific code is jam-packed with assertions. I can't count
the number of times that one such lowly assertion saved me from a
silent but potentially disastrous bug. And yes I find it distressing
that so so few programmers in my line of work use assertions at
all.
kynn
Gabriel Genellina
Nobody says you shouldn't check your data. Only that "assert" is not the
right way to do that. Ok, it's easier to write:
assert x>0 and y>0 and x<y
instead of:
if not (x>0 and y>0 and x<y):
raise ValueError, "x=%r y=%r" % (x,y)
but the assert statement is completely ignored if your script is run with
the -O option, and then no check is done.
assert should be used only to verify internal correctness only - to detect
wrong assumptions in the *code* itself. Once you're confident the code is
OK, you may turn off assertions (and improve program speed). But you must
*always* validate input data no matter what - so assert is not the right
tool.
Even worse, never DO anything inside an assertion - it won't be done in
the optimized version. This method becomes almost empty when assertions
are removed (not the intended behavior!):
def process_all(self):
for item in self.items:
assert self.process(item)
> My scientific code is jam-packed with assertions. I can't count
> the number of times that one such lowly assertion saved me from a
> silent but potentially disastrous bug. And yes I find it distressing
> that so so few programmers in my line of work use assertions at
> all.
I'm just saying that some of those assertions probably should be real "if
... raise" statements instead - not that you remove the checks. You may
use this short function, if you prefer:
def fassert(condition, message='', exc_type=AssertionError):
if not condition:
raise exc_type(message)
--
Gabriel Genellina
Ethan Furman
>
> My scientific code is jam-packed with assertions. I can't count
> the number of times that one such lowly assertion saved me from a
> silent but potentially disastrous bug.
Now imagine that asserts had been disabled for that run...
The issue is not "should you validate your inputs", the issue is "do you
want your validation able to be turned off?"
~Ethan~
Nobody
> Nobody says you shouldn't check your data. Only that "assert" is not the
> right way to do that.
"assert" is not the right way to check your *inputs*. It's a perfectly
reasonable way to check data which "should" be valid, as well as a way to
document what variables are supposed to contain.
Steven D'Aprano
Where are those variables coming from?
The distinction really boils down to this:
* asserts should never fail. If there is any chance that an assertion
might fail outside of test suites, then don't use assert.
You can't control what input the caller provides to a function, so unless
the data is generated inside the function, a caller might provide
something unexpected. Therefore, assert is inappropriate for checking
function parameters, even if that function is "private" and only called
by your own functions.
(1) So-called "private" or "internal" functions have a habit of becoming
public, and then you have asserts in public functions.
(2) If public() calls _private(x), and _private uses assert to check the
value of x, there is a risk that if public() is buggy and supplies an
invalid x, the assertion will never be checked and _private() will return
incorrect results.
(3) assert is absolutely unsuitable for enforcing pre-conditions and post-
conditions, unless such conditions are mere "guidelines", because assert
can be switched off at runtime.
It's a fine line to draw. Obviously, if we think something *truly* can
never fail, we don't bother checking it:
def add(x, y):
result = x+y
assert result-y == x
assert result-x == y
return result
is probably overkill for most programs, and yet surprisingly those
assertions will commonly fail for float arguments!
--
Steven
Jean-Michel Pichavant
convenient way to write in the code "This is a requirement, get lost if
you don't fit in". However it seems we often forget that builtin asserts
can be disabled.
One possible solution for those who prefer to write assertions instead a
'if A then Exception' is to write their own assertion statement and
**raise an exception** that will not be disabled at runtime.
Jea-Michel
Tim Rowe
> Thanks for the encouragement.
[snip]
> into code. And by this I mean not only assumptions about the
> correctness of their code (the typical scope of assertions), but
> also, more broadly, assumptions about the data that they are dealing
> with (which often comes from external sources with abysmal quality
> control).
There we diverge. A lot. If "correctness of the code trumps everything
else" (in fact, if it matters at all) and the external data has
"abysmal quality control" then it *must* be checked for correctness
before it is used. If it is not, you have no idea whether your output
is correct or not. And assertions *will* *not* reliably provide that
checking (because they may not be executed). You *must* actively check
the data, using good old-fasioned "if" statements and so on, because
not to do so is to declare that you *don't* care about correctness.
You *know* the input is often wrong, but you're not bothering to check
it?
--
Tim Rowe
Charles Yeomans
Unless, of course, you want to switch off such checking at runtime, as
you might when using a design-by-contract approach.
Charles Yeomans
J. Cliff Dyer
> On Fri, 10 Jul 2009 03:28:04 +0100, Nobody wrote:
>
> > On Thu, 09 Jul 2009 04:57:15 -0300, Gabriel Genellina wrote:
> >
> >> Nobody says you shouldn't check your data. Only that "assert" is not
> >> the right way to do that.
> >
> > "assert" is not the right way to check your *inputs*. It's a perfectly
> > reasonable way to check data which "should" be valid, as well as a way
> > to document what variables are supposed to contain.
>
> Where are those variables coming from?
>
> The distinction really boils down to this:
>
> * asserts should never fail. If there is any chance that an assertion
> might fail outside of test suites, then don't use assert.
>
I'm no expert, but the more I read this thread, and the more I think on
it, the more I believe that asserts don't really need to exist outside
of test suites. The function that assertions provide is handled in a
far more robust and maintainable way by unit tests and doctests.
Anything else should be handled by more descriptive exceptions.
The use of assertions in regular code may just be a historical baby step
on the way to real code testing.
To play devils advocate for a moment, one possible use case for assert
statements is if you need to test something that you can't easily get
under a proper unittest or doctest. Maybe you need to know the value of
a variable halfway through a method. A judicious assertion can help
supplement your test suite A better solution would be to refactor so
you can get the needed value under test, but if time constraints won't
allow it, then make your assertion and move on, but only to help you
debug the code, not to protect your code at runtime. Even then, why not
use a proper Exception (unless speed is a major issue)?
Even if it is only used internally in a module, I would still prefer a
ValueError to an AssertionError. It tells you what's happening more
clearly. And it protects you if another caller (even one internal to
the class) calls it with a different set of assumptions.
At minimum, I think there's a heavy burden on an author to justify the
use of AssertionErrors rather than other kinds of Exceptions.
Cheers,
Cliff
Robert Kern
> On Fri, 2009-07-10 at 02:57 +0000, Steven D'Aprano wrote:
>>
>>> On Thu, 09 Jul 2009 04:57:15 -0300, Gabriel Genellina wrote:
>>>
>>>> Nobody says you shouldn't check your data. Only that "assert" is not
>>>> the right way to do that.
>>> "assert" is not the right way to check your *inputs*. It's a perfectly
>>> reasonable way to check data which "should" be valid, as well as a way
>>> to document what variables are supposed to contain.
>> Where are those variables coming from?
>>
>> The distinction really boils down to this:
>>
>> * asserts should never fail. If there is any chance that an assertion
>> might fail outside of test suites, then don't use assert.
>>
>
> it, the more I believe that asserts don't really need to exist outside
> of test suites.
Actually, there is a good argument that one shouldn't use an assert statement in
test suites: code can have bugs that only show up under -O so you want to be
able to run your test suite under -O.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
J. Cliff Dyer
> On 2009-07-10 11:50, J. Cliff Dyer wrote:
> > On Fri, 2009-07-10 at 02:57 +0000, Steven D'Aprano wrote:
> >>
> >>> On Thu, 09 Jul 2009 04:57:15 -0300, Gabriel Genellina wrote:
> >>>
> >>>> Nobody says you shouldn't check your data. Only that "assert" is not
> >>>> the right way to do that.
> >>> "assert" is not the right way to check your *inputs*. It's a perfectly
> >>> reasonable way to check data which "should" be valid, as well as a way
> >>> to document what variables are supposed to contain.
> >> Where are those variables coming from?
> >>
> >> The distinction really boils down to this:
> >>
> >> * asserts should never fail. If there is any chance that an assertion
> >> might fail outside of test suites, then don't use assert.
> >>
> >
> > it, the more I believe that asserts don't really need to exist outside
> > of test suites.
>
> Actually, there is a good argument that one shouldn't use an assert statement in
> test suites: code can have bugs that only show up under -O so you want to be
> able to run your test suite under -O.
>
That's an interesting point. Presumably TestCase.assert_() doesn't
suffer from this defect? Otherwise the entire unittest suite is
essentially broken by your argument. I suppose I should have said one
should only raise AssertionErrors in test suites. Practically speaking,
that's what a test suite is: The place where you assert what the code
does.
Robert Kern
> On Fri, 2009-07-10 at 11:57 -0500, Robert Kern wrote:
>> On 2009-07-10 11:50, J. Cliff Dyer wrote:
>>> On Fri, 2009-07-10 at 02:57 +0000, Steven D'Aprano wrote:
>>>>
>>>>> On Thu, 09 Jul 2009 04:57:15 -0300, Gabriel Genellina wrote:
>>>>>
>>>>>> Nobody says you shouldn't check your data. Only that "assert" is not
>>>>>> the right way to do that.
>>>>> "assert" is not the right way to check your *inputs*. It's a perfectly
>>>>> reasonable way to check data which "should" be valid, as well as a way
>>>>> to document what variables are supposed to contain.
>>>> Where are those variables coming from?
>>>>
>>>> The distinction really boils down to this:
>>>>
>>>> * asserts should never fail. If there is any chance that an assertion
>>>> might fail outside of test suites, then don't use assert.
>>>>
>>> it, the more I believe that asserts don't really need to exist outside
>>> of test suites.
>> Actually, there is a good argument that one shouldn't use an assert statement in
>> test suites: code can have bugs that only show up under -O so you want to be
>> able to run your test suite under -O.
>>
>
> That's an interesting point. Presumably TestCase.assert_() doesn't
> suffer from this defect? Otherwise the entire unittest suite is
> essentially broken by your argument.
It explicitly raises AssertionErrors. It does not use the assert statement.
> I suppose I should have said one
> should only raise AssertionErrors in test suites. Practically speaking,
> that's what a test suite is: The place where you assert what the code
> does.
Yup.
Ethan Furman
> On Mon, 06 Jul 2009 21:02:19 -0700, Aahz wrote:
>
>
>>In article <006e795f$0$9711$c3e...@news.astraweb.com>, Steven D'Aprano
>><st...@REMOVE-THIS-cybersource.com.au> wrote:
>>
>>>On Mon, 06 Jul 2009 14:32:10 +0200, Jean-Michel Pichavant wrote:
>>>
>>>>kj wrote:
>>>>
>>>>> sense = cmp(func(hi), func(lo))
>>>>> assert sense != 0, "func is not strictly monotonic in [lo, hi]"
>>>>
>>>>As already said before, unlike other languages, sense in english does
>>>>**not** mean direction. You should rewrite this part using a better
>>>>name. Wrong informations are far worse than no information at all.
>>>
>>>Absolutely.
>>>
>>
>>>From Webster's Dictionary:
>>
>>> 8. (Geom.) One of two opposite directions in which a line,
>>> surface, or volume, may be supposed to be described by the motion
>>> of a point, line, or surface.
>>> [1913 Webster]
>>>
>>>
>>>And from WordNet:
>>>
>>> 2: the meaning of a word or expression; the way in which a word
>>> or expression or situation can be interpreted
>>>
>>>Both meanings are relevant to the way KJ is using the word. Please take
>>>your own advice and stop giving wrong information. As a native English
>>>speaker, I had no difficulty understanding the meaning of "sense" in the
>>>sense intended by KJ.
>>
>>As another native English speaker, I agree with Jean-Michel; this is the
>>first time I've seen "sense" used to mean direction.
>
>
>
> Just goes to show you learn something new all the time.
>
> http://www.merriam-webster.com/dictionary/sense
>
> 7: one of two opposite directions especially of motion (as
> of a point, line, or surface)
>
>
> http://dictionary.reference.com/browse/sense
>
> 18. Mathematics. one of two opposite directions in which
> a vector may point.
>
>
>
> Paraphrasing the Collins Dictionary of Mathematics:
>
> The sense of a vector is the sign of the measure, contrasted with the
> magnitude. Thus the vectors AB and BA have the same direction but
> opposite sense. Sense is also used to distinguish clockwise and anti-
> clockwise.
>
> Sense is, if you like, a "signed direction". "Towards north" (say) as
> opposed to "along the north-south axis".
>
This also illustrates the importance of knowing your target audience. I
have also not seen "sense" used this way before, and from the placement
in the dictionaries I would venture to say it's not common usage outside
of mathematics and the sciences.
Of course, since kj is teaching biologists, odds are decent they know
what he's talking about.
~Ethan~
Steven D'Aprano
>> (3) assert is absolutely unsuitable for enforcing pre-conditions and
>> post-
>> conditions, unless such conditions are mere "guidelines", because
>> assert
>> can be switched off at runtime.
>
>
> Unless, of course, you want to switch off such checking at runtime, as
> you might when using a design-by-contract approach.
So is design-by-contract just another way of saying "let's hope the data
is valid, because if it's not, we're screwed"?
Perhaps Python should have a new statement, `assume`, used just like
`assert` except it never actually performs the test or raises an error.
--
Steven
Terry Reedy
The manual says quite clearly
"The simple form, assert expression, is equivalent to
if __debug__:
if not expression: raise AssertionError"
It should be used when and only when one actually wants the double
condition check, and not as an abbreviation of the second conditional only.
tjr
Tim Rowe
> So is design-by-contract just another way of saying "let's hope the data
> is valid, because if it's not, we're screwed"?
Not at all. Design By Contract is about assigning responsibility for
checking. If you don't assign responsibility then a pile of things end
up getting checked over and over again, because everybody is assuming
responsibility, and some things don't get checked at all because
everyone assumes that somebody else is checking.
In DbC, the pre-condition on data coming in to a program from outside
will usually simply be "true" -- nothing at all is assumed about it.
But when you pass it from an input conditioning routine to a
processing routine, the input conditioning routine should be able to
make guarantees about what it passes, and the processing routine
should be able to assume that those guarantees are met without having
to check it again (after all, what was the conditioning routine there
for?). Assertions might be useful in testing (because they save having
to repeat the same set of test cases on absolutely every test run) and
they're certainly useful in documenting, but if they're any use at all
in the production run-time then there's something wrong with your
development and testing processes.
--
Tim Rowe
Albert van der Horst
Dennis Lee Bieber <wlf...@ix.netcom.com> wrote:
>On 07 Jul 2009 05:05:12 GMT, Steven D'Aprano
><st...@REMOVE-THIS-cybersource.com.au> declaimed the following in
>gmane.comp.python.general:
>
>> Paraphrasing the Collins Dictionary of Mathematics:
>>
>> clockwise.
>>
>regards to quantum spin states in such things as Scientific American
>(though that magazine has either gone down hill in the last 30 years, or
>my expectations have gone up... Current issues read like the first years
>of Discover magazine)
I dropped my subscription when power was expressed in multiples
of US hair dryers. (I could calculate that back, and was appalled
by the energy wastage of US hair dryers. ;-) )
>--
> Wulfraed Dennis Lee Bieber KD6MOG
Groetjes Albert
--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
Albert van der Horst
>
>
>I'm will be teaching a programming class to novices, and I've run
>into a clear conflict between two of the principles I'd like to
>it.
>
<SNIP>
>
>This seemed straightforward enough, until I realized that, to be
>useful to my students in their homework, this _binary_search function
>had to handle the case in which the passed function was monotonically
>decreasing in the specified interval...
>
<SNIP>
>
>Here's the rub: the code above is more general (hence more reusable)
>by virtue of this trick with the sense parameter, but it is also
>a bit harder to understand.
>
>This not an unusual situation. I find that the processing of
>abstracting out common logic often results in code that is harder
>to read, at least for the uninitiated...
Yes, of course. You're teaching, say, green belt programmers.
Good reusable code requires a fifth dan. You may not be up to it
yourself (no offense intended), let alone your students.
Writing a reusable binary search is a very different assignment
from being able to code/use it in a concrete example.
Their choice must be between *writing* a one-off binary search,
versus *using* a brilliantly code, brilliantly documented
binary search that has been given beforehand.
Even assuming the reusable code is perfect in all sense, reusing may
be more effort than writing from scratch.
Then *you* have to explain them about the benefits of reuse.
(The other benefits, of course.)
>
>I'd love to know your opinions on this.
You're welcome.
>kj
Albert van der Horst
>>
>>> When people are fighting over things like `sense`, although sense may
>>> not be strictly wrong dictionary-wise, it smells of something burning...
>>
>> That would be my patience.
>>
>> I can't believe the direction this discussion has taken.
>
>Me neither.
>
>> Anybody sensible
>> would be saying "Oh wow, I've just learned a new meaning to the word,
>> that's great, I'm now less ignorant than I was a minute ago". But oh no,
>> we mustn't use a standard meaning to a word, heaven forbid we disturb
>> people's ignorance by teaching them something new.
>
>A meaning of a word is meaningless if nobody apart the writer
>understands it. The purpose of code is 1) to communicate with the
Exactly. And the OP teaches to scientist. They know sense in that
meaning. Maybe you don't, but that is irrelevant.
<SNIP>
Groetjes Albert.