Why assert is not a function?

[Marco Sulla]

I have a curiosity. Python, as many languages, has assert as a
keyword. Can't it be implemented as a function? Is there an advantage
to have it as a keyword?

[Chris Angelico]

It could, but it would need some magic. A lot of test frameworks have
their own set of assertions and you have to pick the one you want -
for instance:

assertEqual(x, y)

instead of

assert x == y

The trouble is that, if you write a function for this, it will just
get True or False, without any explanation of the cause - making it
somewhat unhelpful when something goes wrong. To compensate, function
equivalents inevitably have to have myriad types of assertions, where
the language just needs one.

ChrisA

[Stestagg]

There is also the, I think seldom used, feature that calling python with
'-O' removes all assert statements at parse/compile time (I believe for
performance reasons)

So for example:

$ python -c 'assert False'
Traceback (most recent call last):
File "<string>", line 1, in <module>
AssertionError

But:

$ python -O -c 'assert False'
[no output]
$

Ignoring the question about this feature being particularly useful, it
would be much harder to implement if assert were a standard function (and
re-bindable), as the 'arguments' in the assert construct are not even
executed when '-O' is used (this is the performance gain):

$ python -O -c 'assert DoesNotExist'
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'DoesNotExist' is not defined

vs.

$ python -O -c 'assert DoesNotExist'
[no output]
$

so the compiler would have to do a lot of work to identify assert calls and
remove them.

Steve

> --
> https://mail.python.org/mailman/listinfo/python-list
>

[Barry]

> On 2 Mar 2021, at 20:54, Marco Sulla <Marco.Sul...@gmail.com> wrote:
>
> I have a curiosity. Python, as many languages, has assert as a
> keyword. Can't it be implemented as a function? Is there an advantage
> to have it as a keyword?

assert condition, expression

Only is condition is false with expression be evaluated.
So you can safely do expensive things I the expression with incuring and cost if the condition is True.

With a function assert the 2nd part would have to evaluated regardless of the state of the condition.
Which would slow down the code for no benefit.

Barry

> --
> https://mail.python.org/mailman/listinfo/python-list
>

[Paul Rubin]

Barry <ba...@barrys-emacs.org> writes:
> assert condition, expression
> Only is condition is false with expression be evaluated.

assert (condition, lambda: expression)

[Mirko]

Am 02.03.2021 um 23:09 schrieb Stestagg:
> Ignoring the question about this feature being particularly useful, it

It is useful because "assert" is primarily (if not purely and
exclusive) a debugging tool during development and testing.

In production code you don't want any asserts, but logging. Having
"assert" being a function would make it much harder to get rid of it
in production code.

[Chris Angelico]

Really?

if PRODUCTION:
def assert(*a, **kw): pass

would work if it were a function :)

ChrisA

[Hexamorph]

Am 03.03.2021 um 00:24 schrieb Chris Angelico:
> On Wed, Mar 3, 2021 at 10:22 AM Mirko via Python-list
> <pytho...@python.org> wrote:
>>
>> Am 02.03.2021 um 23:09 schrieb Stestagg:
>>> Ignoring the question about this feature being particularly useful, it
>>
>> It is useful because "assert" is primarily (if not purely and
>> exclusive) a debugging tool during development and testing.
>>
>> In production code you don't want any asserts, but logging. Having
>> "assert" being a function would make it much harder to get rid of it
>> in production code.
>>
>
> Really?

Yes.

> if PRODUCTION:
> def assert(*a, **kw): pass
>
> would work if it were a function :)

With the "if PRODUCTION" being superfluously in the production code
(not to mention the possible NameError otherwise).

[Chris Angelico]

Whatever method you have for determining whether you're in debug or
prod, you can use it to decide whether or not to create the function.
It's that easy.

That said, though... I prefer to keep my assertions in prod too.

ChrisA

[Greg Ewing]

On 3/03/21 12:24 pm, Chris Angelico wrote:
> if PRODUCTION:
> def assert(*a, **kw): pass
>
> would work if it were a function :)

But would cost you a useless function call for every assert
in production mode.

--
Greg

[Chris Angelico]

A micro-optimization that would almost never actually impact your
code. But considering that I'm willing to keep *the entire check*
active, it's possible I'm slightly biased in favour of debuggability
at the cost of CPU...

ChrisA

[Paul Rubin]

Chris Angelico <ros...@gmail.com> writes:
> it's possible I'm slightly biased in favour of debuggability
> at the cost of CPU...

I like to think all of us are biased that same way. Otherwise, we wouldn't
be using Python in the first place.

[Terry Reedy]

On 3/1/2021 5:51 PM, Marco Sulla wrote:
> I have a curiosity. Python, as many languages, has assert as a
> keyword. Can't it be implemented as a function? Is there an advantage
> to have it as a keyword?

One does not need to turn the test expression into a function. One does
not need to wrap the message expression to delay evaluation.


--
Terry Jan Reedy

[Grant Edwards]

On 2021-03-02, Chris Angelico <ros...@gmail.com> wrote:
> On Wed, Mar 3, 2021 at 10:22 AM Mirko via Python-list><pytho...@python.org> wrote:
>
>> In production code you don't want any asserts, but logging. Having
>> "assert" being a function would make it much harder to get rid of
>> it in production code.
>
> Really?
>

> if PRODUCTION:
> def assert(*a, **kw): pass
>
> would work if it were a function :)

Wouldn't that still evaluate all of the arguments? You get rid of the
value of the assert, but retain almost all of the cost.

I thought the entire point of asser being a keyword was so that if you
disable asserts then they go away completely: the arguments aren't
even evaluated.

--
Grant

[Chris Angelico]

It depends on what the point of "removing the assertions" is, but yes,
that will indeed still evaluate the arguments. IMO the cost of running
assertions isn't that high compared to the value of keeping them
(which is why I never run -O), and the performance argument is a weak
one compared to the much stronger value of having the actual failing
expression available in the exception report.

ChrisA

[David Lowry-Duda]

> assert condition, expression
>
> Only is condition is false with expression be evaluated.

> So you can safely do expensive things I the expression with incuring
> and cost if the condition is True.

I think I've only every used a string as the expression. Have you found
it useful to use complicated, expensive expressions? Could you give an
example?

- DLD

[Barry Scott]

In the test suite of the product I work on its common to put a lot of information into the expression.
For example if a HTTP request returns unexpected content we will include all the headers and
body of the that the request returners.

assert http_status == '200', 'Request failed status %r Body:\n%s' % (http_status, http_header_and_body)

Barry


>
> - DLD
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[Chris Angelico]

That doesn't look like the sort of thing that should be an assertion.
Assertions should be for things that, if your code were bug-free,
could never happen.

ChrisA

[Barry Scott]

> On 3 Mar 2021, at 18:41, Chris Angelico <ros...@gmail.com> wrote:

If the code works the status will be 200 and the test will pass.

Now you can argue that the test suite should use something better then assert,
but assert is what the code that I inherited uses. Its origins go back many many
years. Its would be a lot of work to modernise as there are 1,000s of test suites
affect.

Barry


>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list <https://mail.python.org/mailman/listinfo/python-list>

[Grant Edwards]

Good point. I had forgotten about having the expression available in
the exception output. That's definitly valuable.

[Cameron Simpson]

Did we all see the recently announced ycecream PyPI module? Very cool!

See: https://github.com/salabim/ycecream

Cheers,
Cameron Simpson <c...@cskk.id.au>

[Serhiy Storchaka]

03.03.21 01:24, Chris Angelico пише:

> On Wed, Mar 3, 2021 at 10:22 AM Mirko via Python-list
> <pytho...@python.org> wrote:
>>

>> Am 02.03.2021 um 23:09 schrieb Stestagg:
>>> Ignoring the question about this feature being particularly useful, it
>>
>> It is useful because "assert" is primarily (if not purely and
>> exclusive) a debugging tool during development and testing.
>>

>> In production code you don't want any asserts, but logging. Having
>> "assert" being a function would make it much harder to get rid of it
>> in production code.
>>
>
> Really?
>
> if PRODUCTION:
> def assert(*a, **kw): pass
>
> would work if it were a function :)

assert(expensive_computation())

[Chris Angelico]

Do you have any asserts like that, or is that a purely theoretical
complaint? I have never once seen anything that costly - usually it'll
be something like checking a length (and this isn't C's strlen, since
Python can get lengths of all built-in types quickly), or some simple
checks.

Having assert be a function would not make it much harder to get rid
of. It would just make it harder to get the text.

ChrisA

[Serhiy Storchaka]

11.03.21 20:31, Chris Angelico пише:

>> assert(expensive_computation())
>
> Do you have any asserts like that, or is that a purely theoretical
> complaint? I have never once seen anything that costly - usually it'll
> be something like checking a length (and this isn't C's strlen, since
> Python can get lengths of all built-in types quickly), or some simple
> checks.

I have a lot of asserts when use other programming languages. Even
simple bound check for index can be too expensive for optimized build.
It is less common in Python because it is less common to use -O option
in Python. If most users do not bother to use -O option, you do not want
to make your library slower for them by adding runtime self-checks. It
is not possible to enable optimization on per-package level.

[Cameron Simpson]

On 12Mar2021 05:31, Chris Angelico <ros...@gmail.com> wrote:
>On Fri, Mar 12, 2021 at 3:37 AM Serhiy Storchaka <stor...@gmail.com> wrote:

>> assert(expensive_computation())
>
>Do you have any asserts like that, or is that a purely theoretical
>complaint? I have never once seen anything that costly - usually it'll
>be something like checking a length (and this isn't C's strlen, since
>Python can get lengths of all built-in types quickly), or some simple
>checks.

That's very much in the eye of the beholder. Usually my asserts are
pretty cheap. But there are two scenarios where they accrue significant
cost.

Scenario 1 is where I'm maintaining some data structure, possibly with
fiddly corner cases. That associated class may have a self_check method
to verify integrity, and that can land in an assertion. Potentially
quite expensive.

Scenario 2 is the performance critical function which nonetheless is
complicated (or just... long). I've written a few of these and littering
the code with asserts becomes necessary to check correctness,
particularly if you're modifying it. Not everything lends itself to unit
tests - we often want to be assured of things in the middle of a
process.

In scenario 2, each assert is pretty cheap, but their cumulative effect
has a significant performance impact. Being able to turn them off
altogether is a distinct real world win.

(To those crying "break it up", sometimes that is hard because of the
embodied state, and sometimes that is unwanted because of the
performance impact (function calls aren't free and neither is the
packaging-into-parameters needed to turn an operation into a function
call); for 99% of code both of these are cheap enough, but in this niche
they're a problem.

>Having assert be a function would not make it much harder to get rid
>of. It would just make it harder to get the text.

Hah. I repeat my mention of the ycecream package - very neat!

Cheers,
Cameron Simpson <c...@cskk.id.au>

[dn]

Having read this, and @Chris' points, am wondering if I'm missing a/the
point:-

(Scenarios 1 and 2, plus leaving the asserts to run in case of
'accidents' during production-execution)

When testing the integrity of some collection of data, why use assert
over raising a descriptive and class-identified exception?

Both can be trapped by 'higher-level' code. Both can provide
text-planations.

Is assert so much faster/cheaper than try...except...raise?
--
--
Regards,
=dn

[Ethan Furman]

On 3/11/21 1:45 PM, dn via Python-list wrote:

> Is assert so much faster/cheaper than try...except...raise?

Infinitely faster when they are not there. ;-)

Basically, you are looking at two different philosophies:

- Always double check, get good error message when something fails

vs

- check during testing and QA, turn off double-checks for production for best performance possible.

--
~Ethan~

[Chris Angelico]

There are many hybrids available too though. For instance:

if __debug__ or args.verify:
def verify(thing):
...
raise Whatever
else:
def verify(thing): pass

Yes, you pay the price of a function call even if you're not verifying
the full structural integrity. But that's a lot cheaper than the full
check.

Advantage here is that you can use -O to suppress, or you can control
it with an arg, or whatever.

If you're doing the same check in lots of places, and it's costly,
assertions aren't really a great fit.

ChrisA

[dn]

Perhaps I misunderstood (and haven't gone back to check - mea culpa),
but the impression-gained was that -O many not be used, even "in
production", for some reason?

Perhaps because I've not come from a language where assert played any/a
major rôle, but am still hoping for some discussion/understanding as to
why/when assert might be better than try...except in every/particular
situations...
--
Regards,
=dn

[Cameron Simpson]

On 12Mar2021 12:53, DL Neil <Pytho...@DancesWithMice.info> wrote:
>On 12/03/2021 11.27, Chris Angelico wrote:
>> On Fri, Mar 12, 2021 at 9:10 AM Ethan Furman <et...@stoneleaf.us> wrote:
>>> On 3/11/21 1:45 PM, dn via Python-list wrote:
>>>> Is assert so much faster/cheaper than try...except...raise?
>>>
>>> Infinitely faster when they are not there. ;-)

[...]

>> There are many hybrids available too though. For instance:
>>
>> if __debug__ or args.verify:
>> def verify(thing):
>> ...
>> raise Whatever
>> else:
>> def verify(thing): pass
>>
>> Yes, you pay the price of a function call even if you're not verifying
>> the full structural integrity. But that's a lot cheaper than the full
>> check.
>>
>> Advantage here is that you can use -O to suppress, or you can control
>> it with an arg, or whatever.
>>
>> If you're doing the same check in lots of places, and it's costly,
>> assertions aren't really a great fit.
>
>Perhaps I misunderstood (and haven't gone back to check - mea culpa),
>but the impression-gained was that -O many not be used, even "in
>production", for some reason?

News to me. Perhaps that was someone's scenario.

To me, asserts have 2 primary features: (a) they're easy to write (and
read) versus "if some_test: raise SomeException("blah blah...")" and (b)
they are outright _absent_ from the code when run with -O.

_Provided_ the code called from the assert has no side effects, dropping
the asserts should always make for identical -O behaviour vs no -O.

Chris' example above walks the middle ground providing something richer
that a plain assert while still (almost) vanishing with -O (and no
args.verify mode switch).

>Perhaps because I've not come from a language where assert played any/a
>major rôle, but am still hoping for some discussion/understanding as to
>why/when assert might be better than try...except in every/particular
>situations...

I find assert visually low impact. Try/except is quite wordy and brings
more indentation.

One has to keep in mind the use case.

For me, try/except is for when something might reasonably "go wrong" in
normal use, even niche normal use. Whereas assert is for things which
should _never_ occur. Roughly, again for me, try/except if for catching
misuse and assert is for catching misdesign/misimplementation.

Cheers,
Cameron Simpson <c...@cskk.id.au>

[Chris Angelico]

On Fri, Mar 12, 2021 at 3:53 PM Cameron Simpson <c...@cskk.id.au> wrote:
> For me, try/except is for when something might reasonably "go wrong" in
> normal use, even niche normal use. Whereas assert is for things which
> should _never_ occur. Roughly, again for me, try/except if for catching
> misuse and assert is for catching misdesign/misimplementation.

Something like that, yeah. An assertion failure represents a bug *in
this code*, something that shouldn't ever happen. If it's possible to
trigger the failure with some other piece of code (calling something
with bad arguments, or whatever), then assert is the wrong tool for
the job. Similarly, if you find yourself catching AssertionError
anywhere outside of unit testing, something is horribly wrong
somewhere :)

ChrisA

[Mike Dewhirst]

I haven't been following this thread so please forgive me if this has
been said ...

My understanding of the reason for assert is to support the "design by
contract" programming style of Eiffel as espoused by Bertrand Meyer. I
don't suppose it makes much difference whether it is a function or a
callable for that - and when I first saw it I was appropriately confused
- but I only ever used it when I was absolutely certain the assertion
was bullet-proof. And it is a long time since I did that. I prefer
try-except nowadays.

Mike

>
> ChrisA


--
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Just
ask and I'll send it to you. Your email software can handle signing.

[Chris Angelico]

On Fri, Mar 12, 2021 at 5:03 PM Mike Dewhirst <mi...@dewhirst.com.au> wrote:
>
> On 12/03/2021 4:31 pm, Chris Angelico wrote:
> > On Fri, Mar 12, 2021 at 3:53 PM Cameron Simpson <c...@cskk.id.au> wrote:
> >> For me, try/except is for when something might reasonably "go wrong" in
> >> normal use, even niche normal use. Whereas assert is for things which
> >> should _never_ occur. Roughly, again for me, try/except if for catching
> >> misuse and assert is for catching misdesign/misimplementation.
> > Something like that, yeah. An assertion failure represents a bug *in
> > this code*, something that shouldn't ever happen. If it's possible to
> > trigger the failure with some other piece of code (calling something
> > with bad arguments, or whatever), then assert is the wrong tool for
> > the job. Similarly, if you find yourself catching AssertionError
> > anywhere outside of unit testing, something is horribly wrong
> > somewhere :)
>
> I haven't been following this thread so please forgive me if this has
> been said ...
>
> My understanding of the reason for assert is to support the "design by
> contract" programming style of Eiffel as espoused by Bertrand Meyer. I
> don't suppose it makes much difference whether it is a function or a
> callable for that - and when I first saw it I was appropriately confused
> - but I only ever used it when I was absolutely certain the assertion
> was bullet-proof. And it is a long time since I did that. I prefer
> try-except nowadays.
>

Definitely use something other than assertions for that.

ChrisA

[Richard Damon]

On 3/12/21 12:31 AM, Chris Angelico wrote:
> On Fri, Mar 12, 2021 at 3:53 PM Cameron Simpson <c...@cskk.id.au> wrote:
>> For me, try/except is for when something might reasonably "go wrong" in
>> normal use, even niche normal use. Whereas assert is for things which
>> should _never_ occur. Roughly, again for me, try/except if for catching
>> misuse and assert is for catching misdesign/misimplementation.
> Something like that, yeah. An assertion failure represents a bug *in
> this code*, something that shouldn't ever happen. If it's possible to
> trigger the failure with some other piece of code (calling something
> with bad arguments, or whatever), then assert is the wrong tool for
> the job. Similarly, if you find yourself catching AssertionError
> anywhere outside of unit testing, something is horribly wrong
> somewhere :)
>

> ChrisA

Chris, I would disagree with that statement. An assert says that there
is something wrong with THE code, not THIS code. It is perfectly
reasonable, and good defensive programming, to assert on the
per-conditions to the procedure at its beginning.

Now, it may be true that a 'friendlier' procedure may be defined to
check some of these and return an error, but that then locks that
behavior into the API, so the cost of the check becomes an absolute
requirement.

In langauges like C, the assert for this may be more important because
the language provides more opportunity for 'undefined behavior'.

It is reasonable to skip the input assert if it becomes too expensive
for benefit it provides, or if something else will catch the error. This
likely actually applies to a lot of Python code, so it may seem that it
doesn't apply.

--
Richard Damon

[Chris Angelico]

On Sat, Mar 13, 2021 at 12:11 AM Richard Damon <Ric...@damon-family.org> wrote:
>
> On 3/12/21 12:31 AM, Chris Angelico wrote:
> > On Fri, Mar 12, 2021 at 3:53 PM Cameron Simpson <c...@cskk.id.au> wrote:
> >> For me, try/except is for when something might reasonably "go wrong" in
> >> normal use, even niche normal use. Whereas assert is for things which
> >> should _never_ occur. Roughly, again for me, try/except if for catching
> >> misuse and assert is for catching misdesign/misimplementation.
> > Something like that, yeah. An assertion failure represents a bug *in
> > this code*, something that shouldn't ever happen. If it's possible to
> > trigger the failure with some other piece of code (calling something
> > with bad arguments, or whatever), then assert is the wrong tool for
> > the job. Similarly, if you find yourself catching AssertionError
> > anywhere outside of unit testing, something is horribly wrong
> > somewhere :)
> >
> > ChrisA
>
> Chris, I would disagree with that statement. An assert says that there
> is something wrong with THE code, not THIS code. It is perfectly
> reasonable, and good defensive programming, to assert on the
> per-conditions to the procedure at its beginning.
>
> Now, it may be true that a 'friendlier' procedure may be defined to
> check some of these and return an error, but that then locks that
> behavior into the API, so the cost of the check becomes an absolute
> requirement.
>

It's perfectly reasonable to put "if condition: raise Blah", but is it
really reasonable to use the assert statement, which (a) might not be
run, and (b) raises a very generic exception? Make assertions about
your own code, not other people's.

ChrisA

[Marco Sulla]

On Thu, 11 Mar 2021 at 23:11, Ethan Furman <et...@stoneleaf.us> wrote:
> Basically, you are looking at two different philosophies:
>
> - Always double check, get good error message when something fails
>
> vs
>
> - check during testing and QA, turn off double-checks for production for best performance possible.

In a perfect world, I said the second option is the best. But for the
majority of projects I contributed, speed was not a critical issue. On
the contrary, it's very hard to get meaningful informations about
problems in production, so I'm in favour of the first school :)

[Richard Damon]

My issue with that is that I feel that if the program raises an
exception, if SHOULD document what exceptions it raises and under what
conditions. That means that the detection of bad parameters has now been
moved from being an 'improper' use of the module, to being defined to be
checked for, and thus now a required part of the API.

Exceptions are designed for errors that might be correctable by outer
layers, program errors that you catch with assert tend not to fall into
that category.

Asserts indicate that there is a programming error that has been
detected, and the operation should be aborted. The exception generated
will not be used to 'correct' the error, but might reformat the assert
message and facilitate its reporting, or for an interactive program,
perhaps provide a path for the user to save his work or work around the
program bug.

The are conceptually very different sorts of things, and should not be
interchanged.

Note, it is often hard to tell if the 'impossible' state you ended up in
is truly a fault with your own code or the code that is calling you, so
often the detecting of bad parameters is really akin to detecting your
routine has gotten into a state it should not be in, the only difference
is you have detected this before you have done anything, and thus help
locate where the bug is.

--
Richard Damon

[Robert Latest]

Chris Angelico (and oters) wrote:

[interesting stuff]

I'm a late contributor here, but I'd just say: If you'd prefer a function for
assert, just define a function that does what you want and be done with it.
Much harder to do if assert() were a built-in function but you'd rather have a
keyword ;-)

robert