[Python-ideas] Pre-conditions and post-conditions

[Marko Ristin-Kaufmann]

Hi,

I would be very interested to bring design-by-contract into python 3. I find design-by-contract particularly interesting and indispensable for larger projects and automatic generation of unit tests.

I looked at some of the packages found on pypi and also we rolled our own solution (https://github.com/Parquery/icontract/). I also looked into https://www.python.org/dev/peps/pep-0316/.

However, all the current solutions seem quite clunky to me. The decorators involve an unnecessary computational overhead and the implementation of icontract became quite tricky once we wanted to get the default values of the decorated function.

Could somebody update me on the state of the discussion on this matter?

I'm very grateful for any feedback on this!

[Elazar]

You might also be interested in pep-563. although it is not intended for design by contract, it can help (syntactically).

Elazar 

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Jonathan Fine]

Hi Marko

Thank you for introducing yourself, and clearly stating your question.
That helps us all. You asked:

> Could somebody update me on the state of the discussion on this matter?

I think bring the existing PEP up to date would be a good starting
point. Its content hasn't been changed since 2003 (except for PEP-wide
admin changes. (Recall that Python 3.0 was released in 2008.)

https://www.python.org/dev/peps/pep-0316/
https://github.com/python/peps/commits/master/pep-0316.txt

In fact, revising the PEP might be enough to answer your question.
What do you think, Marko?

Experts: is there a process for revising old PEPs, such as this one?
Or at least a precedent we could follow (or adapt)?

--
Jonathan

[Paul Moore]

On Thu, 16 Aug 2018 at 10:41, Jonathan Fine <jfin...@gmail.com> wrote:
>
> Hi Marko
>
> Thank you for introducing yourself, and clearly stating your question.
> That helps us all. You asked:
>
> > Could somebody update me on the state of the discussion on this matter?
>
> I think bring the existing PEP up to date would be a good starting
> point. Its content hasn't been changed since 2003 (except for PEP-wide
> admin changes. (Recall that Python 3.0 was released in 2008.)
>
> https://www.python.org/dev/peps/pep-0316/
> https://github.com/python/peps/commits/master/pep-0316.txt
>
> In fact, revising the PEP might be enough to answer your question.
> What do you think, Marko?
>
> Experts: is there a process for revising old PEPs, such as this one?
> Or at least a precedent we could follow (or adapt)?

I'm not aware of a formal process, but I'd have thought the following
steps would be a reasonable approach:

1. Review the PEP, and research the discussions that happened at the
time, particularly of interest is why the PEP was deferred.
2. Consider what (if anything) has changed since the original deferral
(which could simply be "time has moved on, people's views may have
changed" but ideally would include a bit more in the way of concrete
motivation).
3. Contact the original PEP author and ask if he is interested in
reopening the discussion, collaborating on a revision, or handing the
PEP over.
4. Start up a discussion here, pointing out the original PEP and
summarising the previous debate and why you want to restart the
discussion. If you're hoping to change the details of the original
PEP, summarise your changes and why you feel they are an improvement
over the original.

To answer the OP's question more directly:

> Could somebody update me on the state of the discussion on this matter?

As far as I am aware, there has been no discussion on this subject
since the PEP 316 discussions which ended up in its deferral. Elazar
mentioned PEP 563, and there *may* have been mention of design by
contract uses in the discussions on that PEP, but you'd have to search
the mailing list archives to confirm that one way or another.

Hence the suggestions that if you want to restart discussion, reviving
PEP 316 is likely the best approach.

Paul

[Marko Ristin-Kaufmann]

Hi Jonathan and Paul,

Thank you very much for your suggestions! I will try to contact the author of the PEP.

Let me clarify a bit a potential misunderstanding. Please mind that contracts are not tied to individual variables, but to expressions. Think of it as defining a lambda which takes as input all the arguments of the function (and a result variable in case of post-conditions) which always needs to evaluate to True.

Cheers,

Marko

[Daniel Moisset]

I think that annotations were suggested because you could write an expression there without getting evaluated. 

I've thought about this problem many times in the past (as a Python dev with a long history working in Eiffel too).... For me one of the crucial issue that is hard to translate into the python model is that the assertions (say, a function precondition) are not conceptually part of the function itself, but the interface of the class. The "natural" python ways of attaching these assertions somehow to the function object do not work when you also use inheritance, because when you override a method the new function object clobbers the previous one. I've experimented at some point on how to put them in classes (and doing metaclass or __getattribute__ tricks) but nothing convinced me). In general, the way that python puts method call and inheritance semantic in a specific layout of runtime objects (which in general is really clever) seems to be a bit alien to the DbC idea where the asbtraction/interface of the class is conceptually separate and has independent information wrt to the runtime objects.

--

Daniel Moisset Technical Leader A:   1 Fore St, EC2Y 9DT London P:   +44 7398 827139   M:   dmoi...@machinalis.com  |   S:   dmoisset

[Wes Turner]

pycontracts may be worth a look.

https://andreacensi.github.io/contracts/

- @contract decorator, annotations, docstrings

IDK if pycontracts supports runtime parameter validations that involve more than one parameter.

Inheritance does appear to be supported,

as are numpy array dimension constraints.

I can't recall whether the pycontracts expression language precedes MyPy compile-time annotations; both with one syntax really would be great.

[Marko Ristin-Kaufmann]

Hi,

I had a look at the messages related to the PEP in question (PEP 316) in the archive. As far as I can tell, the main objection is that you can achieve contracts by implementing it with decorators.

I think that these objections miss what actually Daniel Moisset wrote in his message: contracts are more than pre- and post-condition checks on a function. The inheritance of decorators does not imply just inheriting the pre- and post-conditions, but also relaxing and tightening them (represented by "require else" and "ensure then" in Eiffel). If this is to be used effectively in practice with little overhead then we would either need to introduce new syntax to the language or make the compiler improve the byte code on the fly.

Is there any chance to introduce these constructs in the language or is it too small a feature for such a big change?

Since we wanted to have contracts in Golang, we implemented a tool that synchronizes the documentation of a function with the function code (https://github.com/Parquery/gocontracts). Maybe this is the easier path to follow in Python as well?

@Wes Turner: thanks for pointing to pycontracts. I'm aware of the library. It implements only contracts based on a single property. We found that limiting and rolled out our own solution that suited us much better:
https://github.com/Parquery/icontract/

I also found informative messages on contract breach to be particularly important for fast development and error inspection in the production.

Cheers,

Marko

[Steven D'Aprano]

On Tue, Aug 21, 2018 at 09:06:54AM +0200, Marko Ristin-Kaufmann wrote:

> Is there any chance to introduce these constructs in the language or is it
> too small a feature for such a big change?

I don't think contracts is a small feature. I think it is a HUGE
feature, but under-appreciated by most developers. (Probably due to
unfamiliarity, and the difficulty in using it in a language with no
syntactic support.)

Whether it is practical to add it to Python, I don't know. I suspect
that we would have to develop some sort of third-party solution first,
even if it did not do everything contracts ought to do (or do them less
efficiently) as a proof of concept. PyContracts is probably a good place
to start. For those who missed Wes' email:

https://andreacensi.github.io/contracts/

Cobra is another good place to look, as it demonstrates a
nice syntax that reads like a cross between Python and Eiffel:

http://cobra-language.com/trac/cobra/wiki/Contracts





--
Steve

[INADA Naoki]

On Thu, Aug 16, 2018 at 4:19 PM Elazar <ela...@gmail.com> wrote:
>
> You might also be interested in pep-563. although it is not intended for design by contract, it can help (syntactically).
>

FYI, PEP 563 doesn't help it.

Read this section:
https://www.python.org/dev/peps/pep-0563/#non-typing-usage-of-annotations

"With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated."

--
INADA Naoki <songof...@gmail.com>

[Paul Moore]

On Tue, 21 Aug 2018 at 11:27, Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Tue, Aug 21, 2018 at 09:06:54AM +0200, Marko Ristin-Kaufmann wrote:
>
> > Is there any chance to introduce these constructs in the language or is it
> > too small a feature for such a big change?
>
> I don't think contracts is a small feature. I think it is a HUGE
> feature, but under-appreciated by most developers. (Probably due to
> unfamiliarity, and the difficulty in using it in a language with no
> syntactic support.)

Agreed. And it sounds like there are a lot of subtleties to contracts
that I certainly hadn't appreciated (I don't know if the same is true
of others). For example,

On Tue, 21 Aug 2018 at 08:08, Marko Ristin-Kaufmann
<marko....@gmail.com> wrote:
>
> I think that these objections miss what actually Daniel Moisset wrote in his message: contracts are more than pre- and post-condition checks on a function. The inheritance of decorators does not imply just inheriting the pre- and post-conditions, but also relaxing and tightening them (represented by "require else" and "ensure then" in Eiffel). If this is to be used effectively in practice with little overhead then we would either need to introduce new syntax to the language or make the compiler improve the byte code on the fly.

I've no idea what "relaxing" and "tightening" of contracts involves,
or how it would translate into Python. So I'd imagine that in order to
put together a proposal for adding contracts to Python, you'd need to
explain what contracts are, and how they work, to get past people's
preconceptions that "they are just assertions". Otherwise, it's likely
to be hard to persuade people of the benefits.

Personally, I don't really think I can comment much further, precisely
because it looks like I don't know enough about what contracts are and
how they'd be used to contribute :-)

Paul

[Marko Ristin-Kaufmann]

Hi,

@Paul Moore: thanks for pointing out that many people are not familiar with design-by-contract. I was not aware of that. 

Let me give you a very short introduction into contracts and what they are good for. I'll review some existing libraries and highlight what features we missed (and why we developed our own library). I will finally conclude with why all these solutions (including our own one) are not a replacement for a proper support of design-by-contract in the language.

Design-by-Contract
Design-by-contract was gradually introduced as a concept by Bertrand Meyer in the 1980ies to provide a formal and verifiable interface specification between the software components. Up to then the interfaces were usually defined formally in terms of abstract data types (think of records / structs and classes). He extended abstract data types by adding "contract" conditions that should hold at different points during the execution of a program. The contracts allow you to formally write down your expectations about the program (as opposed to writing it informally in documentation of a class or a function). Hence we can automatically test that they hold either statically or during the execution of the program. This gives us many-fold benefits: 

• contracts prevent code rot (since they can be checked by a compiler or a static analysis tool such as mypy), 
• allow us to have much more sophisticated automatic generation of unit tests and automatic formal proofs of a program, 
• make the documentation explicit, accurate and verifiable and 
• accelerate the development since errors are caught early.

The contracts are categorized as follows depending on at which point in the program they are verified:

• Preconditions are contracts that should hold before the execution of a function. The caller is responsible to fulfill the preconditions.
• Postconditions are contracts that should hold after the execution of a function. The callee is responsible to fulfill the postconditions.
• Invariants should hold throughout the execution of the program. There are two types of invariants: loop invariants and class invariants.
• Loop invariants should hold before and after each iteration step of a loop.
• Class invariants should hold throughout the life time of a class (i.e. between the calls to the public methods). The class invariants are suspended during the construction of an instance and in private methods to avoid cycles. You can think of the constructor method and public methods being responsible to fulfill the class invariants.

The concept of design-by-contract is not limited only to concrete classes, but can be also applied to class hierarchies. Preconditions, postconditions and invariants are inherited. They can be also modified in the following ways:

• The child class needs to fulfill all the invariants of its antecedent classes and its own ones. 
• The preconditions of a function of a child class can "weaken" or "relax" the preconditions of the parent class. In other words, it needs to fulfill either the preconditions of the parent class or its own set of preconditions. This is reflected in Eiffel by using the keyword require else.
• The postconditions of a  a child class can "strengthen" or "tighten" the postconditions of the parent class. The function needs to fulfill all the postconditions of the parent class' function and its own set of postconditions. In Eiffel, this is designated with ensure then keyword.

Invariants operate only on the values of instance properties. Preconditions operate on both function arguments and instance properties. Postconditions need yet one more instrument: they operate on function arguments, instance properties and the result of a function, but can access all these values both at their old state, before the function call, and their new state, after the function call. In Eiffel, you use the keyword old to indicate the value before the function call.

Let me illustrate the concepts by adapting the examples from https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions and https://www.eiffel.org/doc/eiffel/ET-_Inheritance. I will paraphrase the example code in Python assuming that invariant, require, ensure and old were introduced as keywords.

class Account:
    def __init__(self, balance: int) -> None:
        self.balance = balance
        self.deposits = []  # type: List[int]

    def update(self, sum: int)->None:
        require: sum >= 0

        self.balance += sum
        self.deposits.append(sum)

        ensure: len(self.deposits) = len(old self.deposits) + 1, "one more deposit"
        ensure: self.balance = old self.balance + sum, "updated"

    invariant: not self.deposits or self.balance == sum(self.deposits), "consistent balance"
    invariant: self.deposits or self.balance == 0, "zero if no deposits"

class CheckingAccount(Account):
    def update(self, sum: int) -> None:
        require else: self.balance >= sum
        
        self.balance += sum
        self.deposits.append(sum)
        
    invariant: self.balance >= 0

For more examples and details, please see the before-mentioned two web pages on Eiffel.

Current Implementations
A series of open-sourced libraries have been developed to bring design-by-contract to Python.

PyContracts (https://pypi.org/project/PyContracts/) introduced the preconditions and postconditions as special annotations:

@contract
def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]':
     # Requires b to be a nonempty list, and the return
     # value to have the same length.
     ...

new_contract('valid_name', lambda s: isinstance(s, str) and len(s)>0)
@contract(names='dict(int: (valid_name, int))')
def process_accounting(records):
    ...

I personally find these annotations hard to read. The contracts seem mostly focused on type checks and on single arguments. Moreover, without support in IDEs and static analysis tools, annotations are susceptible to code rot if not verified by a linter (as far as I could find out, there are not linters that check pycontracts -- please correct me if I'm wrong). I also don't see how you could elegantly implement a check based on multiple arguments (e.g., sum >= balance in the CheckingAccount example).

Dpcontracts (https://pypi.org/project/dpcontracts/) is a library based on decorators. It encapsulates the arguments of a function and its result as the arguments of the condition function:

>>> @require("`i` must be a positive integer",
...          lambda args: isinstance(args.i, int) and args.i > 0)
... @require("`j` must be a positive integer",
...          lambda args: isinstance(args.j, int) and args.j > 0)
... @ensure("the result must be greater than either `i` or `j`",
...         lambda args, result: result > args.i and result > args.j)
... def add2(i, j):
...     if i == 7:
...        i = -7 # intentionally broken for purposes of example
...     return i + j

It also supports invariants:

>>> @invariant("inner list can never be empty", lambda self: len(self.lst) > 0)
... @invariant("inner list must consist only of integers",
...            lambda self: all(isinstance(x, int) for x in self.lst))
... class NonemptyList:
...     @require("initial list must be a list", lambda args: isinstance(args.initial, list))
...     @require("initial list cannot be empty", lambda args: len(args.initial) > 0)
...     @ensure("the list instance variable is equal to the given argument",
...             lambda args, result: args.self.lst == args.initial)
...     @ensure("the list instance variable is not an alias to the given argument",
...             lambda args, result: args.self.lst is not args.initial)
...     def __init__(self, initial):
...         self.lst = initial[:]
...
...     def get(self, i):
...         return self.lst[i]
...
...     def pop(self):
...         self.lst.pop()
...
...     def as_string(self):
...         # Build up a string representation using the `get` method,
...         # to illustrate methods calling methods with invariants.
...         return ",".join(str(self.get(i)) for i in range(0, len(self.lst)))

Contracts (https://pypi.org/project/contracts/), pyadbc (https://pypi.org/project/pyadbc/) and pcd (https://pypi.org/project/pcd/) are similar in terms of features to dpcontracts, but seem not to be maintained any more.

We found that all the presented methods were a bit impractical to use in everyday programming. Pycontracts forces the programmer to learn a new syntax which is not statically checked. While we liked the features provided by dpcontracts, we found that exception messages thrown at contract breach were uninformative. The programmer is forced to repeat every condition in text if s/he is to make any use of the error once it happens. This can lead to mismatch between the messages and code (similar how the comments tend to rot) and can make debugging and tracing bugs very hard. Moreover, it is tedious and repetitive to document the conditions twice which makes programmers sometimes reluctant to adopt it and apply it widely.

We therefore developed icontract (https://pypi.org/project/icontract/). We decided to leave out class invariants for practical reasons (we rarely use classes in our applications and we needed conditions in production for which the invariants are often impractical), but invariants can be easily added if there is a demand. icontract is based on decorators and uses lambda functions for the conditions that match the argument names of the function. The argument result is reserved for the result of the function in postconditions. Here is the example usage:

>>> @icontract.pre(lambda x: x > 3)
... def some_func(x: int, y: int = 5)->None:
...     pass
...

>>> some_func(x=5)

# Pre-condition violation
>>> some_func(x=1)
Traceback (most recent call last):
  ...
icontract.ViolationError: Precondition violated: x > 3: x was 1

>>> @icontract.post(lambda result, x: result > x)
... def some_func(x: int, y: int = 5) -> int:
...     return x - y
...
>>> some_func(x=10)
Traceback (most recent call last):
  ...
icontract.ViolationError: Post-condition violated: result > x:
result was 5
x was 10

Mind that there is no mandatory description in the contract yet the message is informative. We achieve that by re-executing the condition function and tracing the values by examining its abstract syntax tree. The re-computation can also deal with more complex expressions and outer scope:

>>> class B:
...     def __init__(self) -> None:
...         self.x = 7
...
...     def y(self) -> int:
...         return 2
...
...     def __repr__(self) -> str:
...         return "instance of B"
...
>>> class A:
...     def __init__(self)->None:
...         self.b = B()
...
...     def __repr__(self) -> str:
...         return "instance of A"
...
>>> SOME_GLOBAL_VAR = 13
>>> @icontract.pre(lambda a: a.b.x + a.b.y() > SOME_GLOBAL_VAR)
... def some_func(a: A) -> None:
...     pass
...
>>> an_a = A()
>>> some_func(an_a)
Traceback (most recent call last):
  ...
icontract.ViolationError: Precondition violated: (a.b.x + a.b.y()) > SOME_GLOBAL_VAR:
SOME_GLOBAL_VAR was 13
a was instance of A
a.b was instance of B
a.b.x was 7
a.b.y() was 2

We found informative messages to be a tremendous booster when debugging since you often immediately see what caused the contract breach and not only that it was broken. This often points you directly to the cause of the bug.

Moreover, conditions are much more readable when not cluttered by redundant descriptions. This is important when you use them as part of the documentation that you inspect with an IDE  (e.g., in PyCharm). By our subjective impression, other solutions resulted in hard-to-read documentation in PyCharm. While descriptions can also be added to the icontract conditions, we rarely find that necessary since most conditions are self-explanatory.

Insufficiencies of the Current Libraries

All the libraries described here were not trivial to implement and come with a substantial computational overhead. In one of our benchmarks, we found that having a precondition made a function run at least 6x slower (we traced the slow-down to an additional function invocation which is costly in Python). I don't think that it is possible to implement old keyword for most practical applications since the execution would be even slower.

I found no library so far that supports inheritance, strengthening (ensure then) and weakening (require else) of the contracts out-of-the-box. My intuition tells me that it is possible to implement such a feature in a library, but the implementation will definitely be very complex.

Apart from computational efficiency and complexity of implementation, I also see the variety of libraries as a problem for the adoption of design-by-contract. With multiple solutions and each team having their own preferences there is a dilemma which solution to choose. Without a wide adoption of a single solution we can not expect an emergence of tools such as automatic test generators built on top of the contracts which is where the actual tremendous benefits really await.

A standard solution would allow us to have uniform, widely-adopted and efficient design-by-contracts in Python which no library by itself can achieve.

Sketch of a Solution

Instead of introducing new keywords, Python could introduce a built-in module based on decorators. The interpreters could be extended such as that they in-line the code directly into the function whenever the decorators of this built-in module is encountered. This would substantially reduce the computational overhead while it would allow us to avoid changes to language syntax.

However, before we even start looking at a solution, I see it necessary that we first discuss more to which degree contracts should be introduced to Python and what the use cases would look like.

Cheers,

Marko

[Jacco van Dorp]

Total noob speaking here, but....

Those contracts are mostly important during development right ? Slowdown isn't that much of an issue during development. So you could make a debug mode that enforces the contracts, and a production mode that code users can use during production to stop the slowdown - in this case, your decorators can return their functions/classes unaltered. If they do end up with problems, you can always ask them to run the same inputs with debug mode on to see where it goes wrong. 

For the rest, i'm totally new to design by contract. I do get the draw of it, but im not sure if I'd ever really use it. I tend to solve my problems with a royal sprinkling of logger.debug(f"{val_i_need_to_check}").

[Ivan Levkivskyi]

TBH, I think one of the main points of design by contract is that contracts are verified statically.
For runtime contract checking I would rather recommend hypothesis library (or similar).

--

Ivan

[Steven D'Aprano]

On Mon, Aug 27, 2018 at 09:04:21AM +0200, Jacco van Dorp wrote:
> Total noob speaking here, but....
>
> Those contracts are mostly important during development right ?

That assumes that production code is free of bugs.

Usual practice in the Eiffel world is, I believe, to disable post-
condition checking and other assertions in production code, but leave
pre-condition checks in place.

Note: for those unfamiliar with Design By Contract, pre- and post-
condition checks and assertions are NOT used to detect predictable error
states, such as bad user input or IO errors. They are only used for
testing program correctness. A failed contract or assertion can only
mean a bug in the program. In a totally bug-free program contracts can
safely be disabled.

> Slowdown isn't that much of an issue during development.

For many cases, it isn't an issue during production either.

> So you could make a debug
> mode that enforces the contracts, and a production mode that code
> users can use during production to stop the slowdown

In Eiffel, contracts can be disabled or enable globally, or on a
class-by-class basis.

> - in this case, your decorators
> can return their functions/classes unaltered. If they do end up with
> problems, you can always ask them to run the same inputs with debug mode on
> to see where it goes wrong.
>
> For the rest, i'm totally new to design by contract. I do get the draw of
> it, but im not sure if I'd ever really use it. I tend to solve my problems
> with a royal sprinkling of logger.debug(f"{val_i_need_to_check}").

The differences between design by contract and sprinkling messages in
the log include:

- the compiler (or interpreter) never forgets to read the logs
looking for failures;

- when a failure occurs, the program halts, so you know its broken,
rather than silently continuing and perhaps doing something
unexpected;

- you don't need to read the logs and try to remember why it is
that you're printing the value of ``foo`` and whether its a
good thing or a bad thing that it has the value you're seeing.


Here's an introduction to design by contract:

https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions



--
Steve

[Steven D'Aprano]

On Mon, Aug 27, 2018 at 09:24:20AM +0100, Ivan Levkivskyi wrote:
> TBH, I think one of the main points of design by contract is that contracts
> are verified statically.

No, that's not correct. Contracts may be verified statically if the
compiler is able to do so, but they are considered runtime checks.
Static checks are an optimization.

For example, the Eiffel docs describe one possible contract as "the
graph contains no cycle" and can contain function calls.

https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions



--
Steven

[Ivan Levkivskyi]

On Mon, 27 Aug 2018 at 11:39, Steven D'Aprano <st...@pearwood.info> wrote:

On Mon, Aug 27, 2018 at 09:24:20AM +0100, Ivan Levkivskyi wrote:
> TBH, I think one of the main points of design by contract is that contracts
> are verified statically.

No, that's not correct. Contracts may be verified statically if the
compiler is able to do so, but they are considered runtime checks.

Considered by whom? By people who prefer `assert isinstance(x, int)` over `x: int`? :-)

Contract in 99% of cases is just another word for type (maybe a very complex type like `DAG[T] <: Graph[T]`).

Everything else, like `x >= y` is better expressed as an explicit assert with an assert message.

But again this is rather IMO, than any kind of definition.

There is only one use case I see now where a dedicated syntax would give a large readability gain:

something like `self.x >= self.y`. But on the other hand I think such situations are too rare to justify any _new_ syntax.

--

Ivan

[Jonathan Fine]

Ivan and Steve wrote

>> TBH, I think one of the main points of design by contract is that contracts
>> are verified statically.

> No, that's not correct.

> https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions

The page from Steve supplied (URL above) states

> During development and testing, assertion monitoring should
> be turned on at the highest possible level. Combined with
> static typing and the immediate feedback of compilation techniques
> [...] this permits the development process [...]
> where errors are exterminated at birth.

Based on the Eiffel docs, I find Ivan's opinion reasonable. He said it
was "one of the main points". The goal is to detect errors
immediately. Run-time assertion monitoring and static typing are two
means towards that end.

Our shared problem and goal is to have similar immediate detection of
errors in Python (when the development process requires that degree of
rigour).

--
Jonathan

[Steven D'Aprano]

On Mon, Aug 27, 2018 at 12:12:22PM +0100, Ivan Levkivskyi wrote:
> On Mon, 27 Aug 2018 at 11:39, Steven D'Aprano <st...@pearwood.info> wrote:
>
> > On Mon, Aug 27, 2018 at 09:24:20AM +0100, Ivan Levkivskyi wrote:
> > > TBH, I think one of the main points of design by contract is that
> > contracts
> > > are verified statically.
> >
> > No, that's not correct. Contracts may be verified statically if the
> > compiler is able to do so, but they are considered runtime checks.
> >
>
> Considered by whom? By people who prefer `assert isinstance(x, int)` over
> `x: int`? :-)

No, considered by Bertrand Meyer, the inventor of Design By Contract and
the programming language Eiffel.

Contracts are not limited to the things which static type-checkers are
capable of testing, but can and do involve checks which are impossible
or impractical to test at compile-time, like "today is Tuesday" or "the
account balance is greater than the amount you are trying to withdraw".

> Contract in 99% of cases is just another word for type (maybe a very
> complex type like `DAG[T] <: Graph[T]`).
> Everything else, like `x >= y` is better expressed as an explicit assert
> with an assert message.

Contracts ARE assertions. They are assertions about the input a method
expects, not merely the type but the value, what result it intends to
return, and the invariants of the class after the method is called. Like
assertions, they are called at runtime, and can be disabled.

Using contracts to enforce type-correctness wouild be silly in Eiffel,
because Eiffel already has a powerful static type system.

Sybtax-wise, if you're interested in what is possible in a Python-like
language, you could do worse than check out Cobra:

http://cobra-language.com/trac/cobra/wiki/Contracts


--
Steve

[Chris Angelico]

On Mon, Aug 27, 2018 at 10:50 PM, Steven D'Aprano <st...@pearwood.info> wrote:
> On Mon, Aug 27, 2018 at 12:12:22PM +0100, Ivan Levkivskyi wrote:
>> Contract in 99% of cases is just another word for type (maybe a very
>> complex type like `DAG[T] <: Graph[T]`).
>> Everything else, like `x >= y` is better expressed as an explicit assert
>> with an assert message.
>
> Contracts ARE assertions. They are assertions about the input a method
> expects, not merely the type but the value, what result it intends to
> return, and the invariants of the class after the method is called. Like
> assertions, they are called at runtime, and can be disabled.

Sometimes "type" doesn't mean the same thing to the language and to
the human. Suppose you're trying to create a Python script that
replicates a C program; you might want to declare that a variable is
not of type "integer" but type "32-bit unsigned integer", with
wrap-around. Or, wording it another way: "integer modulo 2**32". Is
that an assertion of type, or of type and value? As a precondition to
a function, requiring that a parameter be an integer no less than zero
and no greater than 4294967295 is, in a sense, checking its type and
its value; but it's kinda just asserting its type.

AIUI, that's what Ivan meant by "complex type" - something that
determines the domain of valid values as well as the concrete type.

ChrisA

[Steven D'Aprano]

On Mon, Aug 27, 2018 at 11:00:22PM +1000, Chris Angelico wrote:

> Sometimes "type" doesn't mean the same thing to the language and to
> the human. Suppose you're trying to create a Python script that
> replicates a C program; you might want to declare that a variable is
> not of type "integer" but type "32-bit unsigned integer", with
> wrap-around. Or, wording it another way: "integer modulo 2**32". Is
> that an assertion of type, or of type and value? As a precondition to
> a function, requiring that a parameter be an integer no less than zero
> and no greater than 4294967295 is, in a sense, checking its type and
> its value; but it's kinda just asserting its type.

It is making an assertion about the value of an instance of type "int".
Its not a separate type requiring an explicit coercion or cast. Its just
a non-negative int less than 2**32.

If the compiler supports the concept of a 32-bit unsigned integer type,
then of course we can change our implementation to use that type instead
of our regular ints. But we can't expect to pass such a 32-bit unsigned
integer to a function which expects a regular int unless they are
duck-type compatible, or the compiler performs automatic coercions.
(So-called "weak typing").

A better example is the one I gave earlier, of a graph with no cycles.
There is a deep fundamental difference between a *statically checked*
DAG with no cycles (a graph which can never contain a cycle because the
compiler won't let you create one) and a *dynamically checked* DAG that
merely has no cycles *now* (it may have had cycles earlier, and it might
have cycles later, but right now it has none).

These are very different semantics, and Eiffel's contracts support the
second kind: runtime value checks.



--
Steve

[Wes Turner]

Runtime checks: data validation & code validation

Compile-time checks: code validation

What sort of data validation is appropriate for assert statements or contacts that may be skipped due to trading performance for more risk ('optimized out')?

Checking the value of a Number?

Checking that a large graph has no cycles?

Checking that a database table exists and has the appropriate relations and constraints?

assert statements are skipped at runtime with -O and -OO whether or not they're in [reorderable] aspects applied with decorators, at the beginning or end of a function, or in methods named something like setUp and tearDown.

https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE

https://docs.python.org/3/using/cmdline.html#cmdoption-o

> -O

>   Remove assert statements and any code conditional on the value of __debug__

> PYTHONOPTIMIZE

>   If this is set to a non-empty string it is equivalent to specifying the -O option. If set to an integer, it is equivalent to specifying -O multiple times.

[Steven D'Aprano]

On Mon, Aug 27, 2018 at 11:01:21AM -0400, Wes Turner wrote:
> Runtime checks: data validation & code validation
> Compile-time checks: code validation
>
> What sort of data validation is appropriate for assert statements or
> contacts that may be skipped due to trading performance for more risk
> ('optimized out')?

That depends on what you mean by "data validation".

Testing for bad input, or testing for predictable error states such as
I/O errors, missing files, permission errors, server down etc is
not appropriate for assertions (which includes contracts).

The rule I use is that assertions are for:

(1) testing your program state, which is under your control; and

(2) communicating the intention of your program as executable
code rather than comments.

The ultimate aim of contracts and assertions is to eventually disable
them when the program is bug-free.

The Eiffel docs say:

It should be clear from the preceding discussion that contracts
are not a mechanism to test for special conditions, for example
erroneous user input. For that purpose, the usual control
structures [...] are available [...] An assertion is instead a
correctness condition governing the relationship between two
software modules (not a software module and a human, or a
software module and an external device). ... Bluntly:

Rule -- Assertion Violation: A run-time assertion violation is
the manifestation of a bug.

https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions

For detecting *external* error states (anything to do with data that
comes from outside your program, like user input) you can never let your
guard down and never disable the test, because servers can always go
down, users can always give you bad data, files can always be corrupt.
It is unsafe to disable these tests and so these should not be
assertions.

For a library function intended to be called by third-parties, the
function arguments aren't under the control of the library author so
should not be tested with assertions. But for an application where the
author controls those function arguments, they are under the author's
control and may be assertions or contracts.

Design By Contract is partly a methodology and partly a set of syntax.
Its a way of thinking about the design of your program. In practice, you
don't have to buy 100% into DBC to get some benefit for it. A study done
a few years back looked at 21 large projects in Eiffel, JML (Java) and
Code Contracts for C# and found that 33% of the classes used contracts.

http://se.ethz.ch/~meyer/publications/methodology/contract_analysis.pdf

Like unit testing, you don't need 100% coverage to get benefit. 10% is
better than nothing, and 20% is better than 10%.

I wrote more about assertions here:

https://import-that.dreamwidth.org/676.html

[Wes Turner]

Thanks for the explanation.

This may be a bit OT,

but is there a good way to do runtime assertions (in particular for data validation) that's as easy as assert?

- self.assertEqual (unittest.TestCase.assertEqual) is hardly usable in other class instances,

- pytest can be used at runtime but has somewhat nontrivial overhead

- I always end up writing e.g. _assert() and _assertEqual() that throw AssertionErrors and helpful error messages just like unittest.

How do contracts differ from checking interfaces with e.g. zope.interface?

https://zopeinterface.readthedocs.io/en/latest/verify.html

I'll read up a bit more on design by contract. I seem to have conceptualized dbc as a composition approach with typed interfaces and type and value assertions, but that's not it?

If the parameter value assertions in pycontracts aren't really contracts, and the mypy compile-time parameter and return type checks are not really contracts, and zope.interface verifications aren't really contracts; what does that leave in terms of abstract data types, preconditions, postconditions, and invariants?

https://en.wikipedia.org/wiki/Design_by_contract

On Monday, August 27, 2018, Steven D'Aprano <st...@pearwood.info > wrote:

[Marko Ristin-Kaufmann]

Hi,

To clarify the benefits of the contracts, let me give you an example from our code base:

@icontract.pre(lambda x: x >= 0)
@icontract.pre(lambda y: y >= 0)
@icontract.pre(lambda width: width >= 0)
@icontract.pre(lambda height: height >= 0)
@icontract.pre(lambda x, width, img: x + width <= pqry.opencv.width_of(img))
@icontract.pre(lambda y, height, img: y + height <= pqry.opencv.height_of(img))
@icontract.post(lambda self: (self.x, self.y) in self)
@icontract.post(lambda self: (self.x + self.width - 1, self.y + self.height - 1) in self)
@icontract.post(lambda self: (self.x + self.width, self.y + self.height) not in self)
def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None:
    self.img = img[y:y + height, x:x + width].copy()
    self.x = x
    self.y = y
    self.width = width
    self.height = height

def __contains__(self, xy: Tuple[int, int]) -> bool:
    x, y = xy
    return self.x <= x < self.x + self.width and \
           self.y <= y < self.y + self.height

We use mypy and type annotations for typing, and we don't need contracts for that. In this particular case, contracts are very handy to formulate how we deal with pixels. If you carefully look at the contracts, you will see:

* pixels are indexed starting from 0 (as oppose to starting from 1)

* region-of-interest needs to fit within an image. It can not be outside of its boundaries

* pixels within the region-of-interest are in [x, x + width), [y, y + height) (where "[" means inclusive and ")" exclusive)

Why not types? These constraints are either impossible or very hard to write as types. Moreover, I doubt that you could give such types meaningful names, and I expect the readability to suffer immensely. I suppose any constraints involving more than a couple of variables is hard to write as a type.

Benefits. You could write all of this in human-readable informal docstring of a class, but then it would never be verified. This way we are sure that contracts are always there. Whenever we test, we can be sure that all the contracted properties hold. Mind that we do not test that they hold for that single test case -- we automatically test that they hold for all the test cases.

This is, in my opinion, far superior to ambiguous human documentation or print statements (which are deleted after the debugging). I suppose any larger code base built by a team of more than one developer needs this kind of rigor. The contracts are not meant to be used only in high-risk applications -- they are meant to improve any code base.

Problem without standard support. The current libraries (dpcontracts, icontract) can deal with pre and postconditions and class invariants of a concrete class.

However, the current libraries break as soon as you have inheritance. There is no way in python to inherit the function decorators and to modify them. If we are to inherit from the above-mentioned class ROI, we need to make sure that the invariants of the parent class hold (e.g., what is contained in a ROI) as well as invariants of the child class. We might want to weaken the requirements (e.g., a ROI that can deal with pixels outside of an image) such that x and y can be an arbitrary numbers, not restricted to 0 <= x < img.width and 0 <= y < img.height respectively.

Additionally, without standard approach, we do not know how to deal with contracts when we have a third-party tool that would like to use them (e.g., for automatic generation of unit tests, static testing or visualization in IDE or in documentation).

@Wes Turner: If I understood you correctly, you are looking for a library that gives you verbose messages when a contract is breached. Please have a look at icontract library: https://github.com/Parquery/icontract

We added informative messages particularly because we wanted to have verbose output when something goes wrong.This was helpful not only during the development, but also in production since failure cases were hard to reproduce and anticipate in the first place (otherwise, they wouldn't have made it into the production). Here is an example with an error message:

[Wes Turner]

I now fully understand your use case.

IIUC, what would make this easier is a syntax for defining preconditions, post conditions, and invariants that supports:

* inheritance, composition (mixins)

This isn't possible with decorators because there's no reference to the post-decorated function/method.

A conventional method naming scheme and composition like _pre_check_[004]_height() would be unreasonably verbose; though it would then be possible to 'relax' specific contracts by name.

* A better flow?

Decorators subvert the normal top-down source layout (and so require up-down eye movements and working memory to remember what's going on)

* Test case generation

Generate tests and fixtures to verify the preconditions, postconditions, and invariants.

* An Interpretor/compilation flag to skip the contracts/tests/assertions

PYTHONOPTIMIZE (-O, -OO), __debug__

Isn't there now an -X <...> interpretor flag in CPython?

[David Mertz]

I admit that I don't care that much about pre- and post-conditions. Like a lot of people I've written a you library to handle some cases, but was never motivates to make it better or use ones folks have refined. I definitely oppose dedicated syntax.

Nonetheless, I'll say that it's really not hard to get a decorator to cooperate with inheritance if you want that. If you decorate a parent class you can simply attach the collection of invariants to the class as notations, along with whatever actual guards the decorator enforces. An inheriting child (also decorated) can just look up those notations and apply them to the child.

I.e. the first thing a decorator can do is lookup the invariants attached to the parent (if any) and apply them to the child (if this behavior is enabled). Easy peasy.

[Steven D'Aprano]

On Tue, Aug 28, 2018 at 07:46:02AM +0200, Marko Ristin-Kaufmann wrote:
> Hi,
> To clarify the benefits of the contracts, let me give you an example from
> our code base:
>
> @icontract.pre(lambda x: x >= 0)
> @icontract.pre(lambda y: y >= 0)
> @icontract.pre(lambda width: width >= 0)
> @icontract.pre(lambda height: height >= 0)
> @icontract.pre(lambda x, width, img: x + width <= pqry.opencv.width_of(img))
> @icontract.pre(lambda y, height, img: y + height <= pqry.opencv.height_of(img))
> @icontract.post(lambda self: (self.x, self.y) in self)
> @icontract.post(lambda self: (self.x + self.width - 1, self.y +
> self.height - 1) in self)
> @icontract.post(lambda self: (self.x + self.width, self.y +
> self.height) not in self)
> def __init__(self, img: np.ndarray, x: int, y: int, width: int,
> height: int) -> None:

Thanks for your example Marko. I think that is quite close to the
ugliest piece of Python code I've ever seen, and I don't mean that as a
criticism of you for writing it or the icontract library's design.

We write code that our language allows us to write, not the code we want
to write. Python has not been designed for contracts, and it shows.

I think this demonstrates a truth about programming languages:

Aethestics matter. Syntax matters.

Some features, no matter how useful and beneficial, have no hope at all
of getting widespread use if the syntax is awkward or the result ugly. I
think that contracts will be doomed to be a minor niche used only by
really keen proponents of the style so long as the best way to write a
contract is to use a sequence of decorator calls with lambdas.

I'm going to re-write that in a pseudo-Eiffel like syntax:

def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None:

require:
x >= 0
y >= 0
width >= 0
height >= 0
x + width <= pqry.opencv.width_of(img)
y + height <= pqry.opencv.height_of(img)

# possibly this should be an invariant, not a post-condition?
ensure:
(self.x, self.y) in self
(self.x + self.width - 1, self.y + self.height - 1) in self

(self.x + self.width, self.y + self.height) not in self

# body of __init__ goes here...


This makes a huge difference to readability:

- block structure to give visual shape to the code;

- related code stays together (all the pre-conditions naturally go
into the require block, post-conditions in the ensure block, and
no chance of interleaving them

@pre(...)
@post(...)
@pre(...)

- no extraneous lambdas (they are pure noise);

- no long sequence of decorators (more than one of each is pure noise);

- possibility to write assertions which take more than one statement.

[Wes Turner]

How are conditions relaxed/overridden in Eiffel without a named reference?

That looks much more readable.

Expressions within such blocks are implicitly assertTrue(s).

What sort of test generation from said nameless expressions would be most helpful?

[Greg Ewing]

Wes Turner wrote:
> I'm going to re-write that in a pseudo-Eiffel like syntax:

Maybe some magic could be done to make this work:

def __init__(self, img: np.ndarray, x: int, y: int, width: int,
height: int) -> None:

def __require__():

x >= 0
y >= 0
width >= 0
height >= 0
x + width <= pqry.opencv.width_of(img)
y + height <= pqry.opencv.height_of(img)

def __ensure__():

(self.x, self.y) in self
(self.x + self.width - 1, self.y + self.height - 1) in self
(self.x + self.width, self.y + self.height) not in self

# body of __init__ goes here...

--
Greg

[Stephan Houben]

I

Op wo 29 aug. 2018 07:53 schreef Greg Ewing <greg....@canterbury.ac.nz>:

Wes Turner wrote:
>     I'm going to re-write that in a pseudo-Eiffel like syntax:

Maybe some magic could be done to make this work:

      def __init__(self, img: np.ndarray, x: int, y: int, width: int,
              height: int) -> None:

          def __require__():
              x >= 0
              y >= 0
              width >= 0
              height >= 0
              x + width <= pqry.opencv.width_of(img)
              y + height <= pqry.opencv.height_of(img)

          def __ensure__():
              (self.x, self.y) in self
              (self.x + self.width - 1, self.y + self.height - 1) in self
              (self.x + self.width, self.y + self.height) not in self

          # body of __init__ goes here...

I have often wished we could get at the AST of a function object.

Then we could inspect the AST and extract these magic functions.

Stephan

[Brice Parent]

I've never used contracts, so excuse me if I didn't get how they would
work, and what they should do.

The last example is about pre-post conditions in a class constructor.
But I imagine this is not the only case where one would want to define
contracts, like probably: within methods that return something and to
functions (and in both cases, we'd want to contractually state some of
the return's specificities).

Is the following function something someone used to work with contracts
would write?

def calculate(first: int, second: int) -> float:
    def __require__():
        first > second
        second > 0
        # or first > second > 0 ?

    def __ensure__(ret):  # we automatically pass the return of the
function to this one
        ret > 1

    return first / second

If so, having a reference to the function's output would probably be
needed, as in the example above.

Also, wouldn't someone who use contracts want the type hints he provided
to be ensured without having to add requirements like `type(first) is
int` or something?

- Brice

Le 29/08/2018 à 07:52, Greg Ewing a écrit :
> Wes Turner wrote:
>>     I'm going to re-write that in a pseudo-Eiffel like syntax:
>
> Maybe some magic could be done to make this work:
>
>      def __init__(self, img: np.ndarray, x: int, y: int, width: int,
>              height: int) -> None:
>
>          def __require__():
>              x >= 0
>              y >= 0
>              width >= 0
>              height >= 0
>              x + width <= pqry.opencv.width_of(img)
>              y + height <= pqry.opencv.height_of(img)
>
>          def __ensure__():
>              (self.x, self.y) in self
>              (self.x + self.width - 1, self.y + self.height - 1) in self
>              (self.x + self.width, self.y + self.height) not in self
>
>          # body of __init__ goes here...
>

_______________________________________________

[Jacco van Dorp]

Op wo 29 aug. 2018 om 03:59 schreef Steven D'Aprano <st...@pearwood.info>:

On Tue, Aug 28, 2018 at 07:46:02AM +0200, Marko Ristin-Kaufmann wrote:
> Hi,
> To clarify the benefits of the contracts, let me give you an example from
> our code base:
>
> @icontract.pre(lambda x: x >= 0)
> @icontract.pre(lambda y: y >= 0)
> @icontract.pre(lambda width: width >= 0)
> @icontract.pre(lambda height: height >= 0)
> @icontract.pre(lambda x, width, img: x + width <= pqry.opencv.width_of(img))
> @icontract.pre(lambda y, height, img: y + height <= pqry.opencv.height_of(img))
> @icontract.post(lambda self: (self.x, self.y) in self)
> @icontract.post(lambda self: (self.x + self.width - 1, self.y +
> self.height - 1) in self)
> @icontract.post(lambda self: (self.x + self.width, self.y +
> self.height) not in self)
> def __init__(self, img: np.ndarray, x: int, y: int, width: int,
> height: int) -> None:


Thanks for your example Marko. I think that is quite close to the
ugliest piece of Python code I've ever seen, and I don't mean that as a
criticism of you for writing it or the icontract library's design.

What, really ? Well, it clearly shows you teach python and don't look much at code written by people who taught themselves. I taught myself, and the first .py file I created was over a 1000 lines, and contained the GUI in a 4-deep nested global dictionary, since I'd never seen a style guide at that point. (I got better)

[Steven D'Aprano]

On Wed, Aug 29, 2018 at 05:52:46PM +1200, Greg Ewing wrote:
> Wes Turner wrote:
> > I'm going to re-write that in a pseudo-Eiffel like syntax:
>
> Maybe some magic could be done to make this work:
>
> def __init__(self, img: np.ndarray, x: int, y: int, width: int,
> height: int) -> None:
>
> def __require__():

The problem with this idea is that methods and functions are not
declarations, but executable code. This __require__ function doesn't
exist except while the __init__ method is running. So it can't be called
before the __init__, it can't be called *automatically* (you need to
call it yourself, from inside the __init__), and it can't be inherited.

Of course with sufficient compiler magic of course the compiler could
special case these methods and do whatever we want, but that seems like
it would be just as much work but much uglier than using dedicated
syntax.


--
Steve

[Steven D'Aprano]

On Wed, Aug 29, 2018 at 09:23:02AM +0200, Jacco van Dorp wrote:
> Op wo 29 aug. 2018 om 03:59 schreef Steven D'Aprano <st...@pearwood.info>:
>
> > On Tue, Aug 28, 2018 at 07:46:02AM +0200, Marko Ristin-Kaufmann wrote:
> > > Hi,
> > > To clarify the benefits of the contracts, let me give you an example from
> > > our code base:

[snip long sequence of @decorator(lambda) calls]

> > Thanks for your example Marko. I think that is quite close to the
> > ugliest piece of Python code I've ever seen, and I don't mean that as a
> > criticism of you for writing it or the icontract library's design.
> >
> >
> What, really ?

I said *close* :-)

> Well, it clearly shows you teach python and don't look much
> at code written by people who taught themselves.

I didn't mean to compare it to code written by beginners. I meant
professional quality. And I didn't mean it was *bad* code.

Python is a remarkable elegant and pretty language, but there are some
things that aren't a good fit to the existing syntax. Contracts are one.
We can't easily write code in a declarative style like Prolog:

sibling(X, Y) :- parent_child(Z, X), parent_child(Z, Y)

(X is a sibling of Y when there exists some Z who is a parent of X and
the same Z is the parent of Y); we have to re-write it in a procedural
style. Some programming styles aren't a natural fit to a given syntax.

> I taught myself, and the
> first .py file I created was over a 1000 lines, and contained the GUI in a
> 4-deep nested global dictionary, since I'd never seen a style guide at that
> point. (I got better)

So far, nothing you describe is *necessarily* ugly or bad code.

The std lib contains at least one file well over 1000 lines, and while
it is complex code, its not ugly code by any means. And I couldn't judge
the elegance of the dict unless I saw it and the alternatives :-)

[Steven D'Aprano]

I didn't want to embarass Ivan any further by seemingly picking on his
opinion about contracts being always statically checked, but when I
asked off-list I got told to reword and post it here. So here it is.

Sorry Ivan if this makes you feel I'm picking on you, that isn't my
intention.


On Mon, Aug 27, 2018 at 12:25:44PM +0100, Jonathan Fine wrote:

[..]

> Based on the Eiffel docs, I find Ivan's opinion reasonable. He said it
> was "one of the main points". The goal is to detect errors
> immediately. Run-time assertion monitoring and static typing are two
> means towards that end.

Ivan said that static checking was a main point. Those Eiffel docs which
you (Jonathon) quoted approvingly describe them as "run-time
assertions". You describe them as "run-time assertions". I described
them as "run-time assertions". So I'm having difficulty in understand
what part of Ivan's opinion that they are compile-time static checks is
"reasonable".

If there's something I'm missing about Ivan's comment that you can see,
I'd like to be enlightened. I don't see the relevance of the "two means
towards that end" -- we have many means towards detecting bugs as early
as possible:

- correctness proofs
- test-driven development
- type checking
- design by contract

(and possibly more). If it was just a throw-away comment and I'm reading
more into it than you intended, that's okay too, but I'd like to
understand what you meant.

> Our shared problem and goal is to have similar immediate detection of
> errors in Python (when the development process requires that degree of
> rigour).

Well, yes, but what's that got to do with the question of whether
contracts are checked statically or at runtime?



--
Steve

[Jonathan Fine]

This is about a difference of opinion regarding design by contract and
static checking, that Steve D'Aprano has re-raised. Steve wrote that
Ivan Levkivskyi's opinion was that:

> contracts [are] always statically checked

This is what Ivan wrote:

> TBH, I think one of the main points of design by contract is that contracts
> are verified statically.

There's no 'always' or 'all' here. I read it to mean 'sometimes' or
'some'. And also, that static verification is a good thing.

My message of support for Ivan quoted the Eiffel docs.

> https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions

> During development and testing, assertion monitoring should
> be turned on at the highest possible level. Combined with
> static typing and the immediate feedback of compilation techniques
> [...] this permits the development process [...]
> where errors are exterminated at birth.

I then wrote:

> Based on the Eiffel docs, I find Ivan's opinion reasonable.

Steve wrote:

> Ivan said that static checking was a main point. Those Eiffel docs which
> you (Jonathon) quoted approvingly describe them as "run-time
> assertions".

I'm sorry, but I'm just not seeing that. Either in what I quoted, or
elsewhere in the page. True, my quote is from a section titled
"Run-time assertion monitoring", but the context to me makes it clear
that in Eiffel static typing IS NOT regarded as a run-time assertion.

By the way, Steve wrote

> when I asked off-list I got told to reword and post it here.

I don't think that's quite right. What I said off-list to Steve was

> If you post [...] to python-ideas, I'd be happy to respond there.

It was my intention that it was up to Steve, whether or not to
re-raise the issue. And now I'm doing my part of the bargain, by
responding happily. I'm now happy to let this particular topic rest.

--
Jonathan

[Steven D'Aprano]

On Wed, Aug 29, 2018 at 02:26:54PM +0100, Jonathan Fine wrote:

> This is about a difference of opinion regarding design by contract and
> static checking, that Steve D'Aprano has re-raised. Steve wrote that
> Ivan Levkivskyi's opinion was that:
>
> > contracts [are] always statically checked
>
> This is what Ivan wrote:
>
> > TBH, I think one of the main points of design by contract is that contracts
> > are verified statically.
>
> There's no 'always' or 'all' here. I read it to mean 'sometimes' or
> 'some'. And also, that static verification is a good thing.

Fair enough, I should not have added "always" in my description.

But you probably shouldn't have skipped the part I wrote earlier:

"Contracts may be verified statically if the compiler is able to do so,

but they are considered runtime checks. Static checks are an
optimization."

In context, you quoted me disagreeing with the "static" claim, but
trimmed out my qualification that contracts may sometimes be statically
verified when possible.


[...]

> > Ivan said that static checking was a main point. Those Eiffel docs which
> > you (Jonathon) quoted approvingly describe them as "run-time
> > assertions".
>
> I'm sorry, but I'm just not seeing that. Either in what I quoted, or
> elsewhere in the page. True, my quote is from a section titled
> "Run-time assertion monitoring", but the context to me makes it clear
> that in Eiffel static typing IS NOT regarded as a run-time assertion.

Of course it isn't. By definition, static typing is done *statically*,
at compile-time, not run-time. That has not been questioned and nobody
has asserted that static typing is done at run-time.

We're discussing whether *contracts* are checked at run-time.

Because contracts are (in general) run-time assertions, they are a good
fit for Python's execution model -- or at least they would be if we can
find a good way to apply contracts to methods and classes. If they were
static, they would be a bad fit and would probably need to be handled by
a separate static checker, like MyPy does for static type checks.



--
Steve

[Greg Ewing]

Jonathan Fine wrote:
>
> My message of support for Ivan quoted the Eiffel docs.
>
>>https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions
>
>>During development and testing, assertion monitoring should
>>be turned on at the highest possible level. Combined with
>>static typing and the immediate feedback of compilation techniques
>>[...] this permits the development process [...]
>>where errors are exterminated at birth.

I think you're misinterpreting the Eiffel docs here. It's saying
that contracts *together* with static typing help to catch a lot
of errors early in the development process. It's not saying that
contracts are verified statically, or that all the errors thus
caught are caught at compile time.

> the context to me makes it clear
> that in Eiffel static typing IS NOT regarded as a run-time assertion.

That's true, but static typing and contracts are *different things*
in Eiffel. Static types are checked at compile time, contracts
are checked at run time.

--
Greg

[Marko Ristin-Kaufmann]

Hi,

I think we got entangled in a discussion about whether design-by-contract is useful or not. IMO, the personal experience ("I never used/needed this feature") is quite an inappropriate rule whether something needs to be introduced into the language or not.

There seems to be evidence that design-by-contract is useful. Let me cite Bertrand Meyer from his article "Why not program right?" that I already mentioned before:

We are back then to the core question. These techniques are simple, demonstrably useful, practical, validated by years of use, explained in professional books (e.g. [6]), introductory programming textbooks (e.g. [7]), EdX MOOCs (e.g. [8]), YouTube videos, online tutorials at eiffel.org, and hundreds of articles cited thousands of times.

6.    Bertrand Meyer, Object-Oriented Software Construction, 2^nd edition, Prentice Hall, 1997.

7.    Bertrand Meyer, Touch of Class: Learning to Program Well Using Objects and Contracts, Springer, 2009, see touch.ethz.ch and Amazon page.

8.    MOOCs (online courses) on EdX : "Computer: Art, Magic, Science", Part 1 and Part 2. (Go to "archived versions" to follow the courses.)

Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. So I'd suggest we clarify this first before we move on.

@David Mertz:

Nonetheless, I'll say that it's really not hard to get a decorator to cooperate with inheritance if you want that. If you decorate a parent class you can simply attach the collection of invariants to the class as notations, along with whatever actual guards the decorator enforces. An inheriting child (also decorated) can just look up those notations and apply them to the child.

I.e. the first thing a decorator can do is lookup the invariants attached to the parent (if any) and apply them to the child (if this behavior is enabled). Easy peasy.

Could you please elaborate a bit? I don't see how the annotations would make the contracts invoked on inheritance. Consider this simple case:

class A:
    @icontract.pre(lambda x: x > 0)
    def some_method(self, x: int)->None:
        pass

class B(A):
    # Precondition should be inherited here.
    def some_method(self, x: int) -> None:
        pass

You would still need to somehow decorate manually the overridden methods even though you would not specify any new contracts, right? Is there a mechanism in Python that I am not aware of that would allow us to accomplish that?

[Eric Fahlgren]

On Wed, Aug 29, 2018 at 3:07 PM Marko Ristin-Kaufmann <marko....@gmail.com> wrote:

Could you please elaborate a bit? I don't see how the annotations would make the contracts invoked on inheritance. Consider this simple case:

class A:
    @icontract.pre(lambda x: x > 0)
    def some_method(self, x: int)->None:
        pass

class B(A):
    # Precondition should be inherited here.
    def some_method(self, x: int) -> None:
        pass

You would still need to somehow decorate manually the overridden methods even though you would not specify any new contracts, right? Is there a mechanism in Python that I am not aware of that would allow us to accomplish that?

A metaclass does this pretty easily.  I have a thing I wrote years ago called MIS (Multi-Inheritance Safety) that is used to ensure you don't do anything stupid in our physics-modeling database.  The database is a collection of ~200 Python classes/mixins all deriving madly from each other to get various behaviors (Has mass? You'll need this.  Has moments?  You'll need this.  Has physical extent, i.e., can be seen?  You'll need these...).

Anyhow, the basemost class has the metaclass, which traverses all the methods in the subclasses and makes sure you don't have methods with the same name in two distinct superclass trees and such things.  It also forces you to be explicit when you overload a method from a base class (among other things, but this one is easy to illustrate).

class MIS(type):

    def __init__(cls, name, bases, namespace):

        mro = cls.mro()[1:-1]  # All the stuff between new class and 'object'

        for method_name, method in namespace.items():

            if isinstance(method, executable_types):

                if not getattr(method, '_its_ok', False):

                    # Make sure it's not in any baser class.

                # Could easily check for decorated pre/post properties and copy them...

def override(func):

    # Mark the function as a valid override...

    func._its_ok = True

    return func

class Base(metaclass=MIS):

    def something(self):

        pass

class Derived(Base):

    @override # Signal that this is ok, otherwise we get an error.

    def something(self): ...

       

[David Mertz]

The technique Eric suggests is probably better than what I had in mind.  But I was thinking you could have an "inherit" decorator for methods (or for a class as a whole).  It's easy enough for a decorator to attach a `.__contracts__` attribute to either the class or the individual methods.  Then the decorator(s) in the child can simply look through the `.__mro__` to find any such parent contracts.  E.g.:

class B(A):

    @inherit_invariants

    def some_method(self, x: int) -> None:

        pass

    

    @precondition(lambda x: x=42, inherit_parent=True)

    def other_method(self, x: int) -> float:

        return 42/5

I'm not writing a library to do this, so you can tweak the API to be different than my example.  But this is already well possible.

On the broader idea:

Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless.

Lots of people have explained this relative to lots of ideas, probably mostly better than I will.  Adding a new feature, even if it is *technically* backwards compatible, has HUGE costs.  All the documentation—the books, articles, blog posts, videos, webinars, tutorials, etc.—about Python has to be updated.  We get a divide between "code that will work in Python 3.9" versus what will run in 3.7.  The cognitive burden of learning Python is increased for everyone in the world (millions of people) because even if they do not use a feature they will encounter code that does.  There is another section of code in the implementation(s) of Python that can potentially have bugs and needs to be maintained by someone.

Obviously, design-by-contract is not *meaningless*! It's a specific feature that is relatively well defined as a concept (and precisely defined in regard to Eiffel specifically; but we might not implement those *exact* semantics).  It's also a feature that no languages in particularly widespread use have decided to implement at the language level.  I've chatted with Meyer; he's definitely very smart and definitely strongly opinionated, but I also think he's wrong about the overall importance of this feature versus lots of others.

In my mind, this feature doesn't come close to meeting the burden of those high costs listed above (and others I did not mention).  But I don't have any say in what the core developers will do, beyond in that they might be influenced by my opinion here.

Yours, David...

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.

[Ivan Levkivskyi]

replying to the list now...

On Thu, 30 Aug 2018 at 00:11, Ivan Levkivskyi <levki...@gmail.com> wrote:

On Wed, 29 Aug 2018 at 13:18, Steven D'Aprano <st...@pearwood.info> wrote:

I didn't want to embarass Ivan any further by seemingly picking on his
opinion about contracts being always statically checked, but when I
asked off-list I got told to reword and post it here. So here it is.

Sorry Ivan if this makes you feel I'm picking on you, that isn't my
intention.

NP, the discussion just shift more towards terminology etc. which is less interesting TBH.

--

Ivan

[Eric V. Smith]

I think a metaclass is a non-starter. If one were used, it would
preclude using contracts in any case where a metaclass were already
used, or where one was needed in the future. I'm sure people will
disagree with me on this.

But, I think a more productive line of thinking is: what could be added
to the language that would let contracts be implementable, and could
also be used for other things, too? Sort of like how PEP 487 adds
customizability that has wide applicability.

Eric

[Hugh Fisher]

> Date: Thu, 30 Aug 2018 00:07:04 +0200
> From: Marko Ristin-Kaufmann <marko....@gmail.com>
...

> I think we got entangled in a discussion about whether design-by-contract
> is useful or not. IMO, the personal experience ("I never used/needed this
> feature") is quite an inappropriate rule whether something needs to be
> introduced into the language or not.
>
> There seems to be evidence that design-by-contract is useful. Let me cite
> Bertrand Meyer from his article "Why not program right?" that I already
> mentioned before:

I don't think that being useful by itself should be enough. I think new features
should also be "Pythonic" and I don't see design by contract notation as a
good fit.

For example, C has the useful & operator which lets you pass &foo as
a pointer/array argument despite foo being a scalar, so assignment to
bar[0] in the called function actually sets the value of foo. It might be
possible to create some kind of aliasing operator for Python so that two
or more variables were bound to the same location, but would we want
it? No, because Python is not intended for that style of programming.

For another example, GPU shading languages have the special keywords
uniform and varying for distinguishing definitions that won't change across
parallel invocations and definitions that will. Demonstrably very useful in
computer games and supercomputer number crunching, so why doesn't
Python have those keywords? Because it's not designed to be used for
such.

For design by contract, as others have noted Python assert statements
work fine for simple preconditions and postconditions. I don't see any
significant difference in readability between existing

def foo(x, y):
assert(x > 0)
# Do stuff
assert(x == y)

and new style

def foo(x, y):
require:
x > 0
# Do stuff
ensure:
x == y

Yes there's more to design by contract than simple assertions, but it's not
just adding syntax. Meyer often uses the special "old" construct in his post
condition examples, a trivial example being

ensure count = old.count + 1

How do we do that in Python? And another part of design by contract (at
least according to Meyer) is that it's not enough to just raise an exception,
but there must be a guarantee that it is handled and the post conditions
and/or invariants restored. So there's more syntax for "rescue" and "retry"

If you want to do simple pre and post conditions, Python already has assert.

If you want to go full design by contract, there's no law saying that Python
is the only programming language allowed. Instead of trying to graft new
and IMHO alien concepts onto Python, what's wrong with Eiffel?

--

cheers,
Hugh Fisher

[Ethan Furman]

On 08/29/2018 04:53 PM, Hugh Fisher wrote:
> From: Marko Ristin-Kaufmann:

>> There seems to be evidence that design-by-contract is useful. Let me cite
>> Bertrand Meyer from his article "Why not program right?" that I already
>> mentioned before:
>
> I don't think that being useful by itself should be enough. I think new features
> should also be "Pythonic" and I don't see design by contract notation as a
> good fit.

I don't see type annotation notation as a good fit, either, and yet we have it. Seems to me that DbC would be just as
useful as type annotations (and I find D'Aprano's example syntax very readable).

> For design by contract, as others have noted Python assert statements
> work fine for simple preconditions and postconditions.

And print statements work fine for simple debugging. New features are not added for the simple cases, but the complex,
or non-obviously correct, or the very useful cases.

> Yes there's more to design by contract than simple assertions, but it's not
> just adding syntax. Meyer often uses the special "old" construct in his post
> condition examples, a trivial example being
>
> ensure count = old.count + 1

I can see where that could get expensive quickly.

> How do we do that in Python? And another part of design by contract (at
> least according to Meyer) is that it's not enough to just raise an exception,
> but there must be a guarantee that it is handled and the post conditions
> and/or invariants restored.

Well, we don't have to have exactly the same kind of DbC as Eiffel does.

--
~Ethan~

[Marko Ristin-Kaufmann]

Hi,

@David Mertz, Eric Fahlgren re inheritance: thank you very much for your suggestions. I will try to see how inheritance can be implemented with metaclasses and annotations and put it into icontract library.

@David Mertz re costs:

Adding a new feature, even if it is *technically* backwards compatible, has HUGE costs.

[...]

It's a specific feature that is relatively well defined as a concept (and precisely defined in regard to Eiffel specifically; but we might not implement those *exact* semantics).  It's also a feature that no languages in particularly widespread use have decided to implement at the language level.  I've chatted with Meyer; he's definitely very smart and definitely strongly opinionated, but I also think he's wrong about the overall importance of this feature versus lots of others [emphasis Marko].

In my mind, this feature doesn't come close to meeting the burden of those high costs listed above (and others I did not mention).  But I don't have any say in what the core developers will do, beyond in that they might be influenced by my opinion here.

I am also aware of the costs; when I wrote "useful", I actually meant "useful and meaningful to implement given the costs that you mention. Please apologize for my inexact wording (I write these emails in the evening after work).

Related to the text I emphasized, would you mind to explain a bit more in-depth which features you have in mind? I see contracts formally written out and automatically verified as a completely indispensable tool in large projects with multiple people involved. As I already outlined, writing them in documentation leads to code rot. Not writing assumptions about the data structures at all causes, in my opinion, tons of problems and very, very slow development since each programmer working on the legacy code needs to figure these assumptions over and over again. While testing is not substitute for thinking, I possible don't see how you can add/refactor code in big projects and consider all the interactions between the components. Unit testing can get you only as far, since you always test only for a concrete case. Contracts get you much further, not to mention the automatically generated tests which I can't conceive practical without some sort of contracts in the code. I have also to admit that I was schooled with contracts so I might have a distorted view on the matter (Bertrand Meyer taught Introduction to Programming and all the software engineering university classes; I also worked as a teaching assistant with him for the Software Architecture class).

I will try to see how far we can implement contracts as a library. Personally, I would prefer dedicated syntax and I do believe that dedicated syntax merits the costs since these constructs would lead to much better programming in the Python world. 

If we can't get new syntax, having contracts in the standard library would do it for me as a compromise -- it would standardize, even if in ugly way (as Steven D'Aprano pointed out), how we deal with contracts in Python and would allow for integration into IDEs and static analysis tools.

[Paul Moore]

On Wed, 29 Aug 2018 at 23:08, Marko Ristin-Kaufmann
<marko....@gmail.com> wrote:
>
> Hi,
> I think we got entangled in a discussion about whether design-by-contract is useful or not. IMO, the personal experience ("I never used/needed this feature") is quite an inappropriate rule whether something needs to be introduced into the language or not.

It's not a key factor, but it is indicative, in the sense that if
no-one has ever needed the feature *in Python*, then it's possibly not
a good fit for how the language is used in real life. That doesn't
exclude the possibility of a new feature offering a new and not
previously considered technique, but if that's the assertion, then
it's up to the individual proposing the new feature to persuade the
community that there's a real and significant benefit.

> Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. So I'd suggest we clarify this first before we move on.

"It could be useful" is nowhere near a strong enough reason for
changing the language. Even "it is demonstrably useful in other
languages" would have a hard time. What you need to demonstrate is
that it would be useful *in Python code*. A good way of doing that is
by reviewing a significant body of real-life Python code (the standard
library is a common choice, but a big project like Django or
SQLAlchemy would also be reasonable) and demonstrating how the code
could be *improved* by using the proposed new feature. Here, the key
is that the change has to be an improvement - readability is one
(somewhat difficult to assess objectively) criterion, as is ease of
identifying bugs, or efficiency (less need to repeat complex
expressions, for example).

Conversely, any proposed new feature needs to address how it impacts
people who *don't* want to use it. That can be as simple as backward
compatibility issues such as adding new keywords (people using that
keyword as a variable name will have to change their code) but can
also address any general runtime impact in the interpreter
(bookkeeping and management of new data structures for example, or
changes to class implementation details). and how people who don't use
the new feature will be impacted if they encounter the feature in code
written by others, or in code reviews.

Also, there's the question of teachability. A proposed new feature
must address how it will be explained to new and existing Python
users. And given the confusion we're seeing in this thread ("aren't
these just runtime assertions?") that's something that contracts will
definitely have to address. I'm not saying that the proposal needs to
offer a full tutorial on design by contract, but at a minimum, it'll
have to cover why contracts aren't just runtime assertions, and how
the differences can be made clear in the documentation and by
trainers.

I hope that helps explain what you'll need to do if you want to take
this proposal forward, and why you're getting pushback in areas that
maybe seem incidental to you. Personally, I'm interested in the
feature, but I'm not sure I'm interested enough to want it in Python.
My main questions are

1. How exactly do these differ from simple assertions? I don't
understand the hints about "relaxing" and "strengthening" contracts in
subclasses, in particular I've no idea how I'd express that in actual
syntax.
2. Are we talking here about adding checks that would run in
production code? Wouldn't that slow code down? How do I deal with the
conflict between wanting tighter checks but not wanting to pay the
cost at runtime of checking things that should never go wrong
(contracts, as I understand it, are for protecting against code bugs,
that's why there's no facility for trapping them and producing "user
friendly" error messages)? I use assertions very sparingly in
production code for precisely that reason - why would I be more
willing to use contracts?

Paul

[David Mertz]

On Thu, Aug 30, 2018 at 3:44 AM Marko Ristin-Kaufmann <marko....@gmail.com> wrote:

Related to the text I emphasized, would you mind to explain a bit more in-depth which features you have in mind? I see contracts formally written out and automatically verified as a completely indispensable tool in large projects with multiple people involved.

This much is rather self-evidently untrue.  There are hundreds or thousands of large projects involving multiple people, written in Python, that have succeeded without using contracts.  That suggests the feature is not "completely indispensable" since it was dispensed with in the vast majority of large projects.

I think that most of these large projects succeed in large part because they have good *unit tests*.  There are several popular frameworks for writing these (some in standard library), but notably none of them require specific syntax changes to make them work.  Some of them *do* use DSLs of sorts as part of how they operate (or various metaprogramming and introspection magic).  There is a whole lot of overlap between what unit tests do and what design-by-contract does, enough so that I believe the latter adds little to a large project (of course you can come up with some specific example that a unit test cannot verify as well as a pre/postcondition.

In writing before about "features" (and Paul Moore) does a better job than me in writing about *costs*, I wasn't discussing design-by-contract specifically.  Various new feature ideas come up here and elsewhere.  A few are accepted, most are rejected.  They all need to be compared to costs like those I mention before their possible advantages can win out.

Yours, David...

[Marko Ristin-Kaufmann]

Hi,

@David Mertz, @Paul Moore: first of all, thank you very much for your thorough answers!

@Paul Moore: Thanks in particular for suggesting a road map. I still think that we need somewhat broader agreement even on this list that contracts are useful at an abstract level before we dig in and find concrete examples in existing code bases, have a more detailed estimation about the costs and how much conflicts the new keyword would cause, analyze how hard it would be to teach the concepts etc. So far, my impression is that the majority rejects outright the idea that the design-by-contract is useful and actually sees it as a redundant approach to software correctness at best, and an useless concept at worst.

Of course, it would be much more convincing if I already performed a much more thorough analysis as you suggested. Unfortunately, I don't have the eloquence nor the resources to accomplish that at this moment. I'd imagine such an analysis to be a collaborative project of multiple interested people. I'd find it a futile effort before we have at least a vague agreement here that design-by-contract would be useful.

Please let me address here the concrete points you raised in your message.

1. How exactly do these differ from simple assertions? I don't
understand the hints about "relaxing" and "strengthening" contracts in
subclasses, in particular I've no idea how I'd express that in actual
syntax.

The contracts to not apply only to a concrete function, but also extend to inherited methods.

Let me make an illustration on a stripped-down example of three classes A, B and C, where B and C inherit from A.

class A:
    # A.some_func operates on sorted lists of integers and 
    # it returns an integer larger than the length of 
    # the input list.
    @icontract.pre(lambda lst: lst == sorted(lst))
    @icontract.post(lambda result: result > len(lst))
    def some_func(self, lst: List[int]) -> int:
        # ...

class B(A):
    # B.some_func inherits contracts from A, but overrides the implementation ->
    #   B.some_func also operates only on sorted lists of integers
    #   and needs to return an integer larger than the length
    #   of the input list.
    def some_func(self, lst: List[int]) -> int:
        # ...


class C(A):
    # C.some_func also inherits the contracts from A.
    #   It weakens the precondition:
    #       it operates either on sorted lists OR
    #           the lists that are shorter than 10 elements.
    #
    #  It strenghthens the postcondition:
    #       It needs to return an integer larger than 
    #       the length of the input list AND
    #           the result needs to be divisible by 2.
    @icontract.post(lambda result: result % 2 == 0)
    def some_func(self, lst: List[int]) -> int:
        # ...

Assume that the function A.some_func needs sorted integers as input (e.g. because it uses binary search).

Now, class B inherits from A -- say B.some_func also uses binary search.

The things get really interesting in the case of class C and its method some_func: to satisfy polymorphism, C.some_func needs to work in all cases where an instance of A could work as well. However, it can do more -- it can also operate in situations where the input list has few elements, but is not necessarily sorted. This is called "weakening" or "relaxing" of a contract.

The "strengthening" or "tightening" of a contract is an analogous concept for the post-conditions. Due to polymorphism, C.some_func needs to return a result that satisfies all the postconditions of its ancestors so that we can safely use it in all the situations where we would also use an instance of the ancestor class. But we can strengthen it in C.some_func: it needs to return the result satisfying the ancestors' postcondition and than be constrained some more.

I omitted the invariants for brevity. They behave the same as the post-conditions: C needs to satisfy all its own invariants as well as all the invariants of its ancestor classes (A in this case).

2. Are we talking here about adding checks that would run in
production code? Wouldn't that slow code down? How do I deal with the
conflict between wanting tighter checks but not wanting to pay the
cost at runtime of checking things that should never go wrong
(contracts, as I understand it, are for protecting against code bugs,
that's why there's no facility for trapping them and producing "user
friendly" error messages)? I use assertions very sparingly in
production code for precisely that reason - why would I be more
willing to use contracts?

There is no way you can use contracts without slowing down the program (as they still needs to be verified at run-time). People who need the production code to run as fast as possible need to disable the contracts (and assertions) altogether. If contracts were fast to be verified (for example, by in-lining the checks into the function body instead of calling a separate function), the cost of many contracts might be still acceptable in the production code. Mind that there is also a gradation: there might be contracts which are simply always too slow to run in production (e.g., check that the input or output is sorted) while others are almost always acceptable (e.g., check that the input arguments are positive integers or one argument is smaller than the other). There is usually a switch mechanism that allows the developer to turn off certain contracts while still enforcing the others.

Please mind that the benefits of the contracts are threefold and are not only reduced to "better assertions":

* They allow us to formally write down the assumptions about the input and output of the functions and invariants of the data structures.

* They verify these assumptions automatically in all possible cases at run-time (be it during the development or during the production). A smaller set of unit tests hence tests a larger portion of the code then if there were no contracts.

* They can be used by downstream tools to automatically generate unit tests and perform static analysis. Imagine if you could just click on a class in Pycharm and generate unit tests for all of its methods automatically within couple of seconds. While these might not cover every use case, this feature would allow you to cover many cases with practically zero work. I don't see how such a tool could be developed without a standardized approach to contracts in python (be it a library or a language construct).

@David Mertz:

Related to the text I emphasized, would you mind to explain a bit more in-depth which features you have in mind? I see contracts formally written out and automatically verified as a completely indispensable tool in large projects with multiple people involved.

This much is rather self-evidently untrue.  There are hundreds or thousands of large projects involving multiple people, written in Python, that have succeeded without using contracts.  That suggests the feature is not "completely indispensable" since it was dispensed with in the vast majority of large projects.

Sorry, David, my bad; English is not my first language. What I meant to say is that I, as in "personally", could not see how I would tackle a large project with a bigger team with a variety of skill levels in efficient manner without a tool to formally write down the assumptions and have them automatically verified. I do understand that achieving large projects is evidently possible without contracts (and many other tools such as static type checks, assertions, and even unit testing). In my case, I would have a much higher overhead working in a team where the assumptions are written in the documentation and never verified since many bugs and obsolete documentation would just go unnoticed. Doing refactoring or adding new features would be hence much more costly for my team and me.

Maybe this is also due to the fact that you refer to frameworks when you refer to "large projects" with many experienced and highly skilled contributors. The developers are most probably keen to update the documentation and thoroughly test the framework and also test the assumptions in combinatorial manner. In contrast, I work on a (larger) project where the comments actually become obsolete almost at the moment we wrote them and the testing budget is limited due to time constraints and non-critical application domain. At present, we are not developing a framework used by many other developers, but a solution based on computer vision. We can live with incorrect code, but it's good that it is as correct as it gets.

Personally, I view contracts as a  way to improve substantially the correctness of the code with very little overhead. In addition, contracts seem to lead to better and more maintainable code since people actually need to reflect more on the assumptions and write them down formally. This is, of course, a completely subjective impression based on my anecdotal experience.

Let me see if I can make the inheritance work with meta-programming and extra magical members (__preconditions__, __postconditions__, __invariants__). Let's consider the icontract library a test case, and then see how much more work would it be to bring it into the standard library.

@Steve D'Aprano: could you maybe point us to a couple of studies showing that design-by-contract is beneficial in practice?

Cheers,

Marko

[Ethan Furman]

On 08/30/2018 01:49 PM, Marko Ristin-Kaufmann wrote:

> classC(A):

> # C.some_func also inherits the contracts from A.
> # It weakens the precondition:
> # it operates either on sorted lists OR
> # the lists that are shorter than 10 elements.
> #
> # It strenghthens the postcondition:
> # It needs to return an integer larger than
> # the length of the input list AND
> # the result needs to be divisible by 2.
> @icontract.post(lambdaresult: result %2==0)

> defsome_func(self, lst: List[int]) ->int:
> # ...

I think you forgot an @icontract.pre() here.

[Wes Turner]

def example_func(x, y):

    def __assert_before__(example_func):

        #implicit, AST-able assertion expressions

    # ... code

    def __assert_after__(example_func):

        #

    

    def __assert_after__invariants_02(example_func):

        # "

But these need to be composed / mixed in in MRO order and overridable; more like a class with a __call__() and metaclassery for source-order composition that has to do substring matches for e.g. __assert__*.

Blocks of expressions identified with keywords wouldn't be overrideable ('relaxed',); and would need to do fancy AST that replicates existing MRO?

[Marko Ristin-Kaufmann]

Hi Ethan,

You are right, I deleted it without noticing.

It should say: pre(len(lst) < 10).

[Jonathan Fine]

I've just read and article which makes a good case for providing pre-conditions and post-conditions.

http://pgbovine.net/python-unreadable.htm

The main point is: "without proper comments and documentation, even the cleanest Python code is incomprehensible 'in-the-large'." I find the article to be thoughtful and well-written.

By the way the author, Philip J Guo, is also the author of

 

    http://pgbovine.net/publications/non-native-english-speakers-learning-programming_CHI-2018.pdf

    http://pythontutor.com/

I recommend all of the above.

-- 

Jonathan

[Greg Ewing]

Jonathan Fine wrote:
> I've just read and article which makes a good case for providing
> pre-conditions and post-conditions.
>
> http://pgbovine.net/python-unreadable.htm

There's nothing in there that talks about PBC-style executable
preconditions and postconditions, it's all about documenting
the large-scale intent and purpose of code. He doesn't put
forward any argument why executable code should be a better
way to do that than writing comments.

Personally I don't think it is. E.g.

def distim(doshes):
for d in doshes:
assert isinstance(d, Dosh)
# do something here
for d in doshes:
assert is_distimmed(d)

This ticks the precondition and postcondition boxes, but
still doesn't give you any idea what a Dosh is and why
you would want to distim it.

--
Greg

[Ivan Levkivskyi]

On Mon, 3 Sep 2018 at 23:51, Greg Ewing <greg....@canterbury.ac.nz> wrote:

Jonathan Fine wrote:
> I've just read and article which makes a good case for providing
> pre-conditions and post-conditions.
>
> http://pgbovine.net/python-unreadable.htm

There's nothing in there that talks about PBC-style executable
preconditions and postconditions, it's all about documenting
the large-scale intent and purpose of code. He doesn't put
forward any argument why executable code should be a better
way to do that than writing comments.

FWIW this article looks more like a typical motivational intro to static types in Python :-)

(Even his comment about types can be partially answered with e.g. Protocols.)

--

Ivan

[Steven D'Aprano]

On Tue, Sep 04, 2018 at 12:08:31AM +0100, Ivan Levkivskyi wrote:
> On Mon, 3 Sep 2018 at 23:51, Greg Ewing <greg....@canterbury.ac.nz> wrote:
>
> > Jonathan Fine wrote:
> > > I've just read and article which makes a good case for providing
> > > pre-conditions and post-conditions.
> > >
> > > http://pgbovine.net/python-unreadable.htm
> >
> > There's nothing in there that talks about PBC-style executable
> > preconditions and postconditions, it's all about documenting
> > the large-scale intent and purpose of code. He doesn't put
> > forward any argument why executable code should be a better
> > way to do that than writing comments.
> >
>
> FWIW this article looks more like a typical motivational intro to static
> types in Python :-)

Did we read the same article? This is no more about static typing than
it is about contracts. It is about the need for documentation.

The only connection here between either static typing or contracts is
that both can be a form of very limited documentation: type declarations
tell you the types of parameters and variables (but not what range of
values they can take or what they represent) and contracts tell you the
types and values (but not what they represent).

The author explicitly states that statically typed languages have the
same problem communicating the meaning of the program. Neither clean
syntax (like Python) nor static types help the reader comprehend *what*
the program is doing "in the large". He says:


What's contained within allCovData and covMap (which I presume
are both dicts)? What are the types of the keys? What are the
types of the values? More importantly, what is the meaning of
the keys, values, and their mapping? What do these objects
represent in the grand scheme of the entire program, and how
can I best leverage them to do what I want to do? Unfortunately,
nothing short of having the programmer write high-level comments
and/or personally explain the code to me can possibly provide me
with such knowledge.

It's not Python's fault, though; I would've faced the same
comprehension barriers with analogous code written in Java or
C++. Nothing is wrong with Python in this regard, but
unfortunately its clear syntax cannot provide any advantages
for me when trying to understand code 'in-the-large'.


and later goes on to say:


(To be fair, static types aren't a panacea either: If I showed
you the same Java code filled with type definitions, then it
would be easier to understand what this function is doing in
terms of its concrete types, but without comments, you still
won't be able to understand what this function is doing in terms
of its actual underlying purpose, which inevitably involves
programmer-intended 'abstract types'.)



--
Steve

[Steven D'Aprano]

On Tue, Sep 04, 2018 at 10:50:27AM +1200, Greg Ewing wrote:
> Jonathan Fine wrote:
> >I've just read and article which makes a good case for providing
> >pre-conditions and post-conditions.
> >
> >http://pgbovine.net/python-unreadable.htm
>
> There's nothing in there that talks about PBC-style executable
> preconditions and postconditions, it's all about documenting
> the large-scale intent and purpose of code.

Indeed. Apart from a throw-away comment about using pre-conditions and
post-conditions, that post has little or nothing to do with contracts
specifically.

> He doesn't put
> forward any argument why executable code should be a better
> way to do that than writing comments.

That's because the article isn't about executable comments versus dumb
comments. Its about the need for writing documentation and comments of
any sort, so long as it helps people understand the purpose of the code,
what and why it does what it does.

If you read the author's other posts, he discusses the advantages of
assertions over dumb comments in other places, such as here:

http://pgbovine.net/programming-with-asserts.htm

As far as dumb comments go, I'm reminded of this quote:

"At Resolver we've found it useful to short-circuit any
doubt and just refer to comments in code as 'lies'."

--Michael Foord paraphrases Christian Muirhead
on python-dev, 2009-03-22


But you're right that assertions can only give you limited assistence in
understanding the large scale structure of code:

- types tell you only what kind of thing a variable is;

- assertions tell you both the kind of thing and the acceptible
values it can take;

- unlike dumb comments, assertions are checked, so they stay
relevant longer and are less likely to become lies.

These can help the reader understand the what and sometimes the how, but
to understand the why you need to either be able to infer it from the
code, or documentation (including comments).

Given the choice between a comment and an assertion:

# x must be between 11 and 17
assert 11 <= x <= 17

I think it should be obvious why the assertion is better. But neither
explain *why* x must be within that range. Unless it is obvious from
context (and often it is!) there should be a reason given, otherwise the
reader has to just take it on faith.

> Personally I don't think it is. E.g.
>
> def distim(doshes):
> for d in doshes:
> assert isinstance(d, Dosh)
> # do something here
> for d in doshes:
> assert is_distimmed(d)
>
> This ticks the precondition and postcondition boxes, but
> still doesn't give you any idea what a Dosh is and why
> you would want to distim it.

I think the author would agree with you 100%, given that his article is
talking about the need to understand the *why* of code, not just what it
does in small detail, but the large scale reasons for it.



--
Steve

[Michel Desmoulin]

Isn't the purpose of "assert" to be able to do design by contract ?

assert test, "error message is the test fail"

I mean, you just write your test, dev get a feedback on problems, and
prod can remove all assert using -o.

What more do you need ?

Le 15/08/2018 à 23:06, Marko Ristin-Kaufmann a écrit :
> Hi,
>
> I would be very interested to bring design-by-contract into python 3. I
> find design-by-contract particularly interesting and indispensable for
> larger projects and automatic generation of unit tests.
>
> I looked at some of the packages found on pypi and also we rolled our
> own solution (https://github.com/Parquery/icontract/
> <https://github.com/Parquery/icontract/>). I also looked into
> https://www.python.org/dev/peps/pep-0316/
> <https://www.python.org/dev/peps/pep-0316/>.
>
> However, all the current solutions seem quite clunky to me. The
> decorators involve an unnecessary computational overhead and the
> implementation of icontract became quite tricky once we wanted to get
> the default values of the decorated function.
>
> Could somebody update me on the state of the discussion on this matter?
>
> I'm very grateful for any feedback on this!

[Jonathan Fine]

Michel Desmoulin wrote:

> Isn't the purpose of "assert" to be able to do design by contract ?
>
> assert test, "error message is the test fail"
>
> I mean, you just write your test, dev get a feedback on problems, and
> prod can remove all assert using -o.
>
> What more do you need ?

Good question. My opinion is that assert statements are good. I like them.

But wait, more is possible. Here are some ideas.

1. Checking the return value (or exception). This is a post-condition.

2. Checking return value, knowing the input values. This is a more
sophisticated post-condition.

3. Adding checks around an untrusted function - possibly third party,
possibly written in C.

4. Selective turning on and off of checking.

The last two, selective checks around untrusted functions, I find
particularly interesting.

Suppose you have a solid, trusted, well-tested and reliable system.
And you add, or change, a function called wibble(). In this situation,
errors are most likely to be in wibble(), or in the interface to
wibble().

So which checks are most valuable? I suggest the answer is

1. Checks internal to wibble.

2. Pre-conditions and post-conditions for wibble

3. Pre-conditions for any function called by wibble.

Suppose wibble calls wobble. We should certainly have the system check
wobble's preconditions, in this situation. But we don't need wobble to
run checks all the time. Only when the immediate caller is wibble.

I think assertions and design-by-contract point in similar directions.
But design-by-contract takes you further, and is I suspect more
valuable when the system being built is large.

Thank you, Michel, for your good question.

--
Jonathan

[Marko Ristin-Kaufmann]

Hi,

I implemented the inheritance via meta classes and function and class attributes for pre/postconditions and invariants, respectively. Unless I missed something, this is as far as we can go without the proper language support with a library based on decorators:
https://github.com/Parquery/icontract (version 1.5.0)

 Note that it is actually a complete implementation of design-by-contract that supports both weakening of the preconditions and strengthening of the postconditions and invariants.

Could you please have a look and let me know what you think about the current implementation?

Once we are sure that there is nothing obvious missing, I'd like to move forward and discuss whether we could add this library (or rewrite it) into the standard Python libraries and what needs to be all fixed till to make it that far.

Cheers,

Marko

[Marko Ristin-Kaufmann]

Hi,

A brief follow-up (latest version 1.5.3): I removed the dependency on meta package so that now all comprehensions and generator expressions work. I still had to depend on asttokens in order to get the source code of the condition function. Is there maybe an alternative solution which uses only standard libraries?

Any thoughts or feedback on the icontract library in general?

Cheers,

Marko

[Marko Ristin-Kaufmann]

Hi,

Let me make a couple of practical examples from the work-in-progress (https://github.com/Parquery/pypackagery, branch mristin/initial-version) to illustrate again the usefulness of the contracts and why they are, in my opinion, superior to assertions and unit tests.

What follows is a list of function signatures decorated with contracts from pypackagery library preceded by a human-readable description of the contracts.

The invariants tell us what format to expect from the related string properties.

@icontract.inv(lambda self: self.name.strip() == self.name)
@icontract.inv(lambda self: self.line.endswith("\n"))
class Requirement:
    """Represent a requirement in requirements.txt."""

    def __init__(self, name: str, line: str) -> None:
        """
        Initialize.

        :param name: package name
        :param line: line in the requirements.txt file
        """
        ...

The postcondition tells us that the resulting map keys the values on their name property.

@icontract.post(lambda result: all(val.name == key for key, val in result.items()))
def parse_requirements(text: str, filename: str = '<unknown>') -> Mapping[str, Requirement]:
    """
    Parse requirements file and return package name -> package requirement as in requirements.txt

    :param text: content of the ``requirements.txt``
    :param filename: where we got the ``requirements.txt`` from (URL or path)
    :return: name of the requirement (*i.e.* pip package) -> parsed requirement
    """
    ...

The postcondition ensures that the resulting list contains only unique elements. Mind that if you returned a set, the order would have been lost.

@icontract.post(lambda result: len(result) == len(set(result)), enabled=icontract.SLOW)
def missing_requirements(module_to_requirement: Mapping[str, str],
                         requirements: Mapping[str, Requirement]) -> List[str]:
    """
    List requirements from module_to_requirement missing in the ``requirements``.

    :param module_to_requirement: parsed ``module_to_requiremnt.tsv``
    :param requirements: parsed ``requirements.txt``
    :return: list of requirement names
    """
    ...

Here is a bit more complex example.
- The precondition A requires that all the supplied relative paths (rel_paths) are indeed relative (as opposed to absolute).
- The postcondition B ensures that the initial set of paths (given in rel_paths) is included in the results.

- The postcondition C ensures that the requirements in the results are the subset of the given requirements.

- The precondition D requires that there are no missing requirements (i.e. that each requirement in the given module_to_requirement is also defined in the given requirements).

@icontract.pre(lambda rel_paths: all(rel_pth.root == "" for rel_pth in rel_paths))  # A
@icontract.post(
    lambda rel_paths, result: all(pth in result.rel_paths for pth in rel_paths),
    enabled=icontract.SLOW,
    description="Initial relative paths included")  # B
@icontract.post(
    lambda requirements, result: all(req.name in requirements for req in result.requirements),
    enabled=icontract.SLOW)  # C
@icontract.pre(
    lambda requirements, module_to_requirement: missing_requirements(module_to_requirement, requirements) == [],
    enabled=icontract.SLOW)  # D
def collect_dependency_graph(root_dir: pathlib.Path, rel_paths: List[pathlib.Path],
                             requirements: Mapping[str, Requirement],
                             module_to_requirement: Mapping[str, str]) -> Package:

    """
    Collect the dependency graph of the initial set of python files from the code base.

    :param root_dir: root directory of the codebase such as "/home/marko/workspace/pqry/production/src/py"
    :param rel_paths: initial set of python files that we want to package. These paths are relative to root_dir.
    :param requirements: requirements of the whole code base, mapped by package name
    :param module_to_requirement: module to requirement correspondence of the whole code base
    :return: resolved depedendency graph including the given initial relative paths,
    """

I hope these examples convince you (at least a little bit :-)) that contracts are easier and clearer to write than asserts. As noted before in this thread, you can have the same behavior with asserts as long as you don't need to inherit the contracts. But the contract decorators make it very explicit what conditions should hold without having to look into the implementation. Moreover, it is very hard to ensure the postconditions with asserts as soon as you have a complex control flow since you would need to duplicate the assert at every return statement. (You could implement a context manager that ensures the postconditions, but a context manager is not more readable than decorators and you have to duplicate them as documentation in the docstring).

In my view, contracts are also superior to many kinds of tests. As the contracts are always enforced, they also enforce the correctness throughout the program execution whereas the unit tests and doctests only cover a list of selected cases. Furthermore, writing the contracts in these examples as doctests or unit tests would escape the attention of most less experienced programmers which are not  used to read unit tests as documentation. Finally, these unit tests would be much harder to read than the decorators (e.g., the unit test would supply invalid arguments and then check for ValueError which is already a much more convoluted piece of code than the preconditions and postconditions as decorators. Such testing code also lives in a file separate from the original implementation making it much harder to locate and maintain).

Mind that the contracts do not replace the unit tests or the doctests. The contracts make merely tests obsolete that test that the function or class actually observes the contracts. Design-by-contract helps you skip those tests and focus on the more complex ones that test the behavior. Another positive effect of the contracts is that they make your tests deeper: if you specified the contracts throughout the code base, a test of a function that calls other functions in its implementation will also make sure that all the contracts of that other functions hold. This can be difficult to implement  with standard unit test frameworks.

Another aspect of the design-by-contract, which is IMO ignored quite often, is the educational one. Contracts force the programmer to actually sit down and think formally about the inputs and the outputs (hopefully?) before she starts to implement a function. Since many schools use Python to teach programming (especially at high school level), I imagine writing contracts of a function to be a very good exercise in formal thinking for the students.

Please let me know what points do not convince you that Python needs contracts (in whatever form -- be it as a standard library, be it as a language construct, be it as a widely adopted and collectively maintained third-party library). I would be very glad to address these points in my next message(s).

Cheers,

Marko

[David Mertz]

I'm afraid that in reading the examples provided it is difficulties for me not simply to think that EVERY SINGLE ONE of them would be FAR easier to read if it were an `assert` instead.

The API of the library is a bit noisy, but I think the obstacle it's more in the higher level design for me. Adding many layers of expensive runtime checks and many lines of code in order to assure simple predicates that a glance at the code or unit tests would do better seems wasteful.

I just cannot imagine wanting to write or work on the kind of codebase that is down here. If some people or organizations want to come in this manner, sure a library is great. But I definitely don't want it in the syntax, nor even in the standard library.

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Michael Lee]

I just want to point out that you don't need permission from anybody to start a library. I think developing and popularizing a contracts library is a reasonable goal -- but that's something you can start doing at any time without waiting for consensus.

And if it gets popular enough, maybe it'll be added to the standard library in some form. That's what happened with attrs, iirc -- it got fairly popular and demonstrated there was an unfilled niche, and so Python acquired dataclasses..

The contracts make merely tests obsolete that test that the function or class actually observes the contracts.

Is this actually the case? Your contracts are only checked when the function is evaluated, so you'd still need to write that unit test that confirms the function actually observes the contract. I don't think you necessarily get to reduce the number of tests you'd need to write.

Please let me know what points do not convince you that Python needs contracts 

While I agree that contracts are a useful tool, I don't think they're going to be necessarily useful for *all* Python programmers. For example, contracts aren't particularly useful if you're writing fairly straightforward code with relatively simple invariants.

I'm also not convinced that libraries where contracts are checked specifically *at runtime* actually give you that much added power and impact. For example, you still need to write a decent number of unit tests to make sure your contracts are being upheld (unless you plan on checking this by just deploying your code and letting it run, which seems suboptimal). There's also no guarantee that your contracts will necessarily be *accurate*. It's entirely possible that your preconditions/postconditions might hold for every test case you can think of, but end up failing when running in production due to some edge case that you missed. (And if you decide to disable those pre/post conditions to avoid the efficiency hit, you're back to square zero.)

Or I guess to put it another way -- it seems what all of these contract libraries are doing is basically adding syntax to try and make adding asserts in various places more ergonomic, and not much else. I agree those kinds of libraries can be useful, but I don't think they're necessarily useful enough to be part of the standard library or to be a technique Python programmers should automatically use by default.

What might be interesting is somebody wrote a library that does something more then just adding asserts. For example, one idea might be to try hooking up a contracts library to hypothesis (or any other library that does quickcheck-style testing). That might be a good way of partially addressing the problems up above -- you write out your invariants, and a testing library extracts that information and uses it to automatically synthesize interesting test cases.

(And of course, what would be very cool is if the contracts could be verified statically like you can do in languages like dafny -- that way, you genuinely would be able to avoid writing many kinds of tests and could have confidence your contracts are upheld. But I understanding implementing such verifiers are extremely challenging and would probably have too-steep of a learning curve to be usable by most people anyways.)

-- Michael

[Marko Ristin-Kaufmann]

Hi David Maertz and Michael Lee,

Thank you for raising the points. Please let me respond to your comments in separation. Please let me know if I missed or misunderstood anything.

Assertions versus contracts. David wrote:

I'm afraid that in reading the examples provided it is difficulties for me not simply to think that EVERY SINGLE ONE of them would be FAR easier to read if it were an `assert` instead.

I think there are two misunderstandings on the role of the contracts. First, they are part of the function signature, and not of the implementation. In contrast, the assertions are part of the implementation and are completely obscured in the signature. To see the contracts of a function or a class written as assertions, you need to visually inspect the implementation. The contracts are instead engraved in the signature and immediately visible. For example, you can test the distinction by pressing Ctrl + q in Pycharm.

Second, assertions are only suitable for preconditions. Postconditions are practically unmaintainable as assertions as soon as you have multiple early returns in a function. The invariants implemented as assertions are always unmaintainable in practice (except for very, very small classes) -- you need to inspect each function of the class and all their return statements and manually add assertions for each invariant. Removing or changing invariants manually is totally impractical in my view.

Efficiency and Evidency. David wrote:

The API of the library is a bit noisy, but I think the obstacle it's more in the higher level design for me. Adding many layers of expensive runtime checks and many lines of code in order to assure simple predicates that a glance at the code or unit tests would do better seems wasteful.

I'm not very sure what you mean by expensive runtime checks -- every single contract can be disabled at any point. Once a contract is disabled, there is literally no runtime computational cost incurred. The complexity of a contract during testing is also exactly the same as if you wrote it in the unit test. There is a constant overhead due to the extra function call to check the condition, but there's no more time complexity to it. The overhead of an additional function call is negligible in most practical test cases.

When you say "a glance at the code", this implies to me that you referring to your own code and not to legacy code. In my experience, even simple predicates are often not obvious to see in other people's code as one might think (e.g. I had to struggle with even most simple ones like whether the result ends in a newline or not -- often having to actually run the code to check experimentally what happens with different inputs). Postconditions prove very useful in such situations: they let us know that whenever a function returns, the result must satisfy its postconditions. They are formal and obvious to read in the function signature, and hence spare us the need to parse the function's implementation or run it.

Contracts in the unit tests.

The API of the library is a bit noisy, but I think the obstacle it's more in the higher level design for me. Adding many layers of expensive runtime checks and many lines of code in order to assure simple predicates that a glance at the code or unit tests would do better seems wasteful.

(emphasis mine)

Defining contracts in a unit test is, as I already mentioned in my previous message, problematic due to two reasons. First, the contract resides in a place far away from the function definition which might make it hard to find and maintain. Second, defining the contract in the unit test makes it impossible to put the contract in the production or test it in a call from a different function. In contrast, introducing the contract as a decorator works perfectly fine in all the three above-mentioned cases (smoke unit test, production, deeper testing).

Library. Michael wrote:

I just want to point out that you don't need permission from anybody to start a library. I think developing and popularizing a contracts library is a reasonable goal -- but that's something you can start doing at any time without waiting for consensus.

As a matter of fact, I already implemented the library which covers most of the design-by-contract including the inheritance of the contracts. (The only missing parts are retrieval of "old" values in postconditions and loop invariants.) It's published on pypi as "icontract" package (the website is https://github.com/Parquery/icontract/). I'd like to gauge the interest before I/we even try to make a proposal to make it into the standard library.

The discussions in this thread are an immense help for me to crystallize the points that would need to be addressed explicitly in such a proposal. If the proposal never comes about, it would at least flow into the documentation of the library and help me identify and explain better the important points.

Observation of contracts. Michael wrote:

Your contracts are only checked when the function is evaluated, so you'd still need to write that unit test that confirms the function actually observes the contract. I don't think you necessarily get to reduce the number of tests you'd need to write.

Assuming that a contracts library is working correctly, there is no need to test whether a contract is observed or not -- you assume it is. The same applies to any testing library -- otherwise, you would have to test the tester, and so on ad infinitum.

You still need to evaluate the function during testing, of course. But you don't need to document the contracts in your tests nor check that the postconditions are enforced -- you assume that they hold. For example, if you introduce a postcondition that the result of a function ends in a newline, there is no point of making a unit test, passing it some value and then checking that the result value ends in a newline in the test. Normally, it is sufficient to smoke-test the function. For example, you write a smoke unit test that gives a range of inputs to the function by using hypothesis library and let the postconditions be automatically checked. You can view each postcondition as an additional test case in this scenario -- but one that is also embedded in the function signature and also applicable in production.

Not all tests can be written like this, of course. Dealing with a complex function involves writing testing logic which is too complex to fit in postconditions. Contracts are not a panacea, but they absolute us from implementing trivial testing logic while keeping the important bits of the documentation close to the function and allowing for deeper tests.

Accurate contracts. Michael wrote:

There's also no guarantee that your contracts will necessarily be *accurate*. It's entirely possible that your preconditions/postconditions might hold for every test case you can think of, but end up failing when running in production due to some edge case that you missed.

Unfortunately, there is no practical exit from this dilemma -- and it applies all the same for the tests. Who guarantees that the testing logic of the unit tests are correct? Unless you can formally prove that the code does what it should, there is no way around it. Whether you write contracts in the tests or in the decorators, it makes no difference to accuracy.

If you missed to test an edge case, well, you missed it :). The design-by-contract does not make the code bug-free, but makes the bugs much less likely and easier to detect early. In practice, if there is a complex contract, I encapsulate its complex parts in separate functions (often with their own contracts), test these functions in separation and then, once the tests pass and I'm confident about their correctness, put them into contracts.

(And if you decide to disable those pre/post conditions to avoid the efficiency hit, you're back to square zero.)

In practice, we at Parquery AG let the critical contracts to run in production to ensure that the program blows up before it exercises undefined behavior in a critical situation. The informative violation errors of the icontract library help us to trace the bugs more easily since the relevant values are part of the error log.

However, if some of the contracts are too inefficient to check in production, alas you have to turn them off and they can't be checked since they are inefficient. This seems like a tautology to me -- could you please clarify a bit what you meant? If a check is critical and inefficient at the same time then your problem is unsolvable (or at least ill-defined); contracts as well as any other approach can not solve it.

Ergonimical assertions. Michael wrote:

Or I guess to put it another way -- it seems what all of these contract libraries are doing is basically adding syntax to try and make adding asserts in various places more ergonomic, and not much else. I agree those kinds of libraries can be useful, but I don't think they're necessarily useful enough to be part of the standard library or to be a technique Python programmers should automatically use by default.

From the point of view of the behavior, that is exactly the case. The contracts (e.g. as function decorators) make postconditions and invariants possible in practice. As I already noted above, postconditions are very hard and invariants almost impossible to maintain manually without the contracts. This is even more so when contracts are inherited in a class hierarchy.

Please do not underestimate another aspect of the contracts, namely the value of contracts as verifiable documentation. Please note that the only alternative that I observe in practice without design-by-contract is to write contracts in docstrings in natural language. Most often, they are just assumed, so the next programmer burns her fingers expecting the contracts to hold when they actually differ from the class or function description, but nobody bothered to update the docstrings (which is a common pitfall in any code base over a longer period of time).

Automatic generation of tests. Michael wrote:

What might be interesting is somebody wrote a library that does something more then just adding asserts. For example, one idea might be to try hooking up a contracts library to hypothesis (or any other library that does quickcheck-style testing). That might be a good way of partially addressing the problems up above -- you write out your invariants, and a testing library extracts that information and uses it to automatically synthesize interesting test cases.

This is the final goal and my main motivation to push for design-by-contract in Python :). There is a whole research community that tries to come up with automatic test generations, and contracts are of great utility there. Mind that generating the tests based on contracts is not trivial: hypothesis just picks elements for each input independently which is a much easier problem. However, preconditions can define how the arguments are related. Assume a function takes two numbers as arguments, x and y. If the precondition is y < x < (y + x)  * 10, it is not trivial even for this simple example to come up with concrete samples of x and y unless you simply brute-force the problem by densely sampling all the numbers and checking the precondition.

I see a chicken-and-egg problem here. If design-by-contract is not widely adopted, there will also be fewer or no libraries for automatic test generation. Honestly, I have absolutely no idea how you could approach automatic generation of test cases without contracts (in one form or the other). For example, how could you automatically mock a class without knowing its invariants?

Since generating test cases for functions with non-trivial contracts is hard (and involves collaboration of many people), I don't expect anybody to start even thinking about it if the tool can only be applied to almost anywhere due to lack of contracts. Formal proofs and static analysis are even harder beasts to tame -- and I'd say the argument holds true for them even more.

David and Michael, thank you again for your comments! I welcome very much your opinion and any follow-ups as well as from other participants on this mail list.

Cheers,

Marko

[Marko Ristin-Kaufmann]

Hi,

Again a brief update.

* icontract supports now static and class methods (thanks to my colleague Adam Radomski) which came very handy when defining a group of functions as an interface via an abstract (stateless) class. The implementors then need to all satisfy the contracts without needing to re-write them. You could implement the same behavior with *_impl or _* ("protected") methods where public methods would add the contracts as asserts, but we find the contracts-as-decorators more elegant (N functions instead of 2*N; see the snippet below).

* We implemented a linter to statically check that the contract arguments are defined correctly. It is available as a separate Pypi package pyicontract-lint (https://github.com/Parquery/pyicontract-lint/). Next step will be to use asteroid to infer that the return type of the condition function is boolean. Does it make sense to include PEX in the release on github?

* We plan to implement a sphinx plugin so that contracts can be readily visible in the documentation. Is there any guideline or standard/preferred approach how you would expect this plugin to be implemented? My colleagues and I don't have any experience with sphinx plugins, so any guidance is very welcome.

class Component(abc.ABC, icontract.DBC):
    """Initialize a single component."""

    @staticmethod
    @abc.abstractmethod
    def user() -> str:
        """
        Get the user name.

        :return: user which executes this component.
        """
        pass

    @staticmethod
    @abc.abstractmethod
    @icontract.post(lambda result: result in groups())
    def primary_group() -> str:
        """
        Get the primary group.

        :return: primary group of this component
        """
        pass

    @staticmethod
    @abc.abstractmethod
    @icontract.post(lambda result: result.issubset(groups()))
    def secondary_groups() -> Set[str]:
        """
        Get the secondary groups.

        :return: list of secondary groups
        """
        pass

    @staticmethod
    @abc.abstractmethod
    @icontract.post(lambda result: all(not pth.is_absolute() for pth in result))
    def bin_paths(config: mapried.config.Config) -> List[pathlib.Path]:
        """
        Get list of binary paths used by this component.

        :param config: of the instance
        :return: list of paths to binaries used by this component
        """
        pass

    @staticmethod
    @abc.abstractmethod
    @icontract.post(lambda result: all(not pth.is_absolute() for pth in result))
    def py_paths(config: mapried.config.Config) -> List[pathlib.Path]:
        """
        Get list of py paths used by this component.

        :param config: of the instance
        :return: list of paths to python executables used by this component
        """
        pass

    @staticmethod
    @abc.abstractmethod
    @icontract.post(lambda result: all(not pth.is_absolute() for pth in result))
    def dirs(config: mapried.config.Config) -> List[pathlib.Path]:
        """
        Get directories used by this component.

        :param config: of the instance
        :return: list of paths to directories used by this component
        """
        pass

[Marko Ristin-Kaufmann]

Hi,

I implemented a sphinx extension to include contracts in the documentation: https://github.com/Parquery/sphinx-icontract

The extension supports inheritance. It lists all the postconditions and invariants including the inherited one. The preconditions are grouped by classes with ":requires:" and ":requires else:".

I was unable to get the syntax highlighting for in-line code to work -- does anybody know how to do that in Sphinx?

The results can be seen, e.g. in this documentation: https://pypackagery.readthedocs.io/en/latest/packagery.html

On a more general note: is there any blocker left why you would not use the contracts in your code? Anything I could improve or fix in icontract that would make it more convincing to use (apart from implementing static contract checking and automatic test generation :))?

Cheers,

Marko

[Anton Agestam]

Hi,

I am going to chirp in on this heated debate with a slightly different angle. I think no one has yet argued that contracts aren't useful, usable or interesting constructs, and neither will I. But I will argue that there is a better solution, and that contracts shouldn't gain language-level support for that reason.

Steven argues that "type declarations tell you the types of parameters and variables (but not what range of values they can take or what they represent)". This in my opinion is fundamentally wrong. What values a parameter can take is exactly what the type tells you. An int represent all whole numbers and a float those plus some values in between. If you find yourself wanting to add restrictions to a parameter of a function, really what your experiencing is that the type you are using isn't properly narrow.

Adding preconditions using contracts to a parameter that adds finer restrictions than what builtin types provide, is equivalent to narrowing the type of that parameter. For instance a function that require its argument to be a positive integer could either add a contract specifying this which will cause an error at runtime, or, it could in addition to the function also provide a specialized type, lets' call this type Nat and use this for its arguments type.

The benefits of implementing the type restriction as a type, as opposed to a contract, is that the signature of the function know reflects its restrictions. This allows IDEs and static type checkers to help the developer spotting these errors before they occur. Whether or not the restriction should be proved at runtime as well is heavily debated here, but note that there are already tools for doing both that exist today: mypy for static type checks, and typeguard for runtime type checks.

So the problem is now, how do we define these refined types? Python doesn't have dependent types after all. A very naive approach is to simply subclass builtins and add the restricting assertions to the __init__ method, and this works. But, we can do better: phantom types. I believe using subclasses of builtin types at runtime will carry some extra weight, but with Python's support for overloading we can make sure that values are represented as builtins at runtime while being represented as narrower types when type checking. This moves the burden of proving that values are properly restricted to the call sites of a function.

The advantages of this approach over contracts is:

- Works out of the box with current type checkers, without any language changes.

- Encourages reuse of types, and therefore arguably better code organisation.

- Parameter restrictions are part of a functions API and are understood by current tooling.

- The restrictions of a function are syntactically grouped by its arguments, instead of being a large pile of checks.

The phantom type approach is equivalent to using contracts with regards to what checks that can be made to a functions arguments and return value at runtime.

For these reasons I'm arguing against giving contracts language-level support, as that would promote a superseded technique.