Deprecate the use of properties in all public API (was: Heavy-computation @property in Matrix class)

[Johan S. R. Nielsen]

In the previous discussion concerning properties on Matrix, there seemed
to be consensus that Matrix.I was a problem. More generally, only
little defence of the general use of properties in the public API was
given.

Therefore, I suggest that we phase out *all* uses of properties in the
public API. The main reasoning is consistency: we are much more
ubiquitously using methods to get information on objects, and it is
inconsistent, confusing and unnecessary to use properties in arbitrary
places, just to save a two keystrokes.

I grep'ed for @property and found no instances of public properties that
could not easily be methods instead (*). Some cases are even
super-inconsistent, such as MPolynomialIdeal.basis (basis() is a method
on all other classes I know of).


I call for the following vote:

[ ] Phase out all uses of properties in the public API and make them
into methods that take no arguments. Add in the developer's manual
somewhere that this is the general policy.

[ ] Phase out properties which perform any non-trivial computation, and
officially condone the use of properties as "getters" of trivial
private information.

[ ] Phase out properties that might (expectedly) throw exceptions, such
as Matrix.I. Condone the use of properties as "getters" of derived
information, such as Matrix.T (transpose).

[ ] Keep things as they are.


For the record, my vote is on phasing out all properties.

There is a technical problem with how to issue proper deprecation
warnings. One possibility is to insert deprecation warnings now for >1
year, and then do the change at a major version. But I think it's more
important to decide how extensively we want to change things first.

Best,
Johan

(*): Simon King hinted earlier at possible performance degradations,
though. I don't know much about this.

[Daniel Krenn]

On 2016-04-29 09:31, Johan S. R. Nielsen wrote:
> [ ] Phase out all uses of properties in the public API and make them
> into methods that take no arguments. Add in the developer's manual
> somewhere that this is the general policy.

Wouldn't you have to get rid of all public attributes as well when doing
this?
Or why should
sage: x.some_attribute
work and
sage: x.some_attribute_by_at_property
be forbidden?

Daniel

[Nicolas M. Thiery]

On Fri, Apr 29, 2016 at 09:31:31AM +0200, Johan S. R. Nielsen wrote:
> I call for the following vote:
>
> [ ] Phase out all uses of properties in the public API and make them
> into methods that take no arguments. Add in the developer's manual
> somewhere that this is the general policy.
>
> [ ] Phase out properties which perform any non-trivial computation, and
> officially condone the use of properties as "getters" of trivial
> private information.
>
> [ ] Phase out properties that might (expectedly) throw exceptions, such
> as Matrix.I. Condone the use of properties as "getters" of derived
> information, such as Matrix.T (transpose).
>
> [ ] Keep things as they are.
>
>
> For the record, my vote is on phasing out all properties.

Consistency is a definitive plus, as well as not impeding
introspection. So I would tend to agree to officially phase out
properties from the public API.

There is one use case for properties in the public API though which I
would like to bring up, namely "glorified methods". Let me explain:

Let P be a parent, and f be a morphism naturally attached to P.
Typical examples are P.product, P.coproduct, ... User wants to:

- use the morphism as a function and apply it on elements
- do operations on the morphism (e.g. compute its rank)

In the current instances, the construction of the morphism may be non
trivial (e.g. define the morphism by linearity), but is cheap (at the
scale of user interaction, e.g. tab completion), and does not raise
exceptions (unless the code is broken in the first place).

In several places, we have used the following convention:

P.f(x) # use the morphism as a function
P.f.rank() # play with the morphism itself

To achieve this, P.f is actually a property (typically a
lazy_attribute), that constructs the morphism.

Alternative syntaxes would be:

P.f()(x) # use the morphism as a function
P.f().rank() # play with the morphism itself

P.f(x) # use the morphism as a function
P.f_morphism().rank() # play with the morphism itself

The merits of the syntax we use are:

- For beginners P.f just feels like as a usual method. When they need
more, they can realize that P.F is actually more than a method

- We don't pollute P's namespace with two gadgets for just one thing.


This is a use case of properties in the public API which I would like
to keep as an exception. Other exceptions of course are the usage of
properties for implementing meta-stuff, like cached_methods, abstract
methods, conditionally defined methods, etc.

I don't have a strong opinion about properties for private
attributes. It seems reasonable indeed to request that they should not
raise exceptions nor do lengthy computations.

Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Johan S. R. Nielsen]

> Wouldn't you have to get rid of all public attributes as well when doing
> this?
> Or why should
> sage: x.some_attribute
> work and
> sage: x.some_attribute_by_at_property
> be forbidden?

My impression is that public attributes generally are frowned upon, in
Sage as in many other projects and languages.

Best,
Johan

[Simon King]

Hi Nicolas,

On 2016-04-29, Nicolas M. Thiery <Nicolas...@u-psud.fr> wrote:
> There is one use case for properties in the public API though which I
> would like to bring up, namely "glorified methods". Let me explain:

So, briefly, these are properties (or lazy attributes) that behave,
syntactically, like methods, right?

Do we consider it as "frustrating experience" when one has to type
M.f()(x) rather than M.f(x), when M.f resp. M.f() is, say, a morphism?

I find it only mildly frustrating.

And how frustrating is it when
M.f?
shows the documentation of the morphism, rather than the documentation
of the underlying construction that outputs the morphism?

Here, I would prefer to see *not* the documentation of the morphism
(which typically is general nonsense on *all* morphisms). Instead, I
want to see the documentation of the construction of that specific
morphism.

What do different approaches give?

sage: class Foo(object):
....: @property
....: def bar(self):
....: "bla"
....: return 1
....:
sage: F = Foo()
sage: F.bar?
gives the desired result (docstring of Foo.bar), whereas
sage: class Foo(object):
....: @lazy_attribute
....: def bar(self):
....: "bla"
....: return 1
....:
sage: F = Foo()
sage: F.bar?
gives general nonsense about integers.

So, from the point of view of documentation, I find
@property fine for API, but @lazy_attribute not. But the problem
is that the computation in @property is repeated when repeatedly
accessing the property. Under that point of view: Why should I
use properties at all, when all what they do is saving me
from typing "()"? In that regard, @lazy_attribute is better.

Best regards,
Simon

[Simon King]

On 2016-04-29, Simon King <simon...@uni-koeln.de> wrote:
> In that regard, @lazy_attribute is better.

Just to be clear: In the other regard (documentation!), it seems
to me that @lazy_attribute is not suitable for API.

Cheers,
Simon

[Nicolas M. Thiery]

On Fri, Apr 29, 2016 at 09:47:14AM +0000, Simon King wrote:
> Just to be clear: In the other regard (documentation!), it seems
> to me that @lazy_attribute is not suitable for API.

In general yes; if a lazy_attribute A.f returns the integer 1, we
cannot configure introspection on this integer to make A.f? useful.

On the other hand, in the use case I mentioned, that would be easy to
fix, e.g. by setting __doc__ appropriately in the constructed
morphism if/when deemed desirable (unlike for integers, we can make
sure that morphisms accept a __doc__ attribute; also the constructed
morphism won't be shared elsewhere where the documentation would not
make sense).

[Jeroen Demeyer]

My vote:

> [X] Phase out properties which perform any non-trivial computation

In certain cases, properties might be useful (but it could very well be
that there are 0 such cases in Sage).

[Jeroen Demeyer]

On 2016-04-29 10:38, Nicolas M. Thiery wrote:
> P.f(x) # use the morphism as a function
> P.f.rank() # play with the morphism itself

cached method also does this.

You can do P.f(x) to call the cached_method f but you also do stuff like
P.f.clear_cache().

I certainly do not want cached methods to become P.f()(x).

[Erik Bray]

I generally feel that properties *should* be used in general for
invariants of some object, regardless of how it's computed in the
first place. I see the point about not using them for "non-trivial"
computations but I also find the lack of a clear definition of
"non-trivial" troubling.

I agree with Jeroen that there may be exceptions. I think a strict
rule against "properties in public APIs" is asking for trouble, and
that this should still be considered on a case-by-case basis.

Perhaps more controversially, I'm fond of creating proxy objects that
can become callable if need be For example an int that usually has an
invariant value, but in might rare cases can be modified by some
additional context that would be provided by "calling" it, in which
case I make a proxy-int that's callable. I think in these cases
Sage's policy has been to make them methods with zero required
positional arguments, which is fine too. It's my preference, but it
also has downfalls, so it's not a sword I care to die on :)

Erik

[Johan S. R. Nielsen]

> Consistency is a definitive plus, as well as not impeding
> introspection. So I would tend to agree to officially phase out
> properties from the public API.
>
> There is one use case for properties in the public API though which I
> would like to bring up, namely "glorified methods".

> ...

Yes, this is a good example of where such syntactic micro-magic can be a
very pleasant experience for the user. I think the cases of properties
that both Jeroen and Erik are arguing to allow are exactly things like
this. It seems that possible doc-issues can be managed, more or less.

So I concede that the rigid stance of officially disallowing all
properties would be "asking for trouble". Perhaps a more realistic
stance is something like:

[ ] Rule of thumb: don't use properties, except if there is a clear,
syntactic benefit that presents a logical exception (to the "method
default") for the user. Properties should particularly be avoided if
exceptions could be thrown or heavy computation performed.

OK, so that's not perfectly well-defined either, but at least things
like MPolynomialIdeal.basis should be method'ized by this rule, as
should the list of properties in the recent AsymptoticRing class (e.g.
summands, growth_group, coefficient_ring, ao.).

Going back to Matrix, I would argue that Matrix.T, Matrix.H, etc. also
fall for this rule. Certainly, Matrix.I does.

Best,
Johan

[Michael Orlitzky]

On 05/02/2016 06:07 AM, Erik Bray wrote:
> On Mon, May 2, 2016 at 10:35 AM, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>> My vote:
>>
>>> [X] Phase out properties which perform any non-trivial computation
>>
>>
>> In certain cases, properties might be useful (but it could very well be that
>> there are 0 such cases in Sage).
>
> I generally feel that properties *should* be used in general for
> invariants of some object, regardless of how it's computed in the
> first place. I see the point about not using them for "non-trivial"
> computations but I also find the lack of a clear definition of
> "non-trivial" troubling.
>

Properties, in any programming language, are syntactic sugar over
getter/setter methods on private member variables. In a language like
C#, they're useful and you have a sensible rule for when to use them:
use properties to get/set private member variables, and methods for
everything else. So basically, properties do no computation at all.

In python, all member variables are public, so the concept of a property
is a bit redundant. It's hard to come up with a "when to use properties"
rule in python, because the only rule that makes sense doesn't apply.
Anywhere you could use a property, you can use a "public" member
variable instead (if you can't, then you wanted a method to begin with).

The one useful feature of @property is that it lets you document your
member variables. If I have a class with a "public" nrows member
variable, then I can't document it so that when a user runs foo.nrows?,
it tells him what that variable is supposed to do. By creating a
property (which is only syntactic sugar over a getter method), I gain
the ability to add a docstring on what would otherwise be a variable.

So, the rule I would put forth is: use @property to document public
member variables, and methods for anything else.

[William Stein]

On Monday, May 2, 2016, Michael Orlitzky <mic...@orlitzky.com> wrote:

On 05/02/2016 06:07 AM, Erik Bray wrote:
> On Mon, May 2, 2016 at 10:35 AM, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
>> My vote:
>>
>>> [X] Phase out properties which perform any non-trivial computation
>>
>>
>> In certain cases, properties might be useful (but it could very well be that
>> there are 0 such cases in Sage).
>
> I generally feel that properties *should* be used in general for
> invariants of some object, regardless of how it's computed in the
> first place.  I see the point about not using them for "non-trivial"
> computations but I also find the lack of a clear definition of
> "non-trivial" troubling.
>

Properties, in any programming language, are syntactic sugar over
getter/setter methods on private member variables. In a language like
C#, they're useful and you have a sensible rule for when to use them:
use properties to get/set private member variables, and methods for
everything else. So basically, properties do no computation at all.

Another hugely relevant factor for us is that sage is used mostly interactively and with extensive use of doc strings and introspections.  Most Python code and libraries (and most C# code) are normally used from other Python code/libraries, not interactively.  Sage is somewhat unusual in this regard.    

 

In python, all member variables are public, so the concept of a property
is a bit redundant. It's hard to come up with a "when to use properties"
rule in python, because the only rule that makes sense doesn't apply.
Anywhere you could use a property, you can use a "public" member
variable instead (if you can't, then you wanted a method to begin with).

The one useful feature of @property is that it lets you document your
member variables. If I have a class with a "public" nrows member
variable, then I can't document it so that when a user runs foo.nrows?,
it tells him what that variable is supposed to do. By creating a
property (which is only syntactic sugar over a getter method), I gain
the ability to add a docstring on what would otherwise be a variable.

So, the rule I would put forth is: use @property to document public
member variables, and methods for anything else.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--
Sent from my massive iPhone 6 plus.

[Erik Bray]

I can say from significant experience that that is *not* the only
useful feature of property.

Let's also not forget the even deeper magic that is the descriptor
protocol in general.

[Michael Orlitzky]

On 05/03/2016 03:51 AM, Erik Bray wrote:
>
> I can say from significant experience that that is *not* the only
> useful feature of property.
>
> Let's also not forget the even deeper magic that is the descriptor
> protocol in general.
>

Well, what's the other useful feature? It lets you call methods without
the parentheses?

[Erik Bray]

You seem to have this view that the sole purpose of getter and setter
methods is to update a private member of some object. Am I
understanding that correctly? Because either I'm misunderstanding
you, or you seem to be confused about why property was created for
Python.

Best,
Erik

[Michael Orlitzky]

On 05/03/2016 09:22 AM, Erik Bray wrote:
>
> You seem to have this view that the sole purpose of getter and setter
> methods is to update a private member of some object. Am I
> understanding that correctly? Because either I'm misunderstanding
> you, or you seem to be confused about why property was created for
> Python.
>

That's not my view on getter/setter methods. Those are useful for doing
all sorts of things: type checking, validation, making variable
access/assignment override-able in subclasses, etc.

I'm questioning whether or not the syntactic sugar AROUND getter/setter
methods is useful. In languages with true private member variables, you
have to write boilerplate getter/setter methods all the time. There, the
trade-off is beneficial. Having some additional language syntax to
eliminate the boilerplate is worth it, even if all of your
getters/setters are trivial (which they should be).

But in python? There's no need to wrap member variables in getter/setter
methods: you can always access them. If you want to do something
nontrivial -- like ensure that you're setting some value to an integer
-- then sure, a setter method is appropriate. But now you have a setter
method that can throw a TypeError. What do we gain by making that a
@property? It changes,

c.set_x(3)

to

c.x = 3

which hides the fact that you're calling a method. Is it worth it? In
the "cons" column, we have

* the new @property syntax
* the mental overhead of having a new concept
* the need for end-user documentation
* the need for a policy regarding when (not) to use @property
* the possibility of bugs like the one that started this thread

In the "pros" column, we have

* no parentheses around the method call
* the ability to add a docstring for c.x (if "x" were otherwise going
to be a public member variable)

[Erik Bray]

On Tue, May 3, 2016 at 4:25 PM, Michael Orlitzky <mic...@orlitzky.com> wrote:
> On 05/03/2016 09:22 AM, Erik Bray wrote:
>>
>> You seem to have this view that the sole purpose of getter and setter
>> methods is to update a private member of some object. Am I
>> understanding that correctly? Because either I'm misunderstanding
>> you, or you seem to be confused about why property was created for
>> Python.
>>
>
> That's not my view on getter/setter methods. Those are useful for doing
> all sorts of things: type checking, validation, making variable
> access/assignment override-able in subclasses, etc.

Okay, right, just making sure....

> I'm questioning whether or not the syntactic sugar AROUND getter/setter
> methods is useful. In languages with true private member variables, you
> have to write boilerplate getter/setter methods all the time. There, the
> trade-off is beneficial. Having some additional language syntax to
> eliminate the boilerplate is worth it, even if all of your
> getters/setters are trivial (which they should be).

I don't think it's just "syntactic sugar". If anything it's setter
and *especially* getter methods that are backwards, but unfortunately
necessary in languages like Java that don't have a natural way to
interpose in attribute access. *Because* of this it's necessary in
Java to explicitly write and use mutator methods for everything, even
though the vast majority that ever get written don't do much beyond
read or update a private member variable. It's the mutator methods
that are the conceptual overhead (for consumers of the API). I call
it premature encapsulation.

> But in python? There's no need to wrap member variables in getter/setter
> methods: you can always access them. If you want to do something
> nontrivial -- like ensure that you're setting some value to an integer
> -- then sure, a setter method is appropriate. But now you have a setter
> method that can throw a TypeError. What do we gain by making that a
> @property? It changes,
>
> c.set_x(3)
>
> to
>
> c.x = 3
>
> which hides the fact that you're calling a method. Is it worth it? In
> the "cons" column, we have
>
> * the new @property syntax

The property class actually predated the @decorator syntax by quite a
bit. The @decorator syntax has countless uses, and in the case of
properties helps make the code more explicit.

> * the mental overhead of having a new concept

As opposed to the mental overhead of unnecessary encapsulation?

> * the need for end-user documentation

Huh?

> * the need for a policy regarding when (not) to use @property
> * the possibility of bugs like the one that started this thread

You're taking the position that anything that *might* (one day, maybe,
or presently) have any side effect other than returning or setting a
member variable must be *syntactically* used as a method. The
Pythonic approach has different semantics. An attribute is some
characteristic of an object "name", "color". A method is something
that object does "run", "compute". This is completely independent of
implementation details. Of course if you want to add validation to a
setter you implement that in a method, but the API is still just
obj.name = 'Foo', because that's how you update an attribute.
Understanding that attribute access *may* have side-effects requires
no more conceptual overhead or documentation than the fact that
methods may have side-effects. Do you wrap every method call with a
try/except? No. Likewise you wouldn't with attribute access unless
the documentation tells you you might have to. (One thing I do miss
from Java is the "raises" syntax.)

As you said, in Python there are no private members so there's no need
to wrap them in setters and getters, and so we don't by default--all
attributes are public, which is often more natural. If an attribute
is safe being public knowledge then it darn well should be. If later
we need to add mutator methods we can do that, but in a quiet way that
doesn't change the existing syntax for accessing that attribute. And
if we add an explicit setter method, in addition to allowing the
undecorated attribute assignment that people were already using, then
you're really getting into trouble.

Anyways we can agree to disagree on this, and even within the Python
community you'll find different opinions, especially regarding things
like how much calculation should be done in the getter of a property,
or what kinds of exceptions should be raised. But by and large you'll
find that the use of @property is considered "Pythonic", and explicit
mutator methods markedly less-so in *most* cases. I feel that Sage is
already badly divorced from the rest of the Python community as it is.

Erik

[William Stein]

On Wed, May 4, 2016 at 6:13 AM, Erik Bray <erik....@gmail.com> wrote:
[...]

> Anyways we can agree to disagree on this, and even within the Python
> community you'll find different opinions, especially regarding things
> like how much calculation should be done in the getter of a property,
> or what kinds of exceptions should be raised. But by and large you'll
> find that the use of @property is considered "Pythonic", and explicit
> mutator methods markedly less-so in *most* cases. I feel that Sage is
> already badly divorced from the rest of the Python community as it is.

The Sage library tends to be used differently than a lot of Python
code. The distinction is that most Sage code is really meant to be
used interactively from a command prompt. This is why doctests work
so well for us, instead of unit tests, which work much better than
doctests for many other projects. The typical usage pattern in Sage
is (1) make foo, (2) foo.[tab], (3) foo.bar exists -- cool! what does
it do? (4) foo.bar?. This is dramatically different than the
typical usage pattern of many Python libraries/programs (e.g., a
django webserver).

-- William

--
William (http://wstein.org)

[Erik Bray]

That's how I try to write most code anyways, since most of the code I
write these days is for consumption of the unintentional developer.
They're "users" in so far as they only want to use my code to build
their own code to get some kind of result. But they're still
developers because the interface I'm offering them is in the form of
objects they can manipulate using a general purpose programming
language. It's good because it really does force you to think about
encapsulation in a practical way. "What attributes and methods do I
want to expose?", "What are just implementation details?"

If anything I think "Pythonic" idioms work well for this kind of
usage. I don't see any contradiction here.

[William Stein]

Let's say we use properties a lot:

(1) make an integer n
(2) n.[tab],
(3) n.bar exists -- cool! what does it do?

Then:

(4) n.bar?

results in either:

(a) a nice docstring about the bar method
(b) a potentially long computation of n.bar, and then the docstring
for pretty much anything, e.g., if n.bar? is an integer, you get the
docstring for an integer, not for what bar is.

If (b) every happens to me when using Sage, I will frankly be confused
and pissed.

I'm just going to be a little blunter. This session below is *STUPID*:

sage: n = matrix(QQ,2)
sage: n.det? # good output
Docstring:
Synonym for self.determinant(...).

EXAMPLES:

sage: A = MatrixSpace(Integers(8),3)([1,7,3, 1,1,1, 3,4,5])
sage: A.det()
sage: n.T? # output -- WTF?
Type: Matrix_rational_dense
String form:
[0 0]
[0 0]
File:
/projects/sage/sage-6.10/local/lib/python2.7/site-packages/sage/matrix/matrix_rational_dense.pyx
Call docstring:
Calling a matrix returns the result of calling each component.

EXAMPLES:

sage: f(x,y) = x^2+y
sage: m = matrix([[f,f*f],[f^3,f^4]]); m
[ (x, y) |--> x^2 + y (x, y) |--> (x^2 + y)^2]
[(x, y) |--> (x^2 + y)^3 (x, y) |--> (x^2 + y)^4]
sage: m(1,2)
[ 3 9]
[27 81]
sage: m(y=2,x=1)
[ 3 9]
[27 81]
sage: m(2,1)
[ 5 25]
[125 625]


---

Unless the above messed up dichotomy is fixed in every possible way
people might use Sage interactively, I'm personally 100% against using
properties for objects users interact with at the top level. They
have only snuck in in a couple of places because I wasn't paying
attention.

[Michael Orlitzky]

On 05/04/2016 09:13 AM, Erik Bray wrote:
>
> I call it premature encapsulation.
>

We can agree there. But when you're programming on a team, the hardest
part is making sure everyone else doesn't do anything stupid. And in the
Java/C# world, half of the team is 22 years old and writing their first
program, so I kind of understand the desire to encapsulate EVERYTHING
before it's necessary.

>> * the mental overhead of having a new concept
>
> As opposed to the mental overhead of unnecessary encapsulation?
>

The variable is encapsulated either way. Declaring a @property creates a
getter/setter method, but you can do the same thing without the property
(which spares you from having to know about @property). The assignment
*syntax* doesn't change if you use a property, but you still need to
know that there's a setter method being used in case you want to e.g.
catch a TypeError on c.x = "derp" and report something useful to the user.

>> * the need for end-user documentation
>
> Huh?
>

I have a matrix m... how come,

* m.rank gives me bullshit
* m.rank() works
* m.T() crashes
* m.T works?

[mmarco]

That is a good example of why using @property might fit bad with the usual sage workflow. On the other hand, there are other examples that could show how it actually might fit better:

sage: n = matrix(QQ, 2)

sage: n.T.[tab]

And the user gets the list of the methods he can call on the traspose of n.

For the user that already knows what T does, it would be convenient to tab-complete on the fly, without needing to store n.T() in a new variable and tab-complete on it.

Overall, I think that the problem with documentation is worse than the gain with tab-completion, but it would be so nice if we could do something like:

sage: n.transpose().[tab] 

and get the corresponding methods shown.

One would be tempted to modify ipython in order to get a function called whenever a tab complition is asked like in

sage: foo.bar().[tab] 

but that could have dangerous side effects if calling bar() mutates foo in some way (or injects something to the namespace, or...).

[Erik Bray]

I'm a little confused about this, because it seems to be an issue with
how command-line interpreter is implementing tab-completion and help
lookup.

I don't think "n.bar?" should first access "n.bar" through normal
attribute access, and then call help() on the result. In fact here I
agree with Michael that it's a very good place to use @property--it's
a clear way to define that a ".bar" attribute exists on instances of
whatever class "n" is an instance of, and provide documentation for
what its purpose is when accessed through normal attribute access.
(Where I disagree is the assertion that it's the *only* use of
property when the experience of an entire developer community would
suggest otherwise).

Right, so apparently "?" isn't implemented well (I didn't realize this
was the case before), and offers little advantage over writing
help(n.T) other than in saved keystrokes. It would make more sense if
it followed what Python *actually* does upon attribute access which
contains roughly:

getter = None
descr = type(obj).__dict__.get(attr)
if descr is not None and hasattr(type(descr), '__get__'):
getter = type(descr).__get__
if hasattr(type(descr), '__set__'):
# True for any property instance
return getter(descr, obj, type(obj))

if attr in obj.__dict__:
return obj.__dict__[attr]

if getter is not None:
return getter(descr, obj, type(obj)

raise AttributeError(attr)

So for the "?" magic to work better it ought to recognize attribute
access, at the very least, and convert that to roughly the same thing,
but returning getter.__doc__ if it is found. I don't find the
behavior of the "?" magic is a compelling driver for major class
design decisions, although I agree that if the current behavior is
what you say it is then I can see why it ends up being one anyways.

> Unless the above messed up dichotomy is fixed in every possible way
> people might use Sage interactively, I'm personally 100% against using
> properties for objects users interact with at the top level. They
> have only snuck in in a couple of places because I wasn't paying
> attention.

Again, seems like an argument *for* properties.

[Volker Braun]

On Wednesday, May 4, 2016 at 5:26:03 PM UTC+2, mmarco wrote:

Overall, I think that the problem with documentation is worse than the gain with tab-completion, but it would be so nice if we could do something like:

sage: n.transpose().[tab] 

You can do that already:

sage: n = matrix(QQ, 2)

sage: %config IPCompleter.greedy=True

sage: n.transpose().[TAB]

Display all 230 possibilities? (y or n)

n.transpose().C                                  n.transpose().hessenbergize                      n.transpose().permute_rows_and_columns

n.transpose().H                                  n.transpose().image                              n.transpose().pfaffian

n.transpose().I                                  n.transpose().indefinite_factorization           n.transpose().pivot_rows

[Erik Bray]

On Wed, May 4, 2016 at 5:26 PM, mmarco <mma...@unizar.es> wrote:
> That is a good example of why using @property might fit bad with the usual
> sage workflow. On the other hand, there are other examples that could show
> how it actually might fit better:
>
>
> sage: n = matrix(QQ, 2)
> sage: n.T.[tab]

And tab completion is an excellent example of an area where optional
static type hinting is incredibly useful (as well as properties /
descriptors defined at class-level, or at least a well-implemented
__dir__ on the metaclass, so that it's possible to know what
attributes a user would expect to find on instances of the class
without instantiating it (this of course being a problem particular to
Python-like dynamic attributes)).

> And the user gets the list of the methods he can call on the traspose of n.
>
> For the user that already knows what T does, it would be convenient to
> tab-complete on the fly, without needing to store n.T() in a new variable
> and tab-complete on it.
>
> Overall, I think that the problem with documentation is worse than the gain
> with tab-completion, but it would be so nice if we could do something like:
>
> sage: n.transpose().[tab]

Again, would need some kind of type-hinting for that to work. This
won't work for all cases though.

> and get the corresponding methods shown.
>
> One would be tempted to modify ipython in order to get a function called
> whenever a tab complition is asked like in
>
> sage: foo.bar().[tab]
>
> but that could have dangerous side effects if calling bar() mutates foo in
> some way (or injects something to the namespace, or...).

It shouldn't have to call bar() if it can be at all avoided (which it
can except in cases where it's impossible to know what type will be
returned without doing the calculation, and Sage probably has many
cases like that, I imagine, but other cases where it will be clear).

[mmarco]

I see. However, as I said before, there are good reasons why that is not the default behaviour. 

[Volker Braun]

On Wednesday, May 4, 2016 at 3:13:21 PM UTC+2, Erik Bray wrote:

I don't think it's just "syntactic sugar".  If anything it's setter
and *especially* getter methods that are backwards, but unfortunately
necessary in languages like Java that don't have a natural way to
interpose in attribute access.

In C++ you could easily implement @property by overloading operator=. But I don't know any example of where its done in the wild. So I don't think that its something thats lacking in the language. Its just that a good C++ API is different from a good Python API.  

IMHO the basic difference is that you can and should use Python interactively. Whereas the C++ scoping and name lookup rules are fairly hostile to tab completion, e.g. Koenig lookup. Notwithstanding the existence of ROOT, which I'd consider as further evidence that C++ isn't meant to be used interactively...

[Erik Bray]

(As an aside, I would tend to agree that matrix.T *should* be a method. But I also see M.T without parens as syntactic sugar itself, and a rough approximation of M^T, with which people are familiar. Perhaps exceptions like these are good uses for a lazily-evaluated proxy object, but I haven't thought through all the possible implications of that for something as complex as a matrix (most of the time I've only used these for built-in types likes int and str).

[Marc Mezzarobba]

Johan S. R. Nielsen wrote:
> [X] Phase out properties that might (expectedly) throw exceptions, such
> as Matrix.I. Condone the use of properties as "getters" of derived
> information, such as Matrix.T (transpose).

--
Marc

[Johan S. R. Nielsen]

William Stein writes:
> Unless the above messed up dichotomy is fixed in every possible way
> people might use Sage interactively, I'm personally 100% against using
> properties for objects users interact with at the top level. They
> have only snuck in in a couple of places because I wasn't paying
> attention.

For the record, the Matrix-properties seem to have been snuck in in
#8094, closed 5 years ago. As mentioned earlier, there are a number of
other @properties in public API here and there, several of them in the
recent AsymptoticRing code.

Best,
Johan

[Johan S. R. Nielsen]

Michael Orlitzky writes:
> I have a matrix m... how come,
>
> * m.rank gives me bullshit
> * m.rank() works
> * m.T() crashes
> * m.T works?

This is in my mind the most compelling reason to phase out properties in
Sage. I think Erik makes many good points, but ultimately I think
consistency is more important. The problems with ? and tab-completion
seem manageable, so I leave that aside.

In many ways, programming math is a software designers worst nightmare,
where designing the good API that is logical and works in all cases
quickly becomes daunting. Michael's example above is nice: the rank of a
matrix is an invariant, so it should just be a read-only property, just
like all the my-first-@property-tutorials tells us, right? But by that
logic, tons and tons of zero-argument methods should be read-only
properties in Sage. And if someone later on implements a clever
algorithm2 for computing the rank of certain types of matrices, and
would like to toggle that by an optional argument, @property is suddenly
a problem. And so where would you draw the line?

Properties are preciously little used in Sage. And I see no clear way of
using them in a consistent manner, in the type of objects that is Sage.
So I feel the rule of consistency tells us we should therefore not use
properties at all (perhaps except for the types of exceptions discussed
by Nicolas Thiery and Simon King).

Best,
Johan

[William Stein]

On Wed, May 4, 2016 at 1:07 PM, Johan S. R. Nielsen
<santa...@gmail.com> wrote:
> Michael Orlitzky writes:
>> I have a matrix m... how come,
>>
>> * m.rank gives me bullshit
>> * m.rank() works
>> * m.T() crashes
>> * m.T works?
>
> This is in my mind the most compelling reason to phase out properties in
> Sage. I think Erik makes many good points, but ultimately I think
> consistency is more important. The problems with ? and tab-completion
> seem manageable, so I leave that aside.
>
> In many ways, programming math is a software designers worst nightmare,
> where designing the good API that is logical and works in all cases
> quickly becomes daunting. Michael's example above is nice: the rank of a
> matrix is an invariant, so it should just be a read-only property, just
> like all the my-first-@property-tutorials tells us, right? But by that
> logic, tons and tons of zero-argument methods should be read-only
> properties in Sage. And if someone later on implements a clever
> algorithm2 for computing the rank of certain types of matrices,

Indeed, right now there are several rank algorithms for matrices over QQ:

Signature : m.rank(self, algorithm='modp')
Docstring : Return the rank of this matrix.

INPUT: * "algorithm" -- either "'modp'" (default) or "'flint'" or "'linbox'"

> Properties are preciously little used in Sage. And I see no clear way of
> using them in a consistent manner, in the type of objects that is Sage.
> So I feel the rule of consistency tells us we should therefore not use
> properties at all (perhaps except for the types of exceptions discussed
> by Nicolas Thiery and Simon King).
>
> Best,
> Johan
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

--
William (http://wstein.org)

[Clemens Heuberger]

[X] Phase out properties which perform any non-trivial computation, and
officially condone the use of properties as "getters" of trivial
private information.

Best, Clemens

[Jason Grout]

FYI, a relevant post just appeared on the ipython dev list: https://mail.scipy.org/pipermail/ipython-dev/2016-May/017099.html

<quote>

A few minutes ago we merged a PR[1] adding jedi[2] integration to IPython, this make the IPython completer a bit smarter, 
as it now knows about situation like:

In[1]: ('je'+'di').upper().<tab>

Where it will infer that you are actually calling a method on a string, which before was requiring setting `IPCompleter.greedy` to `true` 
which has the drawback of evaluating your code with its side effects. 

</quote>

Thanks,

Jason

[Fernando Perez]

On Thu, May 5, 2016 at 9:41 AM, Jason Grout <ja...@jasongrout.org> wrote:

FYI, a relevant post just appeared on the ipython dev list: https://mail.scipy.org/pipermail/ipython-dev/2016-May/017099.html

Indeed! I'm delighted by this, and as mentioned here (https://mail.scipy.org/pipermail/ipython-dev/2016-May/017101.html), it should obviate the need for the greedy completer in many cases.

Feedback from the sage community very much welcome, as always!

ps - note that to get the feature, you need to install Jedi, as the dependency is conditional and the feature is only active if `import jedi` succeeds. So pip away at jedi if you'd like to test it.

--

Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

[Erik Bray]

On Wed, May 4, 2016 at 10:07 PM, Johan S. R. Nielsen
<santa...@gmail.com> wrote:
> Michael Orlitzky writes:
>> I have a matrix m... how come,
>>
>> * m.rank gives me bullshit
>> * m.rank() works
>> * m.T() crashes
>> * m.T works?
>
> This is in my mind the most compelling reason to phase out properties in
> Sage. I think Erik makes many good points, but ultimately I think
> consistency is more important. The problems with ? and tab-completion
> seem manageable, so I leave that aside.
>
> In many ways, programming math is a software designers worst nightmare,
> where designing the good API that is logical and works in all cases
> quickly becomes daunting. Michael's example above is nice: the rank of a
> matrix is an invariant, so it should just be a read-only property, just
> like all the my-first-@property-tutorials tells us, right? But by that
> logic, tons and tons of zero-argument methods should be read-only
> properties in Sage. And if someone later on implements a clever
> algorithm2 for computing the rank of certain types of matrices, and
> would like to toggle that by an optional argument, @property is suddenly
> a problem. And so where would you draw the line?

I think I agree with you here--I think ideally something like rank
should be a property, but there are good exceptions where it shouldn't
be, and so for the sake of consistency it shouldn't be in general.
I think matrix.T is a special case that should be viewed as syntactic
magic (and could be handled better with a better implementation of
"?").

I was mostly just objecting to Michael's earlier comment that
@property is not useful *in general*, and that's demonstrably not
true. But I've come to understand and agree with the overall sentiment
here that it should be employed *carefully* in Sage.

[Johan S. R. Nielsen]

I've created #20904 to deprecate the Matrix.I property which is ready for review.

This seemed to be the only case that most people agreed was bad. There remains Matrix.T and Matrix.H as well as a handful of other public API properties scattered randomly across Sage.

Best,
Johan