tab completion and function aliases cluttering the namespace

5 views
Skip to first unread message

Jason Grout

unread,
Feb 9, 2009, 10:44:45 AM2/9/09
to sage-...@googlegroups.com
Hi all,

Currently, anytime there is a command that contains a noun and an
adjective, there seems to be debate about what order they should be
listed. For example, there have been debates over whether to name a
function right_eigenvectors and eigenvectors_right. It seems the big
argument for eigenvectors_right is that it is easy to find with tab
completion (i.e., if you want to do a eigenvectors, you type
eigenvectors<tab> and pick the right function). The big argument for
right_eigenvectors is that is how things are normally said in English
(the adjective precedes the noun). So to satisfy both, we use both
names, which leads to cluttered namespace in which lots of function
names are actually the same function.

The other day, someone suggested finding a function using syntax similar to:

*eigenvectors*?

This finds all functions that contain the word "eigenvectors". For a
matrix, for example, we get:

sage: m=random_matrix(ZZ,5)
sage: m.*eigenvectors*?
m.eigenvectors_left
m.eigenvectors_right
m.left_eigenvectors
m.right_eigenvectors


If tab completion returned the above choices instead of just the
functions that started with "eigenvectors", I believe that would satisfy
the crowd that wants function names to be easy to complete and guess
using tab completion.

What if, at least for the notebook, and maybe for the command line too,
we listed two sets of completions: the first would be the list given
now, and the second would be the list given above? In other words, if
in the notebook, you typed m.eigenvectors<tab>, you would get a box
containing two lists:
_______________________________________
| |
| list of current completions |
| |
|--------------------------------------
| list of extended completions |
| given by the *word*? trick |
|_____________________________________|


What do people think? I think this would finally answer the tension
between the people that want useful tab completion and the people who
want the function names to look "right".

Too bad English doesn't put the adjectives after the noun :).

Jason

Nick Alexander

unread,
Feb 9, 2009, 11:37:49 AM2/9/09
to sage-...@googlegroups.com
> In other words, if
> in the notebook, you typed m.eigenvectors<tab>, you would get a box
> containing two lists:
> _______________________________________
> | |
> | list of current completions |
> | |
> |--------------------------------------
> | list of extended completions |
> | given by the *word*? trick |
> |_____________________________________|

In the notebook? +1. This is such a good idea I might incorporate it
into sage-mode.

Nick

David Joyner

unread,
Feb 9, 2009, 11:47:16 AM2/9/09
to sage-...@googlegroups.com
+1 for the command line.
Currently to find all the plot commands is not as easy as it should be IMHO

boo...@u.washington.edu

unread,
Feb 9, 2009, 11:55:19 AM2/9/09
to sage-...@googlegroups.com
+1!

Timothy Clemans

unread,
Feb 9, 2009, 11:56:19 AM2/9/09
to sage-...@googlegroups.com
+1 to both commandline and notebook

Carl Witty

unread,
Feb 9, 2009, 1:38:27 PM2/9/09
to sage-...@googlegroups.com
On Mon, Feb 9, 2009 at 7:44 AM, Jason Grout <jason...@creativetrax.com> wrote:
> What if, at least for the notebook, and maybe for the command line too,
> we listed two sets of completions: the first would be the list given
> now, and the second would be the list given above?

+1 for extending tab-completion to mention these extra completions
(both for notebook and command-line).

The way you wrote it, the second list would always be a superset of
the first; I think the first list should be removed from the second.
And maybe the second list should be omitted altogether if it's too
big? For instance, if I type foo.e<TAB>, I'm probably not interested
in the list of all method names that include an 'e' somewhere.

BTW, when I saw the topic, I was expecting to read something like
"Let's use fewer aliases to reduce the length of tab-completion
lists." The message "Let's increase the length of tab-completion
lists to use fewer aliases" was quite a surprise :)

Carl

mhampton

unread,
Feb 9, 2009, 1:39:39 PM2/9/09
to sage-devel
I think this is a great idea. It might not solve the problem of
clutter immediately, because if something like eigenvectors_right is
removed it would break a lot of existing code. But perhaps we could
remove such things in 4.0 after adding deprecation warnings.

-Marshall

Jason Grout

unread,
Feb 9, 2009, 2:05:37 PM2/9/09
to sage-...@googlegroups.com
Carl Witty wrote:
> On Mon, Feb 9, 2009 at 7:44 AM, Jason Grout <jason...@creativetrax.com> wrote:
>> What if, at least for the notebook, and maybe for the command line too,
>> we listed two sets of completions: the first would be the list given
>> now, and the second would be the list given above?
>
> +1 for extending tab-completion to mention these extra completions
> (both for notebook and command-line).
>
> The way you wrote it, the second list would always be a superset of
> the first; I think the first list should be removed from the second.

Good point. I agree.

> And maybe the second list should be omitted altogether if it's too
> big? For instance, if I type foo.e<TAB>, I'm probably not interested
> in the list of all method names that include an 'e' somewhere.

Yes, but I'm not sure how to tell if the list is too big. I certainly
have instances when I want to type the name of a command that starts
with e, but I wasn't sure exactly how the command was spelled, so I did
something like foo.e<TAB>.

Can you think of a good criteria for telling if the extended completion
list is not what the user wanted?

Jason

Carl Witty

unread,
Feb 9, 2009, 3:45:00 PM2/9/09
to sage-...@googlegroups.com
On Mon, Feb 9, 2009 at 11:05 AM, Jason Grout
<jason...@creativetrax.com> wrote:

>
> Carl Witty wrote:
>> And maybe the second list should be omitted altogether if it's too
>> big? For instance, if I type foo.e<TAB>, I'm probably not interested
>> in the list of all method names that include an 'e' somewhere.
>
> Yes, but I'm not sure how to tell if the list is too big. I certainly
> have instances when I want to type the name of a command that starts
> with e, but I wasn't sure exactly how the command was spelled, so I did
> something like foo.e<TAB>.
>
> Can you think of a good criteria for telling if the extended completion
> list is not what the user wanted?

This is a bit arbitrary for my tastes, but I think it (or somthing
similar) could work:

If you type foo.X<TAB>, where X is only one or two characters, then
the extended completion list checks for *_X*. Otherwise (if X is
three or more characters) then the extended completion list is *X*.

Carl

Robert Dodier

unread,
Feb 9, 2009, 4:07:16 PM2/9/09
to sage-devel
On Feb 9, 8:44 am, Jason Grout <jason-s...@creativetrax.com> wrote:

> What do people think? I think this would finally answer the tension
> between the people that want useful tab completion and the people who
> want the function names to look "right".

FWIW I 'm pretty sure aliases are a disaster; whether you
can get most people to agree on a big-endian (<root>_<modifier>)
or little-endian (<modifier>_<root>) naming scheme, you
shouldn't try to accomodate the other via aliases.
Maybe it's too late to do away with existing aliases, but at least
new ones can be discouraged ....

Incidentally I'm in favor of a big-endian naming scheme:
names nearby in a sorted list are more likely related.
(The tab-completion effect is a helpful side-effect.)

About narrow and extensive searches, for Maxima we've
adopted ? foo to indicate an exact search while ?? foo
indicates a search for foo as a substring. (Since regex
characters could be arithmetic operators, allowing any
kind of regex syntax seems problematic.) Maybe Sage
likewise could adopt a distinction between different kinds
of search.

FWIW

Robert Dodier

mabshoff

unread,
Feb 9, 2009, 5:12:30 PM2/9/09
to sage-devel


On Feb 9, 10:39 am, mhampton <hampto...@gmail.com> wrote:
> I think this is a great idea.  It might not solve the problem of
> clutter immediately, because if something like eigenvectors_right is
> removed it would break a lot of existing code.  But perhaps we could
> remove such things in 4.0 after adding deprecation warnings.
>
> -Marshall

We should never intentionally break people's code like that if there
is no pressing issue. Adding deprecation warnings is ok to point
people in the right direction, but from discussions about this at SD12
and in other places it has become clear at least to me that six months
is not even remotely long enough for people to change their code and
adapt.

We really don't want to get the same reputation as Maple (which is not
fully deserved as one might point out in perspective of the recent det
() is dog slow thread) that major new Sage releases break code.
Getting rid of the MPolynomial constructor after 6 months is ok, but
we have had cases where people complained in trac about functions
being removed/renamed after they had been in *one* Sage release (The
jacobian thing).

Breaking backward compatibility might seem like a small annoyance to
people accustomed to the fast changes of Sage, but for the vast silent
majority out there is it a major problem since the documentation from
release to release is not 100% clear that $FOO has been removed and
$BAR should be used. We haven't officially removed any deprecated
functionality yet, so no surprise there, but we do not even mention
deprecated functionality either. As a first step we need a well
maintained wiki page where all such changes are documented with code
snippets explaining before and after to make it near idiot proof. And
for now we need a listing with what functions are even deprecated as a
start. Volunteers?

Cheers,

Michael

john_perry_usm

unread,
Feb 9, 2009, 5:36:05 PM2/9/09
to sage-devel
> This is a bit arbitrary for my tastes, but I think it (or somthing
> similar) could work:
>
> If you type foo.X<TAB>, where X is only one or two characters, then
> the extended completion list checks for *_X*.  Otherwise (if X is
> three or more characters) then the extended completion list is *X*.

I personally prefer that foo.X<TAB> should check for foo.X*, not for
*_X*. I don't follow the logic on that one. I agree with the one or
two characters though.

john perry

john_perry_usm

unread,
Feb 9, 2009, 5:46:49 PM2/9/09
to sage-devel
What if there were a different trigger for the extended completions?
This way the user would have only one box to parse at a time.

john perry

Simon King

unread,
Feb 10, 2009, 2:34:17 AM2/10/09
to sage-devel
Hi!

On Feb 9, 11:46 pm, john_perry_usm <john.pe...@usm.edu> wrote:
> What if there were a different trigger for the extended completions?
> This way the user would have only one box to parse at a time.

Yes!
I could imagine:
1. FOO.X<TAB> searches for attributes that start with X (current
behaviour)
2. FOO.X<Shift-TAB> searches for attributes that *contain* X (new
feature)



Cheers
Simon

Robert Bradshaw

unread,
Feb 10, 2009, 2:49:53 AM2/10/09
to sage-...@googlegroups.com

I like this idea much better. I often use tab completion not just to
find methods, but because it's faster than typing them out
completely. Cluttering foo.ei<tab> with everything that contains ei
would be a step backwards.

- Robert

Franco Saliola

unread,
Feb 10, 2009, 2:56:40 AM2/10/09
to sage-...@googlegroups.com
On Tue, Feb 10, 2009 at 8:49 AM, Robert Bradshaw
<robe...@math.washington.edu> wrote:
>
> On Feb 9, 2009, at 11:34 PM, Simon King wrote:
>
>> Yes!
>> I could imagine:
>>  1. FOO.X<TAB> searches for attributes that start with X (current
>> behaviour)
>>  2. FOO.X<Shift-TAB> searches for attributes that *contain* X (new
>> feature)

+1! For exactly the reasons Robert Bradshaw gave:

> I like this idea much better. I often use tab completion not just to
> find methods, but because it's faster than typing them out
> completely. Cluttering foo.ei<tab> with everything that contains ei
> would be a step backwards.

Franco

--

boo...@u.washington.edu

unread,
Feb 10, 2009, 3:08:58 AM2/10/09
to sage-...@googlegroups.com



On Tue, 10 Feb 2009, Franco Saliola wrote:

>
> On Tue, Feb 10, 2009 at 8:49 AM, Robert Bradshaw
> <robe...@math.washington.edu> wrote:
>>
>> On Feb 9, 2009, at 11:34 PM, Simon King wrote:
>>
>>> Yes!
>>> I could imagine:
>>>  1. FOO.X<TAB> searches for attributes that start with X (current
>>> behaviour)
>>>  2. FOO.X<Shift-TAB> searches for attributes that *contain* X (new
>>> feature)
>

I like this. I'm a little nervous about adding another tab keycode, since supporting Opera makes anything we do with tab obnoxious. Technical details aside, I think it's a great idea. Maybe I'll submit a bug report to Opera if it doesn't work easily.

Martin Albrecht

unread,
Feb 10, 2009, 5:02:36 AM2/10/09
to sage-...@googlegroups.com
> We should never intentionally break people's code like that if there
> is no pressing issue. Adding deprecation warnings is ok to point
> people in the right direction, but from discussions about this at SD12
> and in other places it has become clear at least to me that six months
> is not even remotely long enough for people to change their code and
> adapt.

I disagree, we should break whatever we dislike for a *X.0* release (+
DeprecationWarning for some months) Sage is a young agile fast moving project
and shouldn't be burdened with all kinds of old stuff just lying around. I
really liked Roman's comment on this matter, i.e. that we should change stuff
around while we can.

It seems to me: Sage isn't done yet and pretending it was does more harm than
good by making it more difficult to contribute.

Also, this whole 'don't break people's code' thing is a sham to some extend
anyway since the behavior of functions changes so much even in minor releases
anyway. Also it is those kind of changes that lead to subtle bugs that are
hard to notice in contrast to someting like an AttributeError.

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
_www: http://www.informatik.uni-bremen.de/~malb
_jab: martinr...@jabber.ccc.de

mabshoff

unread,
Feb 10, 2009, 6:41:42 AM2/10/09
to sage-devel


On Feb 10, 2:02 am, Martin Albrecht <m...@informatik.uni-bremen.de>
wrote:

Hi,

> > We should never intentionally break people's code like that if there
> > is no pressing issue. Adding deprecation warnings is ok to point
> > people in the right direction, but from discussions about this at SD12
> > and in other places it has become clear at least to me that six months
> > is not even remotely long enough for people to change their code and
> > adapt.
>
> I disagree, we should break whatever we dislike for a *X.0* release (+
> DeprecationWarning for some months) Sage is a young agile fast moving project
> and shouldn't be burdened with all kinds of old stuff just lying around. I
> really liked Roman's comment on this matter, i.e. that we should change stuff
> around while we can.

Well, there are enough people around that if you remove right_kernel()
or whatever will end up with broken code. Having it deprecated for a
year or two seems to have zero cost to me.

> It seems to me: Sage isn't done yet and pretending it was does more harm than
> good by making it more difficult to contribute.

Why? Can you make a concrete example at what you are driving at?
Methods with underscore are exempted from the deprecation rule since
those are internal, but while some people including you have claimed
that certain APIs in Sage are holding back progress I have not seen
anyone give me an example.

> Also, this whole 'don't break people's code' thing is a sham to some extend
> anyway since the behavior of functions changes so much even in minor releases
> anyway. Also it is those kind of changes that lead to subtle bugs that are
> hard to notice in contrast to someting like an AttributeError.

Again: Please give an example. Behavioral are either unintentional or
bugfixes AFAIK.

I think you really overestimate the pain deprecations and removal of
functionality does cause to the casual user. The GAP 3 -> GAP 4
transition with ample breakage was quite painful to a lot of people
and there are even some examples of very nice code that never moved
from GAP 3 to GAP 4 to this day, either because the author didn't care
because he kept using GAP 3 or alternatively because the author had
left academia and was no longer interested in porting the code.
Another example of when a user community to a large extend did not
move from one version to the incompatible followup release is Macaulay
2.

Overall: I am all for removing cruft, but given that we deprecated
less than 10 functions over the last 8 months or so this API cruft
must not exist to the extend you claim. I talked to Burcin about this
and he raised some concrete examples, but he might want to describe
them since my recollection is hazy.

> Cheers,
> Martin

Cheers,

Michael

> --
> name: Martin Albrecht
> _pgp:http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
> _otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
> _www:http://www.informatik.uni-bremen.de/~malb
> _jab: martinralbre...@jabber.ccc.de

Martin Albrecht

unread,
Feb 10, 2009, 7:08:16 AM2/10/09
to sage-...@googlegroups.com
> > It seems to me: Sage isn't done yet and pretending it was does more harm
> > than good by making it more difficult to contribute.
>
> Why? Can you make a concrete example at what you are driving at?
> Methods with underscore are exempted from the deprecation rule since
> those are internal, but while some people including you have claimed
> that certain APIs in Sage are holding back progress I have not seen
> anyone give me an example.

I have to admit that I don't have any concrete example, so I take your point.
It is a somewhat fuzzy even psychological issue I think. But since I have to
admit it is fuzzy, I take it back until further notice.

I can think of hypothetical examples but nothing concrete right now from the
top of my head.

> > Also, this whole 'don't break people's code' thing is a sham to some
> > extend anyway since the behavior of functions changes so much even in
> > minor releases anyway. Also it is those kind of changes that lead to
> > subtle bugs that are hard to notice in contrast to someting like an
> > AttributeError.
>
> Again: Please give an example. Behavioral are either unintentional or
> bugfixes AFAIK.

Sure thing:

http://www.sagemath.org/hg/sage-main/annotate/b3006824b208/sage/rings/integer.pyx

broke my code in a subtle way, i.e. it didn't crash but gave wrong answers. Of
course, I could easily track it down and fix my code. Sure thing it wasn't
intended to break my code but it did. Maintaining backward compatibility for
this function however would have meant to stick with my ad hoc decision that
digits() should have base two.

At least some code in Sage is not based on a lengthly well thought through
design process but on a developer's 'taste'. The backward compatibility rule
would mean to stick to this decision until the end of time.

> I think you really overestimate the pain deprecations and removal of
> functionality does cause to the casual user.

Do you mean to say 'underestimate'?

> The GAP 3 -> GAP 4
> transition with ample breakage was quite painful to a lot of people
> and there are even some examples of very nice code that never moved
> from GAP 3 to GAP 4 to this day, either because the author didn't care
> because he kept using GAP 3 or alternatively because the author had
> left academia and was no longer interested in porting the code.
> Another example of when a user community to a large extend did not
> move from one version to the incompatible followup release is Macaulay
> 2.

Sure, stuff like this happens (a lot), but I really like Roman's answer to
this: Make sure your new version is 10x more awesome than the last one and
people will switch.

Also, GAP3 was a mature system, Sage is not. I think we should build the
system before thinking about maintaining bug-by-bug backward compatibility.

> Overall: I am all for removing cruft, but given that we deprecated
> less than 10 functions over the last 8 months or so this API cruft
> must not exist to the extend you claim. I talked to Burcin about this
> and he raised some concrete examples, but he might want to describe
> them since my recollection is hazy.

Think about e.g. the planed break up of gen.pyx into elements that make sense.
Maintaining bug-by-bug backward compatibility will be a nightmare because the
file is pretty messy. I'm sure one can do all kind of weird things with
gen.pyx right now that one shouldn't be able to do. Shall we keep this
functionality for backward compatibility?

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
_www: http://www.informatik.uni-bremen.de/~malb

_jab: martinr...@jabber.ccc.de

mabshoff

unread,
Feb 10, 2009, 7:26:10 AM2/10/09
to sage-devel


On Feb 10, 4:08 am, Martin Albrecht <m...@informatik.uni-bremen.de>
wrote:

Hi Martin,

> > > It seems to me: Sage isn't done yet and pretending it was does more harm
> > > than good by making it more difficult to contribute.
>
> > Why? Can you make a concrete example at what you are driving at?
> > Methods with underscore are exempted from the deprecation rule since
> > those are internal, but while some people including you have claimed
> > that certain APIs in Sage are holding back progress I have not seen
> > anyone give me an example.
>
> I have to admit that I don't have any concrete example, so I take your point.
> It is a somewhat fuzzy even psychological issue I think. But since I have to
> admit it is fuzzy, I take it back until further notice.
>
> I can think of hypothetical examples but nothing concrete right now from the
> top of my head.

:)

> > > Also, this whole 'don't break people's code' thing is a sham to some
> > > extend anyway since the behavior of functions changes so much even in
> > > minor releases anyway. Also it is those kind of changes that lead to
> > > subtle bugs that are hard to notice in contrast to someting like an
> > > AttributeError.
>
> > Again: Please give an example. Behavioral are either unintentional or
> > bugfixes AFAIK.
>
> Sure thing:
>
> http://www.sagemath.org/hg/sage-main/annotate/b3006824b208/sage/rings...
>
> broke my code in a subtle way, i.e. it didn't crash but gave wrong answers. Of
> course, I could easily track it down and fix my code. Sure thing it wasn't
> intended to break my code but it did. Maintaining backward compatibility for
> this function however would have meant to stick with my ad hoc decision that
> digits() should have base two.

Yes, in retrospect "#2796: digits default base changed to 10 instead
of 2." was not the best decision we make. It certainly should have
been prominently mentioned in the release notes (I do not recall
details), but the motivation was to make the various interfaces' use
of digits and bits consistent IIRC.

But I think that we are still talking about different kinds of
backward compatibility.

> At least some code in Sage is not based on a lengthly well thought through
> design process but on a developer's 'taste'. The backward compatibility rule
> would mean to stick to this decision until the end of time.
>
> > I think you really overestimate the pain deprecations and removal of
> > functionality does cause to the casual user.
>
> Do you mean to say 'underestimate'?

Yes, I saw my mistake once I hit "post", but I assumed it was clear
that I meant underestimate, so I did not send a correction.

> > The GAP 3 -> GAP 4
> > transition with ample breakage was quite painful to a lot of people
> > and there are even some examples of very nice code that never moved
> > from GAP 3 to GAP 4 to this day, either because the author didn't care
> > because he kept using GAP 3 or alternatively because the author had
> > left academia and was no longer interested in porting the code.
> > Another example of when a user community to a large extend did not
> > move from one version to the incompatible followup release is Macaulay
> > 2.
>
> Sure, stuff like this happens (a lot), but I really like Roman's answer to
> this: Make sure your new version is 10x more awesome than the last one and
> people will switch.

Yes. The way det() vs. determinant() is handles in Maple certainly
seems a little odd, i.e. after five or more years I would expect that
the slow functionality would give the user some kind of indication not
to use it. I do not have Maple documentation handy, but maybe the
documentation in paper form/html gives the user a clue.

> Also, GAP3 was a mature system, Sage is not. I think we should build the
> system before thinking about maintaining bug-by-bug backward compatibility.

We are not talking about bug-for-bug compatibility, we are talking
about removing functionality that is maintainable at next to zero
cost. If something is buggy it will get fixed and I don't think we
ever kept something buggy around not to break anyone's code.

And even though GAP 3 was a mature system the breaking of
compatibility was intentional and while I am certain GAP 4 is a better
system than GAP 3 the changes were significant enough that some people
did not follow even half a decade or later. We had had conversations
off list with some GAP 3 use who did consider switching to Sage since
the GAP 3 -> GAP 4 transition of his code was significant work and so
he considered Sage. The reason he no longer wanted to use GAP 3 was
not so much its limitations, but the plain and simple fact that GAP 3
was 32 bit and hence the limited RAM was too little to run his
computations.

> > Overall: I am all for removing cruft, but given that we deprecated
> > less than 10 functions over the last 8 months or so this API cruft
> > must not exist to the extend you claim. I talked to Burcin about this
> > and he raised some concrete examples, but he might want to describe
> > them since my recollection is hazy.
>
> Think about e.g. the planed break up of gen.pyx into elements that make sense.

Sure, but that will happen transparently without any impact to the
user's code.

> Maintaining bug-by-bug backward compatibility will be a nightmare because the
> file is pretty messy. I'm sure one can do all kind of weird things with
> gen.pyx right now that one shouldn't be able to do. Shall we keep this
> functionality for backward compatibility?

Probably not. But if someone knowingly or unknowingly abuses and
interface code will break. Can you give an example of what you mean? I
don't mean this in a negative way, I am truly curious :). I know that
gen.pyx could certainly use a spring cleaning and that in general as
the coverage creeps up we will encounter all kinds of bad code that
has been around for a while and no one has touched it. Sage must also
be made more consistent and in the process we will have to break some
things - this is unavoidable. It would be nice if we had something
like a major version jump to clearly indicate the before and after,
but it seems that this will be hard to accomplish consistently.

I don't know which bugs you are referring to, but as mentioned above
bugs will be fixed. This is not at all what from my POV deprecation is
about and again: I am all for removing cruft and fixing bugs, but
breaking up files and so on is not what my statements were about.

> Cheers,
> Martin
>

Cheers,

Michael
> --
> name: Martin Albrecht
> _pgp:http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
> _otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
> _www:http://www.informatik.uni-bremen.de/~malb
> _jab: martinralbre...@jabber.ccc.de

Martin Albrecht

unread,
Feb 10, 2009, 7:45:17 AM2/10/09
to sage-...@googlegroups.com
> > Think about e.g. the planed break up of gen.pyx into elements that make
> > sense.
>
> Sure, but that will happen transparently without any impact to the
> user's code.

I am pretty sure it will have an impact on user's code because it is supposed
to clean up the mess that is gen.pyx. Maintaining that interface would defeat
the purpose of the break up in the first place. On the other hand, maybe
that's not what Craig and Robert have in mind anwyay.

> > Maintaining bug-by-bug backward compatibility will be a nightmare because
> > the file is pretty messy. I'm sure one can do all kind of weird things
> > with gen.pyx right now that one shouldn't be able to do. Shall we keep
> > this functionality for backward compatibility?
>
> Probably not. But if someone knowingly or unknowingly abuses and
> interface code will break. Can you give an example of what you mean?

How do we define that someone 'abuses' a class when it is full of weird stuff
(e.g. stuff to make things faster before proper code was written etc.) That
kind of stuff has to go because it encourages bad code to be written.

> I don't mean this in a negative way, I am truly curious :). I know that
> gen.pyx could certainly use a spring cleaning and that in general as
> the coverage creeps up we will encounter all kinds of bad code that
> has been around for a while and no one has touched it. Sage must also
> be made more consistent and in the process we will have to break some
> things - this is unavoidable. It would be nice if we had something
> like a major version jump to clearly indicate the before and after,
> but it seems that this will be hard to accomplish consistently.

The major version jump is *exactly* what I've been lobbying for since SD12 :)

If we can't manage to pull this one off, then at least we should have a
section in the release notes: this and that behavior was changed and might
affect your code. Also, if we can we add deprecation warnings (and eventually
we kill the code if it causes hassles)?

Cheers,
Martin


--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
_www: http://www.informatik.uni-bremen.de/~malb

_jab: martinr...@jabber.ccc.de

Stan Schymanski

unread,
Feb 10, 2009, 7:57:26 AM2/10/09
to sage-...@googlegroups.com
+1 for maintaining backwards compatibility as much as practically possible.

Mathematica 5.2, probably considered a mature program, was upgraded to
Mathematica 6.0 and broke most of my code. This gave me the final
incentive to look for alternatives as I would have to spend hours and
hours to fix my old code and understand the new syntax, anyway. That's
how I found SAGE. If SAGE started to break code with every release and
require hours of work just to get the old code to work again, I would
become very frustrated, and possibly a range of others that use SAGE for
their day-to-day work, too. So please, please, try to minimise the
amount of work needed to port people's code to new versions of SAGE. I
know, it's a cost/benefit consideration and I don't have a good
apprehension of the costs, but I wanted to add my 2 cents to the
benefits of maintaining backward compatibility.

Thanks again for the great work done so far and for openly discussing
these questions.

Stan

PS: To reduce some of the costs, why not just remove deprecated
functions from the tab completion list but still leave them in the name
space?
--
________________________________________

Stan Schymanski
Scientist
Max Planck Institute for Biogeochemistry
Postfach 10 01 64
D-07701 Jena

Phone: +49.3641.576264
Fax: +49.3641.577274
WWW: http://www.bgc-jena.mpg.de/~sschym

Biospheric Theory and Modelling Group
http://www.bgc-jena.mpg.de/bgc-theory/
_________________________________________

mabshoff

unread,
Feb 10, 2009, 8:13:07 AM2/10/09
to sage-devel


On Feb 10, 4:45 am, Martin Albrecht <m...@informatik.uni-bremen.de>
wrote:

Hi Martin,

> > > Think about e.g. the planed break up of gen.pyx into elements that make
> > > sense.
>
> > Sure, but that will happen transparently without any impact to the
> > user's code.
>
> I am pretty sure it will have an impact on user's code because it is supposed
> to clean up the mess that is gen.pyx. Maintaining that interface would defeat
> the purpose of the break up in the first place. On the other hand, maybe
> that's not what Craig and Robert have in mind anwyay.

It is my understanding that the breakup of gen.pyx is purely
logistical, i.e. the file is simply too large and has code that should
live in separate files. AFAIK there are no plans to deprecate
functionality.

> > > Maintaining bug-by-bug backward compatibility will be a nightmare because
> > > the file is pretty messy. I'm sure one can do all kind of weird things
> > > with gen.pyx right now that one shouldn't be able to do. Shall we keep
> > > this functionality for backward compatibility?
>
> > Probably not. But if someone knowingly or unknowingly abuses and
> > interface code will break. Can you give an example of what you mean?
>
> How do we define that someone 'abuses' a class when it is full of weird stuff
> (e.g. stuff to make things faster before proper code was written etc.) That
> kind of stuff has to go because it encourages bad code to be written.

There was a case at SD 12 where Dan Shumow reviewed some Graph theory
patch by Robert Miller. Dan raised the issue that some code that
previously worked was broken and Robert replied that the code Dan
presented worked merely by accident since some of the assumptions in
Robert's code were not fulfilled in Dan's example. That property was
not checked in Robert's code IIRC. Either one of those two can supply
you with all the gory details or if you want me to I can find the
ticket.

I agree with you that such bugs in interfaces need to be fixed, but
that only happens once you have users that are no familiar with the
internals of the code and use it in a way never intended by the author
since the author writes his code in a way it is supposed to be used.
Overall in Sage (as well as any other project) code goes through
phases, i.e. the initial drop which is often shakier than it ought to
be. Then sooner or later other people start to hammer the code and
bugs get shaken out. If we are really lucky someone takes a large
amount of tests and verifies the code against known good third party
implementations. One such example is the graph planarity code where
Jason Grout did throw all graphs below a certain size at the code and
found quite a bunch of problems, in the Sage interface to the external
code as well as the external code itself. Unfortunately this is just
how code matures, i.e. even long standing "internal" testing of some
code by the same group of people will only find so many bugs. Once you
have the "idiot user" [my apologies to the idiot user :)] throw random
things at the interface things break. This has happened so many times
to my non-mathematical as well as mathematical code where I thought
the interface I wrote was idiot proof I no longer claim that such a
thing can be accomplished. But we are getting OT here and I need some
sleep, so I will stop ranting :)

> > I don't mean this in a negative way, I am truly curious :). I know that
> > gen.pyx could certainly use a spring cleaning and that in general as
> > the coverage creeps up we will encounter all kinds of bad code that
> > has been around for a while and no one has touched it. Sage must also
> > be made more consistent and in the process we will have to break some
> > things - this is unavoidable. It would be nice if we had something
> > like a major version jump to clearly indicate the before and after,
> > but it seems that this will be hard to accomplish consistently.
>
> The major version jump is *exactly* what I've been lobbying for since SD12 :)

Yes, I understand your position very well, I just disagree in some of
the finer points :)

> If we can't manage to pull this one off, then at least we should have a
> section in the release notes: this and that behavior was changed and might
> affect your code.

+1 - Minh has been helping out with the release notes and I hope they
have been better and more consistent since then. He certainly can
spell a whole lot better than that idiot who used to write the
notes :p

> Also, if we can we add deprecation warnings (and eventually
> we kill the code if it causes hassles)?

I am not sure if this sentence is done. But I agree that if we
deprecate code we must remove it after some time period. Maybe you
should submit a patch to remove the MPolynomial constructor in Sage
3.3 since it was the first thing we ever deprecated. That way we can
add a section to the release notes about deprecation and removal of
functionality and have it in the default template just like coverage
and all those other sections.

Anyway, we do not fundamentally disagree, just in the details, so a
consensus will be reached at some point.

> Cheers,
> Martin
> --

Cheers,

Michael

> name: Martin Albrecht
> _pgp:http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
> _otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
> _www:http://www.informatik.uni-bremen.de/~malb
> _jab: martinralbre...@jabber.ccc.de

mhampton

unread,
Feb 10, 2009, 8:18:18 AM2/10/09
to sage-devel
I think its hard to get a one-size-fits-all policy here. I am
thinking specifically of the polytope code in polyhedra.py - I am very
happy with the progress so far, but its still young code that has bugs
and some architectural issues. In the last few months more people
have started looking at it and using it, and Arnaud and Sebastien have
begun contributing. The growing number of users means that of course
we should avoid breaking things, but as more developers take a serious
look at it I expect some substantial changes will be desirable. As
much as possible that will be done on the back end, but I can easily
imagine becoming convinced that an overhaul would be worthwhile.

-Marshall

mabshoff

unread,
Feb 10, 2009, 8:40:15 AM2/10/09
to sage-devel


On Feb 10, 5:18 am, mhampton <hampto...@gmail.com> wrote:
> I think its hard to get a one-size-fits-all policy here.  I am
> thinking specifically of the polytope code in polyhedra.py - I am very
> happy with the progress so far, but its still young code that has bugs
> and some architectural issues.  In the last few months more people
> have started looking at it and using it, and Arnaud and Sebastien have
> begun contributing.  The growing number of users means that of course
> we should avoid breaking things, but as more developers take a serious
> look at it I expect some substantial changes will be desirable.  As
> much as possible that will be done on the back end, but I can easily
> imagine becoming convinced that an overhaul would be worthwhile.

Yes, but in that case you will hopefully be able to deprecate old
functionality. And if push comes to shove you can always start a
completely new interface in another file and then transition code over
as it turns out to be useful and stable. And some parts of Sage have
significantly more users than Sage than others. Break generic
infrastructure like ring constructors and we are in a world of pain.

> -Marshall
>
> On Feb 10, 2:57 pm, Stan Schymanski <schym...@gmail.com> wrote:
>
> > +1 for maintaining backwards compatibility as much as practically possible.
>
> > Mathematica 5.2, probably considered a mature program, was upgraded to
> > Mathematica 6.0 and broke most of my code. This gave me the final
> > incentive to look for alternatives as I would have to spend hours and
> > hours to fix my old code and understand the new syntax, anyway. That's
> > how I found SAGE. If SAGE started to break code with every release and
> > require hours of work just to get the old code to work again, I would
> > become very frustrated, and possibly a range of others that use SAGE for
> > their day-to-day work, too. So please, please, try to minimise the
> > amount of work needed to port people's code to new versions of SAGE. I
> > know, it's a cost/benefit consideration and I don't have a good
> > apprehension of the costs, but I wanted to add my 2 cents to the
> > benefits of maintaining backward compatibility.
>
> > Thanks again for the great work done so far and for openly discussing
> > these questions.

No problem. This is a large part why I work on Sage since there is no
handed down decision from some political type :)

> > Stan

I have thought about this some more. Let me use the Linux kernel as an
analogy: There is kernel space and a user space API. Pretty much
anything goes in kernel space provided the user space API is preserved
and keeps working. Changes like splitting up gen.pyx should be
considered kernel space work and if you create a number field that
uses gen.pyx this is part of the user space API and you should not be
aware at all that anything changed had gen.pyx be split up. But that
is not the whole picture and when I talked about preserving backward
compatibility I am talking about the user space API, not the Sage
library internals. Obviously you can write code in user space that
does break when kernel level code changes, i.e. if you cimport
something in Cython code things might go bad if something moved from
gen.pyx into another file. But that seems unavoidable and is more or
less the issue Martin was getting at.

There is also code that should be in the Sage library since it is
tightly coupled with it, but it isn't [yet]. One such example was
pointed out to me in anther context a couple nights ago in IRC by Nick
Alexander. He was talking about the fact that recent Sage releases had
broken Skoruppa's Siegel modular forms code (It is written by a the
group he chairs, but let's call it Skoruppa's code for now). If that
code were in the Sage library and 100% doctested (last time I took a
peek it wasn't) any patch being merged that broke that code would
either cause a fix to Skoruppa's code if it exposed a bug in said code
or alternatively a correction to the patch. I run all doctests in long
mode after each patch I merge and if you follow track you would be
surprised how many obvious and trivial patches get bounced back since
they broke something. This is the driving motivation behind 100%
coverage, not to torture code writes :).

The same applies to pickling where a recent split up of plot.py caused
Mike Hansen to add some less than elegant code in plot.py to still let
us unpickle plot objects. Last week Carl Witty pointed out something
clever that might us work around the problem of having crud like that
for code we move, but that hasn't been tested yet. An analog thing
happened when William, Craig and Robert (IIRC) tracked down some code
changes that cause a bunch of data computed after Dev1 at an L
functions workshop to no longer unpickle and they fixed the problem.
William was quite unhappy that the pickles had been broken and back
then it caused him to push hard for the pickle jar and finish the
project. All these little details make the difference between an ok
and a great project and I am really hoping we will be willing to go
the extra mile.

Cheers,

Michael

William Stein

unread,
Feb 10, 2009, 8:48:38 AM2/10/09
to sage-...@googlegroups.com
On Tue, Feb 10, 2009 at 4:57 AM, Stan Schymanski <schy...@gmail.com> wrote:
>
> +1 for maintaining backwards compatibility as much as practically possible.
>
> Mathematica 5.2, probably considered a mature program, was upgraded to
> Mathematica 6.0 and broke most of my code. This gave me the final
> incentive to look for alternatives as I would have to spend hours and
> hours to fix my old code and understand the new syntax, anyway. That's
> how I found SAGE. If SAGE started to break code with every release and
> require hours of work just to get the old code to work again, I would
> become very frustrated, and possibly a range of others that use SAGE for
> their day-to-day work, too. So please, please, try to minimise the
> amount of work needed to port people's code to new versions of SAGE. I
> know, it's a cost/benefit consideration and I don't have a good
> apprehension of the costs, but I wanted to add my 2 cents to the
> benefits of maintaining backward compatibility.

You have been using Sage for about a year now, or more. In your
opinion, how are we doing so far regarding breaking or not breaking
*your* code with each new release of Sage? When your code does break,
how responsive have we been.

-- William

Stan Schymanski

unread,
Feb 10, 2009, 9:12:07 AM2/10/09
to sage-...@googlegroups.com
William Stein wrote:
>
> You have been using Sage for about a year now, or more. In your
> opinion, how are we doing so far regarding breaking or not breaking
> *your* code with each new release of Sage? When your code does break,
> how responsive have we been.
>
The only time I can remember that something didn't work after an update
was some of my %interact code. The original code I wrote (or copied and
edited) was so bad that it probably shouldn't have worked in the first
place. :)

In general, I find the response to any of my problems on sage-support or
sage-devel incredibly fast and helpful. There are a few that passed
below the radars, though. I thought this had something to do with the
query itself, but since I often get helpful replies straight away if I
re-submit the query after a month or so, I suspect that it is just a
matter of how busy people are and how quickly the message gets buried by
other messages. Some people may not be as obnoxious as I am though, and
they might get frustrated if they never receive an answer. Maybe it
would be good to somehow keep unanswered emails at the top of list. Not
sure if this is possible in Google Groups.

Stan
> -- William
>
> >
>

William Stein

unread,
Feb 10, 2009, 9:38:03 AM2/10/09
to sage-...@googlegroups.com
On Tue, Feb 10, 2009 at 6:12 AM, Stan Schymanski <schy...@gmail.com> wrote:
> In general, I find the response to any of my problems on sage-support or
> sage-devel incredibly fast and helpful. There are a few that passed
> below the radars, though. I thought this had something to do with the
> query itself, but since I often get helpful replies straight away if I
> re-submit the query after a month or so, I suspect that it is just a
> matter of how busy people are and how quickly the message gets buried by
> other messages. Some people may not be as obnoxious as I am though, and
> they might get frustrated if they never receive an answer. Maybe it
> would be good to somehow keep unanswered emails at the top of list. Not
> sure if this is possible in Google Groups.
> Stan

Since you claim to be obnoxious, perhaps you could ping the list if
you see a message
that has sat there unanswered for a while. :-)

William

mabshoff

unread,
Feb 10, 2009, 9:58:52 AM2/10/09
to sage-devel


On Feb 10, 6:12 am, Stan Schymanski <schym...@gmail.com> wrote:
> William Stein wrote:

Hi Stan,

>. Maybe it
> would be good to somehow keep unanswered emails at the top of list. Not
> sure if this is possible in Google Groups.

I am not aware of any mechanism that would let you do that in Google
grops. You can chose "view all" and easily navigate back a couple days
and identify unanswered questions, but given the volume of the lists
once it scrolls off the first page or two it is gone :(. So if
anything goes unanswered just bump it, usually it just slips through
the cracks.

> Stan

Cheers,

Michael

Jason Grout

unread,
Feb 10, 2009, 10:11:22 AM2/10/09
to sage-...@googlegroups.com
Franco Saliola wrote:
> On Tue, Feb 10, 2009 at 8:49 AM, Robert Bradshaw
> <robe...@math.washington.edu> wrote:
>> On Feb 9, 2009, at 11:34 PM, Simon King wrote:
>>
>>> Yes!
>>> I could imagine:
>>> 1. FOO.X<TAB> searches for attributes that start with X (current
>>> behaviour)
>>> 2. FOO.X<Shift-TAB> searches for attributes that *contain* X (new
>>> feature)
>
> +1! For exactly the reasons Robert Bradshaw gave:

+1 as well for exactly the reasons that Robert gave. If we do this, I
think we ought to heavily market the shift-tab functionality.

Jason


Stan Schymanski

unread,
Feb 10, 2009, 10:14:26 AM2/10/09
to sage-...@googlegroups.com
I have, but unfortunately, only for messages I've been interested in
myself, otherwise I don't notice them. Since it is not possible to
automatically highlight unanswered messages, we could encourage people
in the guidelines to repeat their unanswered questions.

Jason Grout

unread,
Feb 10, 2009, 10:16:50 AM2/10/09
to sage-...@googlegroups.com
mabshoff wrote:
>
>
> On Feb 10, 2:02 am, Martin Albrecht <m...@informatik.uni-bremen.de>
> wrote:
>
> Hi,
>
>>> We should never intentionally break people's code like that if there
>>> is no pressing issue. Adding deprecation warnings is ok to point
>>> people in the right direction, but from discussions about this at SD12
>>> and in other places it has become clear at least to me that six months
>>> is not even remotely long enough for people to change their code and
>>> adapt.
>> I disagree, we should break whatever we dislike for a *X.0* release (+
>> DeprecationWarning for some months) Sage is a young agile fast moving project
>> and shouldn't be burdened with all kinds of old stuff just lying around. I
>> really liked Roman's comment on this matter, i.e. that we should change stuff
>> around while we can.
>
> Well, there are enough people around that if you remove right_kernel()
> or whatever will end up with broken code. Having it deprecated for a
> year or two seems to have zero cost to me.

While this thread has gone off into deep waters, on this specific issue,
I don't see a problem with having both right_kernel and kernel_right, as
it is just an alias (or, given the problems that Nick Alexander points
out with aliases and subclassing, maybe a function which immediately
calls another function).

That said, if we'd like to eventually reduce namespace clutter (e.g.,
1-2 years), then I don't see the harm in doing what another person
suggested: remove kernel_right from the tab-completion list. That will
hide it so that (most) people won't write code using it, which cuts down
on the number of breakages in several years.

For what it's worth, the namespace clutter I was really thinking of
addressing was *future* clutter, not necessarily removing existing aliases.

Anyways, back to the regularly scheduled rants... :)

Jason

Simon King

unread,
Feb 10, 2009, 10:18:01 AM2/10/09
to sage-devel
Dear William,

On Feb 10, 2:48 pm, William Stein <wst...@gmail.com> wrote:
> You have been using Sage for about a year now, or more. In your
> opinion, how are we doing so far regarding breaking or not breaking
> *your* code with each new release of Sage? When your code does break,
> how responsive have we been.

Although that question was not addressed to me, my answer is:

I can remember only one change that broke my code: In
sage-3.3.alphasomething, so, very recent.

It concerns multivariate factorization. I had p.factor() in my code,
but at some point I got a NotImplementedError. As Martin pointed out,
it is due to a bug in Singular that the default(!) case p.factor
(proof=True) can not be implemented. So I had to change my code into
p.factor(proof=False)

Since I think I started with Sage 2.0.6, I would say that backwards
compatibility was quite ok.

And I do think that backwards compatibility (at least on the level of
"user space API", as Michael pointed out) is very important!

Cheers
Simon

Jason Grout

unread,
Feb 10, 2009, 10:20:12 AM2/10/09
to sage-...@googlegroups.com
Stan Schymanski wrote:
> I have, but unfortunately, only for messages I've been interested in
> myself, otherwise I don't notice them. Since it is not possible to
> automatically highlight unanswered messages, we could encourage people
> in the guidelines to repeat their unanswered questions.


Of course, those using Thunderbird or another (ahem, "proper" :)
newsreader with the gmane interface can easily see threads with only one
message.

(/me wonders if I just volunteered to answer every orphaned message ever :).

Jason

Carl Witty

unread,
Feb 10, 2009, 10:22:55 AM2/10/09
to sage-...@googlegroups.com

Note that this suggestion won't work on the command line.

Carl

mabshoff

unread,
Feb 10, 2009, 10:23:30 AM2/10/09
to sage-devel


On Feb 10, 7:16 am, Jason Grout <jason-s...@creativetrax.com> wrote:
> mabshoff wrote:

<SNIP>

> > Well, there are enough people around that if you remove right_kernel()
> > or whatever will end up with broken code. Having it deprecated for a
> > year or two seems to have zero cost to me.
>
> While this thread has gone off into deep waters, on this specific issue,
> I don't see a problem with having both right_kernel and kernel_right, as
> it is just an alias (or, given the problems that Nick Alexander points
> out with aliases and subclassing, maybe a function which immediately
> calls another function).

Yep.

> That said, if we'd like to eventually reduce namespace clutter (e.g.,
> 1-2 years), then I don't see the harm in doing what another person
> suggested: remove kernel_right from the tab-completion list.  That will
> hide it so that (most) people won't write code using it, which cuts down
> on the number of breakages in several years.

Yes, I thought this was a particularly clever solution. Let's hope it
isn't too much work to implement this.

> For what it's worth, the namespace clutter I was really thinking of
> addressing was *future* clutter, not necessarily removing existing aliases.

:)

> Anyways, back to the regularly scheduled rants... :)

Well, this thread was/is a lot more fun all the sudden, but I think we
won't be seeing too many rants from me any more on this topic today
since I have to fix about 10 issues for 3.3.rc0 :)

> Jason

Cheers,

Michael

Jason Grout

unread,
Feb 10, 2009, 10:21:53 AM2/10/09
to sage-...@googlegroups.com
boo...@u.washington.edu wrote:
>
>
>
> On Tue, 10 Feb 2009, Franco Saliola wrote:
>
>> On Tue, Feb 10, 2009 at 8:49 AM, Robert Bradshaw
>> <robe...@math.washington.edu> wrote:
>>> On Feb 9, 2009, at 11:34 PM, Simon King wrote:
>>>
>>>> Yes!
>>>> I could imagine:
>>>> 1. FOO.X<TAB> searches for attributes that start with X (current
>>>> behaviour)
>>>> 2. FOO.X<Shift-TAB> searches for attributes that *contain* X (new
>>>> feature)
>
> I like this. I'm a little nervous about adding another tab keycode, since supporting Opera makes anything we do with tab obnoxious. Technical details aside, I think it's a great idea. Maybe I'll submit a bug report to Opera if it doesn't work easily.
>

Since it's tricky, do you want to do this?

Does anyone know off the top of their head how to do something like this
in ipython?

Jason

William Stein

unread,
Feb 10, 2009, 10:25:26 AM2/10/09
to sage-...@googlegroups.com
On Tue, Feb 10, 2009 at 7:18 AM, Simon King <ki...@mathematik.uni-jena.de> wrote:
>
> Dear William,
>
> On Feb 10, 2:48 pm, William Stein <wst...@gmail.com> wrote:
>> You have been using Sage for about a year now, or more. In your
>> opinion, how are we doing so far regarding breaking or not breaking
>> *your* code with each new release of Sage? When your code does break,
>> how responsive have we been.
>
> Although that question was not addressed to me, my answer is:
>
> I can remember only one change that broke my code: In
> sage-3.3.alphasomething, so, very recent.
>
> It concerns multivariate factorization. I had p.factor() in my code,
> but at some point I got a NotImplementedError. As Martin pointed out,
> it is due to a bug in Singular that the default(!) case p.factor
> (proof=True) can not be implemented. So I had to change my code into
> p.factor(proof=False)

Interesting. That API change was my doing, actually. At Sage Days in
San diego, I discovered some new examples that shows that Singular's
factorization gives wrong answers even in the case of GF(p) !! You
might be actually *very* glad to find out about the above, since if
your code depends on factor actually giving a complete factorization,
it might have silently failed. Now you know to be concerned. Of
course, the real solution here will be to implement our own
Singular-free factorization that actually works, then get rid of the
need to pass proof=False in.

> Since I think I started with Sage 2.0.6, I would say that backwards
> compatibility was quite ok.
> And I do think that backwards compatibility (at least on the level of
> "user space API", as Michael pointed out) is very important!
>
> Cheers
> Simon
>
> >
>



--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

kcrisman

unread,
Feb 10, 2009, 10:30:15 AM2/10/09
to sage-devel
I don't have much of an opinion about the various tab-completion
options :)

But I would echo mabshoff's warnings about backwards incompatibility.
What would happen is not that people (and here I mean people who
download Sage, use it, and forget about any community - certainly
never email sage-support) would get upset about the breakage or
something. Instead, they would blissfully use the previous version of
Sage until it was so behind the times that there was no point in using
it, and when the next deal for a M comes by or a new professor steps
in who isn't very interested in using computers in the class or
whatever - Sage gets dumped. It's not exactly "hurting" anyone, but I
think it isn't a good strategy. (This is why I think that it is so
important to preserve, as options, bad but easy syntax for basic
things one would use in freshman/sophomore math courses.)

I don't think is as much of a problem for the research end, because
people that serious about it would be quite happy to contribute to
conversations like the one we are having here. But I think it is
quite important for there to be a long tail of users who never ask
anything on support but perhaps 5% of whom end up contributing to the
project long term.

One thing no one seems to have suggested is something that almost any
other freeware (open source or not), and pretty much all costware that
I have do - an automatic upgrade warning. I have no idea how this
would work in our context, but at least for certain releases, e.g.
bugfixes after major releases, perhaps the initialization sequence
would send a little message that "Sage 3.3 is now available! Go to
www.sagemath.org if you wish to upgrade.". Maybe this is a bad, or
unimplementable, idea, but at any rate I'll throw it out there. Oh,
and yes - 6 months is a VERY short time for a deprecation warning; if
you only have time to even think about a new Sage build in the summer,
there might be have been two sets of deprecated things in the
meantime!

Now, all that said, I think that so far the arguments for lots of
deprecation have come up against the reality that usually it's both-
and not either-or for deprecation, and that a change in documentation
that reminds someone that there is a better option suffices -
especially with the global function names, which I would view as the
most visible and hence most important ones not to change functionality
for. Sage seems to hit the balance precisely because both users on
the cutting edge and those a little behind it can actually let their
voices be heard. So maybe this is all a bit theoretical, at least at
the global namespace level.

- kcrisman

Stan Schymanski

unread,
Feb 10, 2009, 10:37:02 AM2/10/09
to sage-...@googlegroups.com
Hi Jason,

Could you post a link to a description of how to use TB as a "proper"
newsreader to access Google Groups? I only found this:
http://forums.mozillazine.org/viewtopic.php?f=30&t=636724

Not sure how to move this to the "recent unanswered sage-support posts"
discussion on sage-support.

Cheers,
Stan

Carl Witty

unread,
Feb 10, 2009, 10:38:53 AM2/10/09
to sage-...@googlegroups.com

Hmm... I was guessing that xterm had no way to pass a "shift-tab" to
an application, but actually it looks like I was wrong; it works for
Emacs, so it presumably works (or could work) for Sage. xterm sends
ESC [ Z
I have no idea how hard it would be to actually catch that. (In the
worst case, I could imagine that we would have to modify both readline
and IPython.)

I also don't know what other terminal emulators besides xterm do.

Carl

William Stein

unread,
Feb 10, 2009, 10:48:57 AM2/10/09
to sage-...@googlegroups.com
At least we can, since we ship both of those as part of Sage :-)

>
> I also don't know what other terminal emulators besides xterm do.
>
> Carl
>
> >
>



Jason Grout

unread,
Feb 10, 2009, 10:50:21 AM2/10/09
to sage-...@googlegroups.com
Stan Schymanski wrote:
> Hi Jason,
>
> Could you post a link to a description of how to use TB as a "proper"
> newsreader to access Google Groups? I only found this:
> http://forums.mozillazine.org/viewtopic.php?f=30&t=636724
>

Set up a new newsgroup account in Thunderbird. The server will be
news.gmane.org. Once you have the account in thunderbird set up, do
"Manage newsgroup subscriptions". You want newsgroup
gmane.comp.mathematics.sage.devel.

Let me know if you need more info.

Jason

Nicolas M. Thiery

unread,
Feb 10, 2009, 5:58:47 AM2/10/09
to sage-...@googlegroups.com
> >> On Feb 9, 2009, at 11:34 PM, Simon King wrote:
> >>> Yes!
> >>> I could imagine:
> >>>  1. FOO.X<TAB> searches for attributes that start with X (current
> >>> behaviour)
> >>>  2. FOO.X<Shift-TAB> searches for attributes that *contain* X (new
> >>> feature)

+1 as well

And a big +1 in general for this discussion. In MuPAD-Combinat we had
gone for the "eigenvectors_right" convention precisely for this
question of tab completion. When moving to sage, we discovered the
bla.*eigenvectors*? trick, and decided accordingly to drop the ugly.

Note: to me the most natural thing to do would be (and this would be
consistent with "?"):

FOO.X<TAB>: current behavior: searches for attributes that start with X

FOO.*X<TAB>: extended behavior: searches for attributes that contain X

One advantage is that this should require no change in the sage
interfaces. In particular, this does not require playing with special
inputs like Shift-TAB, which can be messy in terminals.

Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

Robert Bradshaw

unread,
Feb 10, 2009, 3:01:14 PM2/10/09
to sage-...@googlegroups.com

One disadvantage of this is that if one types Foo.X<tab> and doesn't
see the command one wants, one has to go all the way back and insert
the * rather than doing shift-tab at that point to get more options.
I'm not trying to mitigate the terminal issues though, no reason we
can't provide both options.

Also, I am +1 for the idea of removing deprecated functionality from
the tab completion lists too.

- Robert


Simon King

unread,
Feb 11, 2009, 2:44:32 AM2/11/09
to sage-devel
On Feb 10, 11:58 am, "Nicolas M. Thiery" <Nicolas.Thi...@u-psud.fr>
wrote:
...
> One advantage is that this should require no change in the sage
> interfaces. In particular, this does not require playing with special
> inputs like Shift-TAB, which can be messy in terminals.

Right, I didn't realize that Shift-TAB may have a special meaning. But
if it were possible to implement it, I would like it.

An alternative idea:
FOO.X<TAB> --> all attributes starting with X
FOO.X<TAB><TAB> (press TAB twice) --> all attributes containing X

Perhaps this is easier to implement.

Cheers
Simon

Simon King

unread,
Feb 11, 2009, 2:50:09 AM2/11/09
to sage-devel
Dear William,

slightly OT:

On Feb 10, 4:25 pm, William Stein <wst...@gmail.com> wrote:
...
> On Tue, Feb 10, 2009 at 7:18 AM, Simon King <k...@mathematik.uni-jena.de> wrote:
> > It concerns multivariate factorization. I had p.factor() in my code,
> > but at some point I got a NotImplementedError. As Martin pointed out,
> > it is due to a bug in Singular that the default(!) case p.factor
> > (proof=True) can not be implemented. So I had to change my code into
> > p.factor(proof=False)
>
> Interesting. That API change was my doing, actually. At Sage Days in
> San diego, I discovered some new examples that shows that Singular's
> factorization gives wrong answers even in the case of GF(p) !! You
> might be actually *very* glad to find out about the above, since if
> your code depends on factor actually giving a complete factorization,
> it might have silently failed.

Don't worry. I am happy if I find some factors, and it is no problem
for me if I get reducible factors.

Only I found it strange that the default case is not implemented.
However, Martin said that the default in Sage must yield a provably
correct result, but Singular can't provide it, so, the default can't
be implemented.

Cheers,
Simon

William Stein

unread,
Feb 11, 2009, 12:32:50 PM2/11/09
to sage-...@googlegroups.com

Yes, in Sage there was a lot of discussion and it was decided that
default implementations of algorithms should always be "proof=True",
unless it is crystal clear from the function name (e.g.,
next_pseudoprime).

<rant> The Singular documentation -- and me the authors of Singular
(?) -- seem to claim that they have implemented an algorithm that is
supposed to give a provably correct result. But it's not only not
"provable correct", it's frequently not correct at all. </rant>

-- William

Robert Bradshaw

unread,
Feb 12, 2009, 4:13:55 PM2/12/09
to sage-...@googlegroups.com

I actually like this perhaps a bit better, easier to type and to
remember. It is also more likely to be "discovered."

- Robert


Franco Saliola

unread,
Feb 12, 2009, 7:01:14 PM2/12/09
to sage-...@googlegroups.com

iPython, by default, lists all possible completions when the TAB key
(or Ctrl-l) is pressed. But this behaviour can be changed (via the
user's personal ipythonrc file) to: TAB completes, but does not show
completions; TAB-TAB shows completions.

I guess I wanted to point out that:

(1) With a special, non-default setting in iPython, TAB-TAB has some
significance.

(2) The fact that iPython already has a mechanism to detect and handle
two TAB keypresses should make implementation easier.

If we decided to implement Simon's suggestion (which I like), we
should keep the first issue in mind. Perhaps that behaviour can be
relegated to TAB-TAB-TAB? (If the option is set, then: TAB completes,
but does not show completions; TAB-TAB shows completions; TAB-TAB-TAB
shows all completions containing ....)

Take care,
Franco

--

Reply all
Reply to author
Forward
0 new messages