Call for vote about ticket #10963: axioms and more functorial constructions

[Nicolas M. Thiery]

Dear Sage developers,

This is a call for vote about the ticket:

#10963: axioms and more functorial constructions [1]

Sorry this post is long; it's trying to give a fair summary of the
huge discussion on [1]. Here is the table of contents:

- What is the ticket about
- Relevance
- History
- Debate
- Personal opinion
- Call for vote


What is the ticket about
------------------------

The main feature implemented by #10963 is an infrastructure for axioms
which enriches the current category infrastructure. Namely, if you
have a category, you can now ask for the (full) subcategory of the
objects satisfying a given axiom. For example:

sage: Groups().Finite()
Category of finite groups

Of course, this can be iterated:

sage: Magmas().Associative().Commutative().Unital().Inverse()
Category of commutative groups

By itself this is a little improvement, since it gives for example a
single entry point for all the variants of groups (or of magmas, ...);
in particular this will allow, as a later step, to remove a lot of
stuff from the global name space. This is in line with what we already
have for functorial constructions:

sage: Groups().CartesianProducts()
Category of cartesian products of groups

What's really new is that axioms play nicely with intersection:

sage: Groups() & Magmas().Commutative() & Sets().Finite()
Category of finite commutative groups

sage: (CommutativeAdditiveGroups() & Monoids()).Distributive()
Category of rings

sage: Rings().Division() & Sets().Finite() # Wedderburn's theorem
Category of finite fields

Fine, this is cute, but why is this relevant?

Relevance
---------

As time passes, the hierarchy of abstract classes for parents,
elements, and morphisms in Sage is getting larger and larger. From the
beginning, the purpose of categories in Sage has been to exploit the
strong semantic provided by the math to keep such large hierarchies
maintainable. The key is to fight hard against duplication and toward
single point of truth. Just as a trivial example, Sage deduces
automatically from the above that the abstract class for finite groups
(Groups().Finite().parent_class) should inherit from the abstract
class of groups and the abstract class of finite monoids.

Axioms are an important step to scale further by controling the
potential combinatorial explosion of the number of classes and
categories. For details, see the section on axioms in the primer [2].

History
-------

The work on this ticket started three years ago, growing up out of
strong immediate needs for the hierarchies of categories appearing in
the representation theory of semigroups or Hopf algebras. But the
infrastructure is also beneficial for non exotic categories like the
ones we have seen in the examples above. For example, it made it
practical to implement categories for the many variants of algebras
(unital or not, associative or not, commutative or not, ...)

For the design of the category infrastructure, we had some strong
source of inspiration from other systems like Axiom, MuPAD, or
Magma. As far as I know, no other system has a non trivial axiom
infrastructre like this one. Which means that it took a long time --
three years -- to analyse the needs, explore potential idioms, and
implement and reimplement the infrastructure until it felt right. It
also took as a prerequisite to fix a bunch of technical hurdles, in
particular #11935 and #13589 (thank you by the way to all who worked
and helped on those tickets, and in particular Simon and Nils).

In those three years, this ticket benefited from continuous
discussions with the other category fans, and in particular Florent
Hivert and Simon King. I also presented the framework at the occasion
of several Sage days to see how natural it would feel to new users and
developers. Most importantly, the framework was put to intensive use
in actual code to test whether the approach was sensible, viable, and
practical.

The undesirable consequence of this long history is that the ticket
has become large: it contains the axiom infrastructure, the
reorganization of the Sage categories using that infrastructure, a
bunch of new categories, and quite some related stuff that aggregated
to it along the way and would be painful to separate away (mostly
improvements to the existing functorial constructions).

Status
------

All test passes. The code is 100% doctested. All dependencies are
merged. Much time was spent on performance too, in particular checking
that the Sage startup time did not increase non trivially. Simon King
completely reviewed the code three months ago (thanks!!!) and gave a
positive review back then.

Since then, Simon, Nils, and Volker spotted and tracked down a bug
which is fixed now. I am super grateful to them because it was tricky:
the code was actually too paranoid! It did an assertion check too
early during some initialization, which caused a recursion loop
involving imports, and ended up with a silent garbage collection
issue.

This triggered a long discussion on trac where basically each and
every design choice was put again to trial, a good thing in itself!
Based on this discussion (thanks by the way to all the participants),
I added a 1500-lines overview text about axioms, including rationale
for the design decisions, the specification of the infrastructure, and
a detailed sketch of proof of correctness. This text, and some further
little improvements, are under review by Simon. There are also a few
further small suggestions from the reviewer's that I have offered to
implement, against my own opinion, if they really cared about
them. It's a matter of days before this is done.

This ticket is holding up a bunch of other tickets that depend on it,
either because they implement themselves new categories with axioms,
or use features from the ones implemented here. This includes:

- #14102: Non symmetric Macdonald polynomials. This is one of the main
outcomes of last year's ICERM semester. Up to one glitch that
appeared when merging with recent versions of Sage, it has been
ready to go since last June. It's an important feature and end users
are waiting for it.

- Work by PhD students (Jean-Baptiste Priez, ...) on Hopf algebras, a
large piece of which could go in very soon once #10963 is in. This
code is one of the major pieces of their PhD work, and they deserve
to get credit for it.

- #11111 and follow-ups on representation theory of finite dimensional
algebras and semigroups by Saliola, Aladin, Florent, myself, ...

- Further work on categories (support for morphisms, ...)

- A bunch of further stuff in the former Sage-Combinat queue

Taking this in consideration, since last June, basically all the
energy I could spend on Sage-development was devoted to finalizing
this ticket and getting it merged.

Debate
------

In my view, the most non obvious point about this ticket is whether
the general approach just makes any sense and is viable. De facto, it
took me three years to convince myself! Now I am convinced :-) And I
tried to be convincing in the documentation. Now, if you wonder about
this point, which is legitimate, actually sane, please read [2] and
decide for yourself. Comments and opinions are very welcome. Just
please create a new thread so that we don't mix the discussions.

As far as I can tell from the long trac discussion, there is a
reasonable consensus about this first point; I might be biased though.

There is also a consensus about a certain number of desirable further
features (see the TODO's in the documentation) that are best postponed
to follow-up tickets since this one is already way too large! One such
feature is to improve and make the overview text more concise, as it
matures thanks to the feedback from more and more developers reading
it.

Now, as often, there can be variants on how the approach is
implemented. And diverging opinions about those variants. Volker
dislikes some syntactical and implementation choices, and would like
to change some them. I dislike his counter propositions.

The difficulty in this discussion is that this is more about wild
guesses about the future; at this point, we lack concrete amo to back
up clear cut decisions; we just have forged different intuitions about
the right balances between different principles. That's fine by
itself: I like that developpers have opinions; it's a richness for the
Sage community. However we are stuck in an endless discussion with no
consensus emerging. And a lot of time is wasted on it when there is so
much else to do.

I think the divergence points are essentially about:

(1) The usage of nested classes: currently if Cs() is a category which
implements stuff for its subcategory Cs().A() of the objects
satisfying the axiom A, then that stuff is to be put in a class
Cs.A

Typically this class can be a nested class, or it can be put
elsewhere, e.g. in a lazily imported module, with a link from Cs;
this is very flexible.

(2) The "header" for categories with axioms

{{{
class Blah(Category):
# The currently implemented syntax
class Finite(CategoryWithAxiom):
...

# Volker's proposal, something like:
class ChooseYourName(axioms.Finite.Category):
...
}}}

plus possibly some attribute setting.

(3) Internal implementation points.


There is also a preexisting performance issue in some special
situations which is made worse by this ticket (see #15801). We have a
plan to fix it, and I am happy to settle this with Simon shortly after
#10963 is merged.

Personal opinion
----------------

About (1): we have been using the same syntax for functorial
constructions ever since the initial implementation of categories back
in 2009. It has proven a very practical way to structure the code, we
never had a problem with it, and developers learned it all right. I
don't see why we should deviate from this syntax for axioms given that
axioms are of the same nature as functorial construction, just with
more properties. If we decide for another syntax, then we should
change the syntax as well for functorial constructions. I am *not*
undertaking this change myself.

About (2): I had considered something similar earlier on, and
practical usage told me to drop it. In any cases, it's a rather small
change to the interface (one line or two per new category with
axiom). It can be implemented in a follow up ticket. Even in a couple
months there won't be hundreds of new categories with axioms
implemented out there. If we convince ourselves that we should really
change the interface, the categories with axioms in the Sage library
will be easy to fix. And we can warn and help the couple developpers
that will have implemented some in their private code.

About (3): the code is already functional. It certainly can be further
improved, but this can be postponned at no cost to followup tickets.


Conclusion: In such a situation where it comes to opinion against
opinion, I believe that decisions that have been battlefield tested
for sanity and practicality over a non trivial period (years) on a non
trivial body of code (50 categories just in the Sage library itself)
are as good as any other. The ticket is already way too big (I take my
share of the blame for it) and this is holding up the work of
many. Hence further development should go in follow up tickets,
especially if it's unclear whether they actually improve the
situation.

Let's not deviate even more from incremental development and the usual
"release early, release often".

Call for vote
-------------

[ ] Merge this ticket, and move on with further improvements in follow
up tickets
[ ] This ticket should be held up until everybody is happy
[ ] This ticket does not make any sense

Let me conclude by thanking everybody who has been involved in this
ticket. And also those who worked so hard on the new Sage development
workflow! It's been a life saver in the last months, and the previous
workflow has some responsibilities on the size of this ticket.

Cheers,
Nicolas

PS: I have tried hard to make a fair account of the situation. Yet
please read everything with a grain of salt, for this has been the
most frustrating development experience I ever had.

[1] http://trac.sagemath.org/ticket/10963
[2] http://sage.math.washington.edu/home/nthiery/sage-6.0/src/doc/output/html/en/reference/categories/sage/categories/primer.html

[Nathann Cohen]

> This ticket is holding up a bunch of other tickets that depend on it, 
> either because they implement themselves new categories with axioms, 
> or use features from the ones implemented here. This includes: 

This ticket, like every ticket on our trac server, will be merged when the reviewers will be happy with it. Period.

If you have any problem with that, please remember the 9999 times you were told that Sage-combinat was a fork of Sage, and too expensive to maintain. If you got other persons in trouble because of that now that Sage uses git, you are the only one to blame. You can be sorry, but that is no reason to merge anything before the reviewers (=not me, I don't get half of what this ticket is about) are happy.

Respect that. Don't attempt to short-cut the review process with polls.

> Let's not deviate even more from incremental development and the usual 
> "release early, release often". 

Is that a f*** joke ? Your ticket was created three years ago, and the branch is thousands of lines long. Sage-combinat is all about not releasing early and often. It's about keeping it all unreviewed for months while you work on it, before creating a patch-bomb ticket on Sage's trac server. If you had worked incrementally this would have been settled ages ago, not at the last minute.

Nathann

[Volker Braun]

First of all, Nicolas has done a lot of work on this ticket and I very much agree with his goal of providing a mixin framework for categories (called "axioms" on the ticket) so that you don't have to, say, separately write a FiniteCategory for each Category. Its also the hardest for me to say "No" since I generally want to make progress and push things forward. But I really feel uncomfortable with the syntax which IMHO deviates too much (and unnecessarily so) from normal Python.

I also think we agree that the implementation where a recursive computation is done on startup is not ideal. Relatively simple implementation mistakes in new category/axioms end up with an infinite recursion on startup, which I think is rather unhelpful for developers. There is also no way to doctest this recursion beyond "sage manages to start up". This is related to the fact that adding an assertion in the wrong place can easily lead to hard-to-debug infinite recursions, as Nicolas already mentioned. But then, this could all be improved on a later ticket.

What really bothers me with the ticket is the reliance on implicit magic over explicitly specifying relations between categories. From the ticket, this example:

sage: class Cs(Category):

....:     def super_categories(self):

....:         return [Sets()]

....:     class Finite(CategoryWithAxiom):

....:         class ParentMethods:

....:             def foo(self):

....:                 print "I am a method on finite C's"

implements

sage: P = Parent(category=Cs().Finite())

sage: P.foo()           # ok, nice

I am a method on finite C's

sage: P.is_finite()     # Where does that come from?

True

The theory here is that Sets defines a Finite axiom, and by some metaclass magic the inner class Cs.Finite ends up being a subcategory of Sets.Finite. But none of that is apparent from reading the code, you can pretty much only figure it out through reading the sage/categories/sets_cat.py or introspection if your Sage install manages to start up. It is well documented, which is nice but IMHO it would be even nicer if you could understand what is going on by just reading the code. Nicolas thinks this amount of breverity is a desirable feature, and I think we should make Cs.Finite be explicit about which axiom it implements (and, therefore, not require that it is named Cs.Finite if you don't want to). We've explored various ways to implement this syntax, either through inheritance or a class attribute, but Nicolas doesn't like any.

Nicolas keeps saying that we can change the syntax later, but I think that should be done right or not at all on this ticket (one can easily implement the axiom functionality without introducing the subclass syntax).

[Nathann Cohen]

> sage: P = Parent(category=Cs().Finite())
> sage: P.foo() # ok, nice
> I am a method on finite C's
> sage: P.is_finite() # Where does that come from?
> True

Is this method named 'is_finite' because the axiom's name is "finite"
? If the axiom is "HeyHeyHey" will the ".is_heyheyhey" method be
defined ? I am trying to understand, and I remember you mentionned
something about the name of a class being used some time ago...

If so, what is the point of defining this function ? It always returns
"true", and if you try .is_finite() on an object which does not have
Finite as an axiom you are meant to interpret AttributeException as a
"I do not know" ?

Nathann

[Volker Braun]

On Friday, March 7, 2014 1:50:29 PM UTC, Nathann Cohen wrote:

Is this method named 'is_finite' because the axiom's name is "finite"

No, because the Sets.Finite class implements a "is_finite" parent method (which always returns True). This happens to be the only parent method that finite sets implement, which is why I used it in this example. Snip from sage/categories/finite_sets.py:

class FiniteSets(CategoryWithAxiom):

    r"""

    The category of finite sets

    EXAMPLES::

        sage: C = FiniteSets(); C

        Category of finite sets

        sage: C.super_categories()

        [Category of sets]

        sage: C.all_super_categories()

        [Category of finite sets,

         Category of sets,

         Category of sets with partial maps,

         Category of objects]

        sage: C.example()

        NotImplemented

    TESTS::

        sage: TestSuite(C).run()

        sage: C is Sets().Finite()

        True

    """

    class ParentMethods:

        def is_finite(self):

            """

            Returns ``True`` since ``self`` is finite.

            EXAMPLES::

                sage: C = FiniteEnumeratedSets().example()

                sage: C.is_finite()

                True

            """

            return True

[Nathann Cohen]

Yoooooo !

> No, because the Sets.Finite class implements a "is_finite" parent method
> (which always returns True). This happens to be the only parent method that
> finite sets implement, which is why I used it in this example.

Oh, I see. And you would like to have an explicit link between this
Cs.Finite and Sets.Finite ?

Nathann

[Volker Braun]

On Friday, March 7, 2014 2:30:38 PM UTC, Nathann Cohen wrote:

Oh, I see. And you would like to have an explicit link between this
Cs.Finite and Sets.Finite ?

Yes. 

[kcrisman]

Two comments on this reply to the post:

Respect that. Don't attempt to short-cut the review process with polls.

That is true in general, but sometimes there have been cases where it really was more realistic to go to sage-devel.  That is particularly true with spkg upgrades at times, or the recent discussion about SPARC, or things like that.  With a huge change of some kind like this, it seems reasonable to invite the whole community to respond, since probably no one can ever be 100% happy with any given ticket - even one-character changes have sparked discussions, I think.

 

> Let's not deviate even more from incremental development and the usual 
> "release early, release often". 

Is that a f*** joke ? Your ticket was created three years ago, and the branch is thousands of lines long. Sage-combinat is all about not releasing early and often. It's about keeping it all unreviewed for months while you work on it, before creating a patch-bomb ticket on Sage's trac server. If you had worked incrementally this would have been settled ages ago, not at the last minute.

Again, note it says "deviate even more".  Sage-combinat does release early and often, but not to the main Sage branch, if I understand it right, because it's often not ready (?).  And the incremental approach may not have been appropriate in this case - though we don't want to spend eons debating the right underlying stuff, it's true.

[Nicolas M. Thiery]

Dear Volker,

Thanks much for your time on expressing here your opinion! Hopefully
this will help others have a synthetic view of the situation and bring
in their opinion so that we can reach quickly a consensus.

Just two micro-notes.

On Fri, Mar 07, 2014 at 05:25:12AM -0800, Volker Braun wrote:
> I also think we agree that the implementation where a recursive
> computation is done on startup is not ideal. Relatively simple
> implementation mistakes in new category/axioms end up with an infinite
> recursion on startup, which I think is rather unhelpful for developers.
> There is also no way to doctest this recursion beyond "sage manages to
> start up".

To be precise, the recursion error occurs on startup only if you
fiddle with those fundamental categories that are created upon Sage's
startup. And you can already run into a similar problem without this
ticket if you screw up your implementation of the super_categories
methods (I did a bunch of times!). Recursion errors are not unusual
when you apply an algorithm for acyclic graphs on a graph that
accidently contains a cycle :-)

But yes we certainly agree that one has to be a bit more careful
because of the deduction rules, and that there is room for improvement
in the error reporting.

My point being that you can infer this information from the code by
using the math:

P is a finite Cs. A Cs is a set. Therefore P is a finite set.

This indeed requires to learn, beyond standard Python, that Cs.Finite
implements the category of finite Cs, that super_categories specifies
the relation between Cs and Sets, and that Sage will do this kind of
deduction. But that's not a whole lot of documentation to digest
either.

Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Nicolas M. Thiery]

Hi Nathann, Karl,

On Fri, Mar 07, 2014 at 07:37:12AM -0800, kcrisman wrote:
> Again, note it says "deviate even more". Sage-combinat does release early
> and often, but not to the main Sage branch, if I understand it right,
> because it's often not ready (?).

Off topic, but since this was brought out :-)

Over the five years of its existence, the Sage-Combinat queue did
release a lot of stuff to the standard Sage library. Any volunteer for
doing some real stats? In the mean time, as a far fetched guess, I
would say a good 90% of what was developed there ended up in Sage. The
10% remaining is now either being worked on in the form of branches,
or rotting away waiting for someone to take it over, or not.

So long the queue, and thanks for the fish. It played its role to let
people collaborate together while we had a crappy Sage workflow. Now
it's much easier to informally form small teams to work on a project
by project basis. Also, as we grow up, people are progressively moving
out of the core and working on peripheral features, which means less
interactions and smaller teams.

[Nicolas M. Thiery]

On Fri, Mar 07, 2014 at 02:46:17PM +0000, Simon King wrote:
> I suppose this is some legacy. In the bad old times when we had no
> category framework, you would use the idiom
> if R.is_finite():
> do something
> and you could not do what you can do now, namely
> if R in FiniteSets():
> do something
> (and in some cases the new idiom is faster than the old).

More than legacy, the point is that the two idioms don't have the same
semantic:

- R in FiniteSets():
Does Sage already know that R is a finite set?

- R.is_finite():
Is R finite? Answering may require a lengthy calculation; A
NotImplementedError would be a valid "answer".

[Nils Bruin]

On Friday, March 7, 2014 8:33:50 AM UTC-8, Nicolas M. Thiéry wrote:

- R in FiniteSets():
  Does Sage already know that R is a finite set?

- R.is_finite():
  Is R finite? Answering may require a lengthy calculation; A
  NotImplementedError would be a valid "answer".

 

Shouldn't that be "Does Sage know that R is a finite set *by construction*?" If the "is_finite" result gets cached in the categories we'd violate immutability of parents, i.e.,

sage: R in FiniteSets()
False
sage: R.is_finite()
True
sage: R in FiniteSets()
True

would be bad. I know there are some cases where we've swallowed this badness in favour of some gains in efficiency elsewhere. I guess one could argue it's the category that gets mutated, not R, but I'm not sure that's entirely fair.
 

[Anne Schilling]

Hi Nicolas,

Could you please comment on Simon's point below?
I did not easily find this in the category primer. Where is it discussed there?

> One detail that I find fishy: It is possible that a category class (say,
> PermutationGroups, but it could be a different name, I don't recall)
> has an attribute (say, Finite) which is a lazily imported class (here:
> FinitePermutationGroups), but then the instance
> P = PermutationGroups()
> does *not* inherit the "Finite" attribute from its class! Instead, by
> some magic, is is replaced by a cached method, which then returns an
> instance of FinitePermutationGroups.
>
> That's a detail that I find (a) confusing, (b) likely to result in
> difficult-to-debug errors, (c) not needed, since in the end the cached
> method P.Finite() returns type(P).Finite() anyway.
>
> Nicolas' counter argument was (a) the cached method is documented, so
> that you can easily introspect the specification of the "Finite" axiom,
> (b) tests pass and people use it without problem, so let's not speculate
> on potential difficult-to-debug errors that have not shown up in the
> existing use cases yet, and (c) it is needed, because of the documentation
> (perhaps Nicolas gave further arguments, I am writing this without
> re-reading the comments on trac).

Also, Volker told me that the proof of concept that the syntax he prefers
can be implemented can be found here

http://trac.sagemath.org/15701

I think the consensus is that the feature that is implemented in 10963
is definitely worthwhile for Sage. So something that actually works with
full functionality to me at the moment seems more convincing than a proof
of concept.

Volker, how quickly can you get your way of doing things implemented
and tested in real life situations? Last time you said within a month,
but that was a over a month ago.

Best,

Anne

[Simon King]

Hi Nils,

On 2014-03-07, Nils Bruin <nbr...@sfu.ca> wrote:
> Shouldn't that be "Does Sage know that R is a finite set *by=20
> construction*?" If the "is_finite" result gets cached in the categories=20

> we'd violate immutability of parents, i.e.,

I think that that is not to happen.

Instead, if Sage knows by construction that R is finite, then
R in FiniteSets()
returns True, and moreover R inherits a method is_finite() that
immediately returns True (but it does not override a custom is_finite()
method that R might have inherited from somewhere else). Fine.

But if Sage does not know by construction that R is finite, then
R in FiniteSets()
currently does return False (and should do so in future), even if
R.is_finite()
returns True after a lengthy computation (that hopefully is cached).

> I know there are some cases where we've swallowed this=20

> badness in favour of some gains in efficiency elsewhere.

It seems that was the worst idea I ever had. Sorry.

Best regards,
Simon

[Florent Hivert]

Dear All,

I want to add my testimony about this as a user of the framework.

> What really bothers me with the ticket is the reliance on implicit magic
> over explicitly specifying relations between categories. From the ticket,
> this example:

> The theory here is that Sets defines a Finite axiom, and by some metaclass
> magic the inner class Cs.Finite ends up being a subcategory of Sets.Finite.

> [...]

> Nicolas thinks this amount of breverity is a desirable feature, and I think
> we should make Cs.Finite be explicit about which axiom it implements (and,
> therefore, not require that it is named Cs.Finite if you don't want
> to).

I completely second Nicolas on that point. There is a clash between Python's

"explicit is better that implicit"

and mathematical usage. I really think that what's allows us to do
mathematics, is *context*, that is implicit. Otherwise any mathematical proof,
reasoning or computation is just too verbose. I also think that this unique
point is what's makes writing mathematical software hard. Please think about
what's implicitly done for example when I write:

sage: K.<x> = PolynomialRing(PolynomialRing(QQ, 'y'), 'x')
sage: x + 1
x + 1

Hum what's the meaning of '1' here ?

Anyway, category (as its mathematical meaning) are part of this huge context
which comes with any mathematical computation.

Now going back to Sage, I really think that if we don't keep the syntax *as
short as possible* (as in math: `let's G1, G2 be a finite commutative group
and Phi be a morphism from G1 to G2`) then this will be much to verbose to be
programmed with and each user will start to makes it's own shortcut which is
much worse.

I should also note that each other CAS which developed such a similar thing
(eg. Magma, Axiom, MuPAD) relies on the Compiler/Interpreter -- that is
language magic -- to implement the feature. I think its good to be in Python
which is not a specific build language for a CAS but which should also add the
necessary magic to Python to be able to do maths easily without the burden of
syntax. As soon as sufficiently developers know the inside magic, my
experience tells me that the user doesn't care.

> We've explored various ways to implement this syntax, either through
> inheritance or a class attribute, but Nicolas doesn't like any. Nicolas
> keeps saying that we can change the syntax later, but I think that should be
> done right or not at all on this ticket (one can easily implement the axiom
> functionality without introducing the subclass syntax).

The only drawback I personally see with this subclass syntax is the documentation.

So I'm really tempted to vote for inclusion right now. I think people *other
than the one involved in the ticket* should express themselves before we start
counting votes.

Cheers,

Florent

[Volker Braun]

I don't understand how you your comment relates to the question. Of course I agree that the intended result (coercion in your example, or systematically provide methods for parents/elements on this ticket) should work effortlessly. The question is, how should the library implementation look like? We could just run the preparser over all Sage library files, but good reasons we do not do that. This makes the library code sometimes more cumbersome to write, for example 3/5 is 0 if you write it in the Sage library and not if you write it on the command line. IMHO the Sage library ought to be good (and readable) Python code first and foremost. We do (and should) provide domain-specific extensions for interactive use, but not for writing the Sage library itself.

[Simon King]

Hi Florent,

On 2014-03-08, Florent Hivert <Florent...@lri.fr> wrote:
>> The theory here is that Sets defines a Finite axiom, and by some metaclass
>> magic the inner class Cs.Finite ends up being a subcategory of Sets.Finite.
>> [...]
>> Nicolas thinks this amount of breverity is a desirable feature, and I think
>> we should make Cs.Finite be explicit about which axiom it implements (and,
>> therefore, not require that it is named Cs.Finite if you don't want
>> to).
>
> I completely second Nicolas on that point. There is a clash between Python's
>
> "explicit is better that implicit"
>

> and mathematical usage. ...

> Now going back to Sage, I really think that if we don't keep the syntax *as
> short as possible* (as in math: `let's G1, G2 be a finite commutative group
> and Phi be a morphism from G1 to G2`) then this will be much to verbose to be
> programmed with and each user will start to makes it's own shortcut which is
> much worse.

I am not sure if Volker and you talk about the same meaning of "syntax".

It seems to me that you take the position of a user, who has a category
Cs() and wants to get the category of finite cs. Then, it is convenient
if one can just do Cs().Finite(). This syntax is clear and simple.

And I suppose that Volker agrees with you here. FWIW, *I* do.

However, it seems to me that Volker was arguing from the position of a
developer. Again, the nested class syntax seems nice and concise to me:

sage: class Cs(Category):
....: def super_categories(self):
....: return [Sets()]
....: class Finite(CategoryWithAxiom):
....: class ParentMethods:
....: def foo(self):
....: print "I am a method on finite C's"

But, as Volker has pointed out, the objects O of Cs().Finite() also inherit
from the parent class of Sets().Finite(). There is no surprise from a
mathematical point of view: O is a c and O is finite, every c is a set, the
property "finite" applies to sets, thus O is a finite set.

But is the mathematical syllogism really stated in the code in a
sufficient level of clarity? Volker, do I understand correctly that
this is your concern?

My opinion on it, from a user perspective:
- Yes, it is surprising for a novice user to see that O.is_finite()
is available. If O.__class__ is a Cython extension class, you wouldn't
even find an is_finite() method in O.__class__.mro(). But: This criticism
also applies to the *existing* category framework. I don't think that
the surprise will be worse with #10963. And as soon as one has
understood what the category framework is about, the existence of
O.is_finite() is *not* surprising.
- A couple of years back, I did not like the category framework, since
it was very difficult to find the source code of stuff. Now,
introspection has improved, and `O.is_finite??` will show you the
code. And with the possibility to easily find the source code, I
started to find the category framework much easier to understand.

My opinion, from a developer perspective:
- I wouldn't implement a statistical tests, since I don't know about
statistics. So, I suppose a developer who wants to implement a
category knows about categories.
- The code above clearly states that Cs() is a sub-category of Sets().
Thus, knowing about categories, the developer should not be surprised
that applying the axiom "Finite" to Sets() is relevant for Cs().Finite().

However, something that I have mentioned earlier (and hasn't been
answered yet) is the fact that sometimes Cs().Finite is not the class
Cs.Finite, but is a cached method that is gotten by some black magic
that I find not transparent, and (I think) for virtually no benefit.

Nicolas, is documentation the only reason to make Cs().Finite be a
cached method overriding Cs.Finite? Then I think we should find a
different way to make Cs().Finite/Cs.Finite point the user to the
specifications of the "Finite" axiom. Why not have a class method
specification() for a CategoryWithAxiom C, that will print the
documentation of the defining axiom of C (taken from where this
axiom is defined)?

Note that this documentation/specification problem could be solved by
making axioms proper objects (that know about their specification!),
rather than just strings. IIRC, Volker has provided a draft
implementation.

Best regards,
Simon

[Volker Braun]

On Saturday, March 8, 2014 2:48:03 AM UTC, Anne Schilling wrote:

Volker, how quickly can you get your way of doing things implemented
and tested in real life situations?

I still think that it would be relatively easy. Even faster would be to not introduce the subclass syntax on the ticket and leave that for later. But so far Nicolas  never liked any syntax that I proposed nor made any suggestions for how a more explicit syntax ought to look like. 

[Nicolas M. Thiery]

On Fri, Mar 07, 2014 at 09:48:11AM -0800, Nils Bruin wrote:
> Shouldn't that be "Does Sage know that R is a finite set *by
> construction*?" If the "is_finite" result gets cached in the categories
> we'd violate immutability of parents, i.e.,
>
> sage: R in FiniteSets()
> False
> sage: R.is_finite()
> True
> sage: R in FiniteSets()
> True
>
> would be bad. I know there are some cases where we've swallowed this
> badness in favour of some gains in efficiency elsewhere. I guess one could
> argue it's the category that gets mutated, not R, but I'm not sure that's
> entirely fair.

Changing on the fly the category of a parent to a subcategory with
more structure (e.g. more operations) would certainly be bad
w.r.t. immutability; it's really creating a different parent.

In the above case however, it's about changing the category to a full
subcategory, which means that no structure is added: you just have
learned more information. In principle, I believe this would be a
feature. GAP does it a lot for example. There are some technicalities
though (upgrading the class of the parent *and* of all its elements)
which means that I am not sure how to do this robustly yet. So yes,
for now, I would be fine with your formulation.

Cheers,
Nicolas

PS: I changed the subject of this e-mail, since we are deviating topic

[Nicolas M. Thiery]

On Sat, Mar 08, 2014 at 12:24:13PM +0000, Simon King wrote:
> I am not sure if Volker and you talk about the same meaning of "syntax".

I assume Florent's point can be summarized as follow: in the context
of Sage, and in particular of categories, the general Python rule
"explicit is better than implicit" is to be taken with a grain of
salt. Basically, "explicit" should be interpreted as "explicit from
the code or from the math context".

Cheers,
Nicolas

[Nicolas M. Thiery]

On Sat, Mar 08, 2014 at 04:13:59AM -0800, Volker Braun wrote:
> I don't understand how you your comment relates to the question. Of course
> I agree that the intended result (coercion in your example, or
> systematically provide methods for parents/elements on this ticket) should
> work effortlessly. The question is, how should the library implementation
> look like? We could just run the preparser over all Sage library files,
> but good reasons we do not do that. This makes the library code sometimes
> more cumbersome to write, for example 3/5 is 0 if you write it in the Sage
> library and not if you write it on the command line. IMHO the Sage library
> ought to be good (and readable) Python code first and foremost. We do (and
> should) provide domain-specific extensions for interactive use, but not
> for writing the Sage library itself.

I certainly agree, and I bet Florent as well, that we don't want
preparsing in the Sage library. It's more about what the right balance
is between the usual principles (explicit vs implicit, DRY, single
point of truth) to achieve the most readable code.

[Nathann Cohen]

> I assume Florent's point can be summarized as follow: in the context
> of Sage, and in particular of categories, the general Python rule
> "explicit is better than implicit" is to be taken with a grain of
> salt. Basically, "explicit" should be interpreted as "explicit from
> the code or from the math context".

Tricks like this one makes it hard for people to work with category
code, and in particular to learn how it works by looking at the code.
Somebody said previously on this thread that "if somebody is playing
with categories, then he must have read the manual first" and I
believe that this is already wrong. Sometimes, you (= I ) have to deal
with category code, even though you do not know what it is, just
because you encounter a bug or because some part of the code forces
you to do something with categories (facades in Posets, for instance
?). That's why you have standards like that, to make sure that
everybody can coexist and work together.

To me it's a bit like for global variables. It's correct code, it
compiles, but it is hard to guess the source of the problem when you
are fighting with a bug. Even if it is documented somewhere, as you
have no idea what you are looking for exactly. Making the link
explicit is exactly the hint you need to tell you where you should go
in order to understand what happens.

Nathann

[Andrew]

On Friday, 7 March 2014 23:14:44 UTC+11, Nicolas M. Thiéry wrote:

        Dear Sage developers,

This is a call for vote about the ticket:

     #10963: axioms and more functorial constructions [1]

 

Dear Nicolas,

I think that it is good that this is being done (thank you!) but, like some others, I'm not fond of the use of implicit code and black magic whenever this makes it hard to track down the real source of some of the code. If everything is well documented and explained then this is probably OK, although I feel that one of the problems with python/sage documentation is that we are encouraged to write good documentation for methods but there is little emphasis of writing a self-contained general overview of what is going on -- I've even had reviewers take this out when I have tried to write it.

To take the category framework as an example, I have fought with it and won on several occasions but it is always been a painful experience because I find that the documentation is not very helpful. Of course, perhaps it is just me but I find that most of the example code in the documentation is artificial and treats only simplified situations and that many little "secrets" are not documented at all. To be fair good documentation is hard to write, especially when you have a complex piece of code. Even so, in my experience there is quite a lot of undocumented magic in the category framework.

I hope that the documentation for the functorial constructions patch is more helpful.

Andrew

[Volker Braun]

"Explicit from the context" is an oxymoron. If you can only be figure it out from the context, then it is by definition implicit. Only something that is written down in Python code is explicit in Python. I guess that is a bit of an English language issue. 

Or, as the old joke goes, "It depends on what the meaning of the words 'is' is."

[Simon King]

Hi Volker,

On 2014-03-09, Volker Braun <vbrau...@gmail.com> wrote:
> "Explicit from the context" is an oxymoron. If you can only be figure it=20
> out from the context, then it is by definition implicit. Only something=20

> that is written down in Python code is explicit in Python.

But what other syntax do you suggest?

Doing
class Cs(Category):
def super_categories(self):
return [Sets()]
*explicitly* states that Cs is a sub-category of the category of sets.

Additionally doing
class Cs(Category):
class Finite(CategoryWithAxiom):
class ParentClass:
def some_method(self):
return "I am a finite c"
is also (I think) *sufficiently* explicit about Cs.Finite() being a
sub-category of Cs(), defined by the axiom Finite.

So, Cs.Finite() is a sub-category of Sets(). Do you think that an
additional explicit statement is needed to tell that Cs.Finite()
additionally is a sub-category of Sets.Finite()?

I believe that the rule implemented in #10963
"Cs().SomeAxiom() is a sub-category of SCs().SomeAxiom() for all
super-categories SCs() of Cs() to which SomeAxiom can be applied"
is clear enough.

What I am really not happy about is that in some cases (or in all? If
not in all, then in what cases?) Cs.Finite is overridden in Cs() by a
cached method.

Nicolas hasn't answered yet if documentation respectively semantic
specification of the "Finite" axiom is the only reason for this highly
intransparent trick. If it is, then I think it would be better to turn
axioms into objects (currently they are just names) carrying their own
documentation and specification.

Best regards,
Simon

[Volker Braun]

On Sunday, March 9, 2014 2:16:51 PM UTC, Simon King wrote:

    class Cs(Category):
        class Finite(CategoryWithAxiom):
            class ParentClass:
                def some_method(self):
                    return "I am a finite c"
is also (I think) *sufficiently* explicit about Cs.Finite() being a
sub-category of Cs(), defined by the axiom Finite.

This snippet defines a finite axiom that is different from the finite axiom for sets, and your parents will not have a "is_finite()" method. Only once you join finite Cs with (not necessarily finite) Sets then suddenly you also inherit "is_finite()". Depending on your viewpoint this is an awesome feature or devastating if you want new developers to use the category framework.

 

What I am really not happy about is that in some cases (or in all? If
not in all, then in what cases?) Cs.Finite is overridden in Cs() by a
cached method.

Really, the whole subclass-as-axiom magic is just an attempt at syntactic sugar. The only thing that matters is that Cs().Finite() returns an instance of the the axiom. There are various functionally equivalent ways to implement it. There is yet another syntax to achieve the same end, namely Cs().with_axiom("Finite"). Except for the strings I would be much happier with that since it doesn't require an open-ended list of magic attribute names for axioms.

 

[Simon King]

Hi Volker,

On 2014-03-09, Volker Braun <vbrau...@gmail.com> wrote:

> ------=_Part_25_1398588.1394377030563
> Content-Type: text/plain; charset=UTF-8

>
> On Sunday, March 9, 2014 2:16:51 PM UTC, Simon King wrote:
>>
>> class Cs(Category):
>> class Finite(CategoryWithAxiom):
>> class ParentClass:
>> def some_method(self):
>> return "I am a finite c"
>> is also (I think) *sufficiently* explicit about Cs.Finite() being a
>> sub-category of Cs(), defined by the axiom Finite.
>>
>
> This snippet defines a finite axiom that is different from the finite axiom
> for sets, and your parents will not have a "is_finite()" method.

The snippet was supposed to be a continuation of the first snipped,
which stated that Cs() is a sub-category of Sets(). Sorry for not being
clear.

> Really, the whole subclass-as-axiom magic is just an attempt at syntactic
> sugar. The only thing that matters is that Cs().Finite() returns an
> instance of the the axiom. There are various functionally equivalent ways
> to implement it. There is yet another syntax to achieve the same end,
> namely Cs().with_axiom("Finite"). Except for the strings I would be much
> happier with that since it doesn't require an open-ended list of magic
> attribute names for axioms.

OK, that is the syntax that you want for *using* axioms: Not Cs.Finite() or
Cs().Finite(), but Cs.with_axiom("Finite") or even better
Cs.with_axiom(axioms.Finite). Fair enough.

But your answer is orthogonal to what I was asking you: What syntax to you
suggest for *implementing* axioms? Do you think stating that Cs() is a
subcategory of Sets() is not good enough for having Cs().Finite() being
a sub-category of Sets().Finite()? What else is there to code, in your
opinion?

Best regards,
Simon

[Nicolas M. Thiery]

On Sun, Mar 09, 2014 at 02:16:51PM +0000, Simon King wrote:
> What I am really not happy about is that in some cases (or in all? If
> not in all, then in what cases?) Cs.Finite is overridden in Cs() by a
> cached method.

It is done uniformly. The semantic of Cs().Finite() is always to
return the subcategory the finite objects of Cs(). Accordingly,
Cs().Finite is always the method that calls the machinery to construct
this category.

> Nicolas hasn't answered yet if documentation respectively semantic
> specification of the "Finite" axiom is the only reason for this
> highly intransparent trick.

Technically we could definitely do without it. I believe it's a nice
feature for the user to always be referred to that method, and to hide
the implementation detail of whether Finite is actually implemented in
Cs. Now if you have a strong opinion against it, we could remove it.

Just, you will need to choose and implement some trick so that
introspection on Cs().Finite bring up the appropriate documentation.

[Nicolas M. Thiery]

On Sun, Mar 09, 2014 at 05:50:29AM -0700, Andrew wrote:
> I think that it is good that this is being done (thank you!) but, like
> some others, I'm not fond of the use of implicit code and black magic
> whenever this makes it hard to track down the real source of some of the
> code.

I belong to the "some others":-) I am the first one to be struck when
this happens!

The question is whether this ticket makes things worse in
practice. After having used it quite some, I believe not, especially
since I tweaked things to this end. But I'd love to have more feedback
arising from concrete use.

> If everything is well documented and explained then this is probably
> OK, although I feel that one of the problems with python/sage
> documentation is that we are encouraged to write good documentation for
> methods but there is little emphasis of writing a self-contained general
> overview of what is going on -- I've even had reviewers take this out when
> I have tried to write it.

I agree: we need more overview documents in Sage!

> To take the category framework as an example, I have fought with it and
> won on several occasions but it is always been a painful experience
> because I find that the documentation is not very helpful. Of course,
> perhaps it is just me but I find that most of the example code in the
> documentation is artificial and treats only simplified situations and that
> many little "secrets" are not documented at all.
> To be fair good documentation is hard to write, especially when
> you have a complex piece of code. Even so, in my experience there
> is quite a lot of undocumented magic in the category framework.

Concrete examples would be helpful so that we can see if it was
actually due to "black magic in the infrastructure" as is discussed
here. Or "just" a problem of navigating the large body of generic code
implemented in the categories; problem that we would have as well with
plain abstract classes. Of course in both cases we should seek to
improve things! In particular, each areas (modules, algebras, groups,
...) should have overview documents.

> I hope that the documentation for the functorial constructions patch is
> more helpful.

I put a lot of work into that :-) It would be helpful if you could
read the updated primer [1] and the overview documentation about
axioms [2] and report!

Cheers,
Nicolas

[1] http://sage.math.washington.edu/home/nthiery/sage-6.0/src/doc/output/html/en/reference/categories/sage/categories/primer.html
[2] http://sage.math.washington.edu/home/nthiery/sage-6.0/src/doc/output/html/en/reference/categories/sage/categories/category_with_axiom.html

[Simon King]

Hi Nicolas,

On 2014-03-10, Nicolas M. Thiery <Nicolas...@u-psud.fr> wrote:
>> Nicolas hasn't answered yet if documentation respectively semantic
>> specification of the "Finite" axiom is the only reason for this
>> highly intransparent trick.
>
> Technically we could definitely do without it. I believe it's a nice
> feature for the user to always be referred to that method, and to hide
> the implementation detail of whether Finite is actually implemented in
> Cs.

That's the point. Why should one hide the implementation? If I do "??"
then I *want* to see code. Hiding code is the root of all evil, as it
results in confusion.

> Just, you will need to choose and implement some trick so that
> introspection on Cs().Finite bring up the appropriate documentation.

We speak about *two* different types of documentation:
1. Documentation of Cs().Finite
2. Documentation/specification of the axiom "Finite", also pointing to
the place where it is defined.

Note that this seems to be in accordance to your own terminology: You
distinguish between "a category defines an axiom" and "a category
implements an axiom".

Let C = Cs(). I'd like to be able to easily access both the implementation
and the specification.

And I'd be confused if C.Finite?? would show me the specification
(thus, Sets.Finite), even though Cs *implements* "Finite".

Again, making axioms stand-alone objects (rather than strings) could
provide a satisfying answer:
- C.Finite? documents the implementation
- C.Finite.axiom? documents the specification, as in Sets.Finite.

Cheers,
Simon

[Nicolas M. Thiery]

On Sat, Mar 08, 2014 at 10:43:46PM +0100, Nathann Cohen wrote:
> Tricks like this one makes it hard for people to work with category
> code, and in particular to learn how it works by looking at the code.
> Somebody said previously on this thread that "if somebody is playing
> with categories, then he must have read the manual first" and I
> believe that this is already wrong. Sometimes, you (= I ) have to deal
> with category code, even though you do not know what it is, just
> because you encounter a bug or because some part of the code forces
> you to do something with categories (facades in Posets, for instance
> ?). That's why you have standards like that, to make sure that
> everybody can coexist and work together.

Yup. But on the other hand making everything completely explicit as in
a plain hierarchy of abstract classes would require much
duplication. Actually a potentially exponential growth of
duplication. That would not make the code any easier to read. So it's
all about finding the right balance. All I am saying is that, thanks
from the strong semantic provided by math, the right balance is
shifted a bit compared to more mainstream code.

> To me it's a bit like for global variables. It's correct code, it
> compiles, but it is hard to guess the source of the problem when you
> are fighting with a bug. Even if it is documented somewhere, as you
> have no idea what you are looking for exactly. Making the link
> explicit is exactly the hint you need to tell you where you should go
> in order to understand what happens.

One difference being that each global variable needs to be documented,
whereas here it's a one time shot to learn the meaning of the syntax.

[Nicolas M. Thiery]

On Sun, Mar 09, 2014 at 05:55:24AM -0700, Volker Braun wrote:
> "Explicit from the context" is an oxymoron.

Sure, replace "Explicit from the code or the math context" to
"Explicit from the code or the math" if you prefer. It does not change
my point.

Best,

[Volker Braun]

On Monday, March 10, 2014 12:27:00 AM UTC, Simon King wrote:

But your answer is orthogonal to what I was asking you: What syntax to you
suggest for *implementing* axioms? Do you think stating that Cs() is a
subcategory of Sets() is not good enough for having Cs().Finite() being
a sub-category of Sets().Finite()? What else is there to code, in your
opinion?

IMHO the contained-class makes it clear what the base category is, but not what the axiom is. For example:

class Cs(Category): 

    class Finite(CategoryWithAxiom): 

        pass

class Ds(Category): 

    class Endlich(CategoryWithAxiom): 

        pass

Now Cs.Finite implements an axiom, but Ds.Endlich does not (since you haven't added the "Endlich" string to a global list somewhere). If you had to specify the axiom then not only wouldn't you be able to make that mistake, you'd also know exactly what axiom is being implemented. Some possibilities that we already went over on the ticket are

class Ds(Category): 

    class Endlich(InnerCategory, axioms.Finite): 

        pass

class Ds(Category): 

    class Endlich(InnerCategory): 

        __axiom_classes__ = [axioms.Finite]

class Ds(Category): 

    class Endlich(axioms.Finite.Category): 

        pass

[Volker Braun]

On Monday, March 10, 2014 11:30:32 AM UTC, Nicolas M. Thiéry wrote:

Sure, replace "Explicit from the code or the math context" to
"Explicit from the code or the math" if you prefer. It does not change
my point.

"Explicit on a piece of paper in the third drawer of my desk"

Its either explicit (=in the code) or implicit (=not in the code).

[Nicolas M. Thiery]

On Fri, Mar 07, 2014 at 04:32:52AM -0800, Nathann Cohen wrote:
> This ticket, like every ticket on our trac server, will be merged when the
> reviewers will be happy with it. Period.
> Respect that. Don't attempt to short-cut the review process with polls.

Certainly. I am not trying to "vote out a reviewer". The discussion
was at a stalemate due to diverging opinions on two points (a
technical point; and a point about how to proceed). I am therefore
bringing it on sage-devel to get *more* opinions so that we can
finally get to a consensus.

This already lead to some progress since this has been the occasion
for laying out the positions and divergence points, and have a serene
discussion about it. And the discussion is serene because it's about
opinions on facts, not on people.

> > Let's not deviate even more from incremental development and the usual
> > "release early, release often".
> Is that a f*** joke ? Your ticket was created three years ago, and the
> branch is thousands of lines long. Sage-combinat is all about not
> releasing early and often. It's about keeping it all unreviewed for months
> while you work on it, before creating a patch-bomb ticket on Sage's trac
> server. If you had worked incrementally this would have been settled ages
> ago, not at the last minute.

I already broke it down and got merged a bunch of stuff that could go
in earlier. Just look at the dependency list: #11224, #8327, #10193,
#12895, #14516, #14722, #13589, #14471, #15069, #15094, #11688,
#13394, #15150, #15506, #15757, #15759 (not all are mine). The list
does not even show dependencies like #11935 or #715 by Simon, which
were developed outside of the Sage-Combinat queue, and took a *long*
time to fix because they were *hard*.

The axiom infrastructure by itself is not splitable, and could not
have gone in much earlier due to the dependencies. What makes the
ticket big is the refactoring of the library using the axiom
infrastructure. I believe it's helpful to review it at the same time
as the infrastructure anyway because it illustrates how it works at a
large scale. You don't have to agree with that, but it could not have
been released earlier anyway.

The only stuff that could potentially have been split off was about
functorial constructions (maybe 20% of the patch?), and this is what I
take the blame for.

[Nicolas M. Thiery]

On Mon, Mar 10, 2014 at 11:15:22AM +0000, Simon King wrote:
> That's the point. Why should one hide the implementation? If I do "??"
> then I *want* to see code. Hiding code is the root of all evil, as it
> results in confusion.

If you want to look at the implementation of the category of finite
Cs, do as usual:

sage: Ds = Cs().Finite()
sage: Ds??

- Cs().Finite? is about what the "Finite" operation does (constructing
the subcategory of finite Cs).

- Cs().Finite()? is about the end result.

Note by the way that you really want to use the second idiom to get
the implementation of the category. In the following example:

sage: Magmas().Unital().Associative()
Category of monoids

you really have to execute the method to know what the end result is,
since there is no Magmas.Unital.Associative class.


The above ways to access the documentation sounds consistent with what
we do for other constructions. For example when you do:

sage: R.fraction_field?

you get the documentation of the fraction_field method, not of the
class implementing the fraction field.

I guess this answers the rest of the e-mail, but please let me know if
not!

[Nicolas M. Thiery]

On Mon, Mar 10, 2014 at 04:39:08AM -0700, Volker Braun wrote:

> On Monday, March 10, 2014 11:30:32 AM UTC, Nicolas M. ThiA(c)ry wrote:
>
> Sure, replace "Explicit from the code or the math context" to
> "Explicit from the code or the math" if you prefer. It does not change
> my point.
>
> "Explicit on a piece of paper in the third drawer of my desk"
> Its either explicit (=in the code) or implicit (=not in the code).

You are right, "the code" was not explicit enough, since everything I
spoke about is in the code :-) Let me rephrase it as:

"Explicit from the syntax or from the semantic".

E.g. If I use the name "Associative", I assume the reader will have
some idea what it is about, especially within a the math context given
by the name of the containing category.

[Nicolas M. Thiery]

On Mon, Mar 10, 2014 at 04:35:25AM -0700, Volker Braun wrote:
> IMHO the contained-class makes it clear what the base category is, but not
> what the axiom is. For example:
> class Cs(Category):
> class Finite(CategoryWithAxiom):
> pass
> class Ds(Category):
> class Endlich(CategoryWithAxiom):
> pass
> Now Cs.Finite implements an axiom, but Ds.Endlich does not (since you
> haven't added the "Endlich" string to a global list somewhere).

No. What makes Endlich an axiom (or not) is that some super category
has defined it (or not) [1].

As an analogy, when an abstract class C declares a method foo, and you
implement this method in some subclass D, you don't write explicitly
in D.foo that you override the method foo declared in C.

If you need that information, you either use the semantic (here the
math), or you walk the hierarchy. Of course, since the hierarchy is
built dynamically here, you need to walk the hierarchy dynamically;
introspection does that nicely for you (yes, unless Sage does not
start at all; but in that case we are speaking of the very basic
axioms and you can still use the math).

Cheers,
Nicolas

[1] The fact that Sage needs a list of the names of all the axioms
being defined is an implementation detail. Recall that it is used
mostly for printing purposes (and allowing for syntactical sugar in
some spots). We could get rid of it without changing the syntax of how
axioms are defined and implemented.

[Nathann Cohen]

> Certainly. I am not trying to "vote out a reviewer".

From your original post:

===
Call for vote
-------------

[ ] Merge this ticket, and move on with further improvements in follow
    up tickets
[ ] This ticket should be held up until everybody is happy
[ ] This ticket does not make any sense
===

I don't see how I this can be interpreted differently from trying to get a ticket in through a poll despite the reviewer's remarks. Really, I don't. I claim that this is the only sensible explanation.

> The discussion
> was at a stalemate due to diverging opinions on two points (a
> technical point; and a point about how to proceed). I am therefore
> bringing it on sage-devel to get *more* opinions so that we can
> finally get to a consensus.

Still, really. I can re-read your poll a thousand times, and all I see is "he does not want to change his branch despite the reviewer's remark, so he tries to force it with a poll".

> This already lead to some progress since this has been the occasion
> for laying out the positions and divergence points, and have a serene
> discussion about it. And the discussion is serene because it's about
> opinions on facts, not on people.

The review is indeed a perfect place to discuss design problems. Especially when you find reviewers who actually care, which is definitely the case on this ticket. You must be ready to change the design, though. And not use the dependencies as an excuse to "fix it later".

> I already broke it down and got merged a bunch of stuff that could go
> in earlier. Just look at the dependency list: #11224, #8327, #10193,
> #12895, #14516, #14722, #13589, #14471, #15069, #15094, #11688,
> #13394, #15150, #15506, #15757, #15759 (not all are mine). The list
> does not even show dependencies like #11935 or #715 by Simon, which
> were developed outside of the Sage-Combinat queue, and took a *long*
> time to fix because they were *hard*.

Well, I regret that all those persons had to base their own work on a patch that it not in yet. Even though I  hope to steer away as much as possible from category code in the future, it is quite clear from everybody's messages that they find it useful. If it is that fundamental, please don't include a piece of code that you  think is not good enough or will have to be changed.

The stuff that "is to be implemented later" often does not get implemented at all.

> The axiom infrastructure by itself is not splitable, and could not
> have gone in much earlier due to the dependencies. What makes the
> ticket big is the refactoring of the library using the axiom
> infrastructure. I believe it's helpful to review it at the same time
> as the infrastructure anyway because it illustrates how it works at a
> large scale. You don't have to agree with that, but it could not have
> been released earlier anyway.

I just want to make sure that the code is clean and understandable for everybody (even persons to which you did not explain it in person), because I do not want to *need* your or Florent's help when I encounter a bug (like #14019 or the three broken cartesian product functions I reported some time ago). If only elected people can play with this code, it may as well be proprietary software.

My only protection against that is the review process: sensible people who spend a lot of time trying to understand the code and make it clear for everybody else. If you skip it, I've got none.

Nathann

[Volker Braun]

On Monday, March 10, 2014 12:48:17 PM UTC, Nicolas M. Thiéry wrote:

No. What makes Endlich an axiom (or not) is that some super category
has defined it (or not) [1].

[1] The fact that Sage needs a list of the names of all the axioms

being defined is an implementation detail

And this whole thread is about the implementation details. But fine, let's say "Endlich" is how a novice might attempt to define an axiom. Only to end up with mysterious printing errors.

[Volker Braun]

On Monday, March 10, 2014 12:25:17 PM UTC, Nicolas M. Thiéry wrote:

E.g. If I use the name "Associative", I assume the reader will have
some idea what it is about

You are of course free to assume that. Even if that assumption is correct, it is still implicit in the source code. 

[Nicolas M. Thiery]

On Mon, Mar 10, 2014 at 06:19:11AM -0700, Volker Braun wrote:

> On Monday, March 10, 2014 12:48:17 PM UTC, Nicolas M. ThiA(c)ry wrote:
>
> No. What makes Endlich an axiom (or not) is that some super category
> has defined it (or not) [1].
>
> [1] The fact that Sage needs a list of the names of all the axioms
> being defined is an implementation detail
>
> And this whole thread is about the implementation details. But fine, let's
> say "Endlich" is how a novice might attempt to define an axiom. Only to
> end up with mysterious printing errors.

No mysterious printing error. This is immediately detected and reported:

class Cs(Category):
def super_categories(self):
return [Sets()]

class Endlich(CategoryWithAxiom):
pass

sage: Cs()
Category of cs
sage: Cs().Endlich()
...
ValueError: could not infer axiom for the nested class <class '__main__.Cs.Endlich'> of <class '__main__.Cs'>

sage: sage.categories.category_with_axiom.all_axioms+=("Endlich",)
sage: Cs().Endlich()
...
AttributeError: 'super' object has no attribute 'Endlich'

I am happy to change those error messages for something more helpful;
What do you suggest?

- ``Unknown axiom name 'Endlich'. Please add it to sage.categories.category_with_axiom.all_axioms''
- ``Axiom 'Endlich' not defined by Categories of Cs or its super categories ''

By the way, all_axioms would better be renamed all_axiom_names; agreed?

Best,
Nicolas

[Niles Johnson]

I would prefer not to use "mathematical context" or "semantic" when trying to read code (for debugging, or for learning).  My reason for this is that I don't believe the computer "knows" mathematics, so I wouldn't use my mathematical knowledge to try to guess what the code is doing.  Using mathematical or other semantic information is useful to guess what developers were thinking when they wrote the code.  But (especially when debugging) I don't assume that every developer knows what I know.  Therefore I will want a more reliable way to determine where/how methods are implemented.

[Nicolas M. Thiery]

Hmm, as is, the argument is a bit general, and I believe it would rule
out the very principle underlying categories :-) So let's make this
more concrete. Here is the kind of knowledge we are speaking about:

- a finite field is a ring, but also a finite set, etc.
- a unital magma that is associative is a unital semigroup, that is a monoid
- a finite division ring is a field

Sage "knows" about those, either because the information is directly
hardcoded (e.g. a unital semigroup is a monoid, a finite division ring
is a field), or deduced iteratively from such. The role of categories
is to make the code manageable by using more deductions to reduce the
amount of information you need to hardcode. Similar strategies have
been successfully used in one form or the other in GAP, Magma, Axiom,
MuPAD, and since we are getting bigger than those, we are exploring
further.

As an analogy, in plain Python, when one creates a class one does not
hard code all the super classes. Only the direct ones. And as in plain
Python, the easiest way to recover where/how a method is implemented
is to use introspection (rather than exploring the hierarchy by
looking at the code).

This being said, where would you put the threshold between appropriate
and overboard use of "math knowledge"?

Cheers,
Nicolas

[Nathann Cohen]

> This being said, where would you put the threshold between appropriate
> and overboard use of "math knowledge"?

When somebody who does not understand the math is unable to understand
how the code works, then it went overboard ?

Nathann

[Nicolas M. Thiery]

Hi Volker,

On Sat, Mar 08, 2014 at 04:59:16AM -0800, Volker Braun wrote:
> I still think that it would be relatively easy. Even faster would be to
> not introduce the subclass syntax on the ticket and leave that for later.
> But so far Nicolas never liked any syntax that I proposed nor made any
> suggestions for how a more explicit syntax ought to look like.

I reckon that, after trying a lot of syntax variations and writing
quite some code with it I have developed a strong opinion about the
current syntax. I gladly implemented quite a few suggestions from the
reviewers. But there are some changes I am really reluctant to do
against my intuition.

I understand it's frustrating from a reviewer's perspective. And I
respect your opinion too. So if you or someone else implements the
alternative syntax, and battlefield tests it on the whole sage
categories so that we can all forge an informed opinion, and if at the
end there is a consensus that this is better, I won't veto it even if
still goes against my opinion by then.

But if you also have a very strong opinion that the current syntax is
not good enough to be merged as a preliminary step, please do your
changes shortly.

As for leaving the subclass syntax for later, do you mean:

- Temporarily rewriting the whole patch without that syntax, to
possibly undo it later? This would sound like a lot of work, most
likely (I believe) for little value.

- Delaying the updating of the Sage library to use axioms to a second
step? The tickets that depend on #10963 need this updating, so this
would not improve much the situation. And in the mean time we would
not get the battlefield test which I believe is very important.

[Nicolas M. Thiery]

Fair enough :-) But this remains rather vague.

Would someone have a concrete use case where a piece of the current
code which is explicit enough becomes not explicit enough with #10963?

[Nathann Cohen]

> Fair enough :-) But this remains rather vague.

Not really. I think Volker's example from before qualifies.

Nathann

[Nicolas M. Thiery]

On Mon, Mar 10, 2014 at 02:00:59PM +0100, Nathann Cohen wrote:
> Still, really. I can re-read your poll a thousand times, and all I see is
> "he does not want to change his branch despite the reviewer's remark,
> so he tries to force it with a poll".

Forcing? Hmm. To force anything, I would need to have coercive power
on the community, which I certainly don't. You can see from the rest
of the post that I tried very hard to *not* put a bias in the
outcome. And if you want to know the details, I did get in touch
personally with a couple developers, asking them to "read the post,
forge your own opinion and vote accordingly; or don't vote!".

A very good measure of my lack of coercive power, and in general the
resilience of our community to pressures of all sorts, is that many
people expressed opinions, but no one voted. That's fine! Actually I
expected that: I trust people in our community to only vote if they
have a strong opinion. And it's very hard to forge oneself a strong
opinion in such a situation.

A couple other notes:

- I gladly made changes when I was convinced by them.

- It's not a change I think should be done and am just lazy to
implement. It's a change I dislike (sorry to have opinions)!

> > I already broke it down and got merged a bunch of stuff that could go
> > in earlier. Just look at the dependency list: #11224, #8327, #10193,
> > #12895, #14516, #14722, #13589, #14471, #15069, #15094, #11688,
> > #13394, #15150, #15506, #15757, #15759 (not all are mine). The list
> > does not even show dependencies like #11935 or #715 by Simon, which
> > were developed outside of the Sage-Combinat queue, and took a *long*
> > time to fix because they were *hard*.
>
> Well, I regret that all those persons had to base their own work on a
> patch that it not in yet.

The list above is about tickets #10963 depended on. Not the tickets
that depend on it. I was arguing that I had indeed been doing
iterative development.

> If it is that fundamental, please don't include a piece of code

> that you A think is not good enough or will have to be changed.

For whatever it's worth, *I* think it's good enough. And don't worry
too much, it's been one of the most reviewed patch around.

Cheers,

[Niles Johnson]

Hi Nicolas,

On Monday, March 10, 2014 11:16:17 AM UTC-4, Nicolas M. Thiéry wrote:

Hmm, as is, the argument is a bit general, and I believe it would rule
out the very principle underlying categories :-)

Oh -- maybe that's why I have a hard time understanding categories :)

 

...

 

The role of categories
is to make the code manageable by using more deductions to reduce the
amount of information you need to hardcode. Similar strategies have
been successfully used in one form or the other in GAP, Magma, Axiom,
MuPAD, and since we are getting bigger than those, we are exploring
further.

That makes sense -- I see the need for something along these lines.  After scanning the ticket discussion and documentation, I think the purpose is to provide some way of mixing in some (but not all) methods from one class to another.  And it is desirable that these be "top-level" methods -- that makes sense too.  But the way this is done does seem confusing (at least, as I understand it from examples here, on the ticket, and the documentation on defining a new axiom [1]).  Is the global axiom list necessary so that Sage knows where to find the relevant "mix-in" methods?  

On the question of which category should contain the actual definition of a method, a more even-handed approach would be to define a separate class so there is no arbitrary selection of a "largest category".  But it sounds like at this stage you would like *fewer* implementation ideas, not more ;)

As you said, implementing axioms involves some guessing about the future, and some opinions on what is useful/natural/convenient/pythonic/mathematical etc.  Maybe another useful exercise (for those of us just joining the discussion) would be a list of desirable properties for a notion of axioms in sage.  I think you gave something like this in the "relevance" section of your first post, but that is more about why we want axioms, rather than what properties they should have.  I've already made my attempt above, but I'll attempt to say it more clearly, as I understand it:

* An "axiom" is a collection of methods that can be called from two or more different classes.

* The methods inherited from an axiom are top-level.

* Each "axiom" has a mathematical description.

* If class Cs includes the methods from "axiom" A, then it must be true (either as a theorem or by definition) that every instance of "Cs with A" satisfies the description of the axiom A.  

The last two seem is especially important as new methods may be added to the "axiom", without necessarily testing them on Cs.  So there needs to be a theoretical guarantee that any (correctly implemented) method in A will work correctly for "Cs with A"

I guess under this definition then any time B is a subclass of A, the methods of A not overridden by B are an "axiom" since they're accessible by A and B.  But we also want to make just some of the methods of A into an axiom, without having to move them to a new superclass for both A and B.

Lastly, now that I've been thinking about this for a while, could someone explain how topological groups or Lie groups might be implemented in this framework?  The axiom documentation stresses that axioms define full subcategories, but Topological groups are not a full subcategory of sets (or groups), so I guess maybe one would define a category of topological spaces, and a Discrete axiom, making Sets the category of discrete topological spaces?  Would one have to move methods from Sets to TopologicalSpaces then?

I do mean this as a genuine question, even though it seems silly to insist that sage represent sets as discrete topological spaces (obviously we would want discrete *equivariant* topological spaces with trivial group action ;)

This being said, where would you put the threshold between appropriate
and overboard use of "math knowledge"?

I think Nathann Cohen gave a good answer, but I think another point is that different mathematical backgrounds are going to give different but equivalent ways of describing the same categories.  So it should be as easy as possible to figure out which description sage is using.

[1] http://sage.math.washington.edu/home/nthiery/sage-6.0/src/doc/output/html/en/reference/categories/sage/categories/category_with_axiom.html#defining-a-new-axiom

  

[Volker Braun]

On Monday, March 10, 2014 1:45:56 PM UTC, Nicolas M. Thiéry wrote:

I am happy to change those error messages for something more helpful;
What do you suggest?

- ``Unknown axiom name 'Endlich'. Please add it to sage.categories.category_with_axiom.all_axioms''
- ``Axiom 'Endlich' not defined by Categories of Cs or its super categories ''

 

As you know, I suggest making axioms some sort of object. And explictly set it in the nested class. Then you never have to guess the axiom of the nested class, so you can't get that error.

 

By the way, all_axioms would better be renamed all_axiom_names; agreed?

 

Yes, even better if you don't need it because you define the sort order inside the axiom. 

[Travis Scrimshaw]

Hey everyone,
   I've been using #10963 in developing #14901 (Lie algebras). I first gave 'Lie' as an axiom of NonAssociativeNonUnitalAlgebras (which I just asked Nicolas for how to do it without really looking at the examples), but decided that I didn't want _mul_() to give the Lie bracket, so I just made the category LieAlgebras be a subcategory of Modules. Next I implemented two axiom subcategories of WithBasis and FiniteDimensional (on my own) and I felt that it was natural and straightforward. This was about half a year ago (before much of the documentation was written).

   Now I feel that Volker's suggestion of making axioms into objects is a somewhat heavy-handed approach to something which is basically to act as a property, close to decorator interfaces in Java but here axiom determine inheritance. So I'd think a list/tuple of strings specifying what axioms are implemented is a better way to do it if we don't want to use Nicolas' implementation (which I don't object to).

   On that note, I think reviewers shouldn't hold up tickets because they don't like the current implementation without providing a working alternative and can demonstrate why it's better. This is what follow-up tickets are for; because if it's such a big issue, then people *will* work on it. Our development model is somewhat fast, however I've been told that Facebook pushes changes to it's live server after possibly only a quick check (I've see FB change radically from one day to the next because of this). So I will officially vote and say let's merge this ticket in as we can always revisit this.

Best,
Travis

[Nathann Cohen]

>    On that note, I think reviewers shouldn't hold up tickets because they
> don't like the current implementation without providing a working
> alternative and can demonstrate why it's better. 

Do you think that a patch should automatically be merged when it has been waiting for a reviewer for a long time ?

From time to time, I think that what a ticket implement is not a good idea. I usually say so on the ticket and refuse to review it (i.e. #13624).

I still think that implementing the reviewer's remarks is the author's job, though. Otherwise you will end up with careless reviews, because the reviewers have no time to improve somebody else's code. And I wouldn't want to see careless patches either, written assuming that the reviewer will fix the problems.

Nathann

[John H Palmieri]

On Tuesday, March 11, 2014 12:40:41 PM UTC-7, Nathann Cohen wrote:

>    On that note, I think reviewers shouldn't hold up tickets because they
> don't like the current implementation without providing a working
> alternative and can demonstrate why it's better. 

Do you think that a patch should automatically be merged when it has been waiting for a reviewer for a long time ?

With regards to #10963, the ticket had been reviewed and indeed had gotten a positive review, and then some other people looked at it and started asking questions. So Nicolas is not trying to bypass the review process, but rather trying to sort out a disagreement among the various participants on the ticket. That's probably too brief to adequately summarize what's going on, but anyway, I think your question is not really relevant to this particular ticket.

--
John

[Nathann Cohen]

Hey John no worries, I was only answering Travis' post and this (rethorical) question was just meant as a way to show that I did not concur with his view that reviewers should have to implement their remarks when the review gets long.

Nathann

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/hupt_5776j0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Anne Schilling]

I agree with John. I actually think Nicolas is quite patient trying to answer all questions.

My suggestion would be either for Volker to implement his alternative on a different ticket, so we
can see it in action and test it, or to let Nicolas' patch go in (provided of course that there are no
real issues, but rather only technical disagreements about the implementation details).
If we do not have Volker's alternative and we want this feature in sage, then I think the only
way one could vote is for Nicolas' patch to get in! And I think we do want this feature in Sage!

Best,

Anne

[Nathann Cohen]

Helloooooo !

> I agree with John. I actually think Nicolas is quite patient trying to answer all questions.

You are so kind.

> My suggestion would be either for Volker to implement his alternative on a different ticket, so we
> can see it in action and test it, or to let Nicolas' patch go in (provided of course that there are no
> real issues, but rather only technical disagreements about the implementation details).
> If we do not have Volker's alternative and we want this feature in sage, then I think the only
> way one could vote is for Nicolas' patch to get in! And I think we do want this feature in Sage!

If you want to get this ticket inside of Sage there is an easy way :
review it. The review is already going on, join it if you like (on the
ticket) ! And doing this job is the only way to know if a patch is
ready to get in or not.

Nathann

[Volker Braun]

On Tuesday, March 11, 2014 8:42:04 PM UTC, Nathann Cohen wrote:

If you want to get this ticket inside of Sage there is an easy way :
review it.

+1 

also would save me a lot of time

[Nicolas M. Thiery]

On Mon, Mar 10, 2014 at 11:36:01AM -0700, Niles Johnson wrote:
> Oh -- maybe that's why I have a hard time understanding categories :)

:-)

> That makes sense -- I see the need for something along these
> lines. After scanning the ticket discussion and documentation, I
> think the purpose is to provide some way of mixing in some (but
> not all) methods from one class to another. And it is desirable
> that these be "top-level" methods -- that makes sense too.

Hmm, I am not sure to see what you mean. See below too.

> But the way this is done does seem confusing (at least, as I
> understand it from examples here, on the ticket, and the
> documentation on defining a new axiom [1]).

You might want to check the primer as well (there is a link at the
beginning of the axiom doc).

> Is the global axiom list necessary so that Sage knows where to
> find the relevant "mix-in" methods?

Nope. Roughly speaking it's basically just used for printing out
categories, and to make FiniteGroups() work as an alias for
Groups().Finite(), which is mostly for backward compatibility.

> On the question of which category should contain the actual definition of
> a method, a more even-handed approach would be to define a separate class
> so there is no arbitrary selection of a "largest category". But it sounds
> like at this stage you would like *fewer* implementation ideas, not more
> ;)

:-)

Still, if you can elaborate on what you mean on an example, e.g. for
in which class to put the definition of the Distributive axiom. that
can give ideas for the future.

> As you said, implementing axioms involves some guessing about the
> future, and some opinions on what is
> useful/natural/convenient/pythonic/mathematical etc. Maybe
> another useful exercise (for those of us just joining the
> discussion) would be a list of desirable properties for a notion
> of axioms in sage. I think you gave something like this in the
> "relevance" section of your first post, but that is more about
> why we want axioms, rather than what properties they should have.

Yup.

> Lastly, now that I've been thinking about this for a while, could
> someone explain how topological groups or Lie groups might be
> implemented in this framework? The axiom documentation stresses
> that axioms define full subcategories, but Topological groups are
> not a full subcategory of sets (or groups), so I guess maybe one
> would define a category of topological spaces, and a Discrete
> axiom, making Sets the category of discrete topological spaces?
> Would one have to move methods from Sets to TopologicalSpaces
> then? I do mean this as a genuine question, even though it seems
> silly to insist that sage represent sets as discrete topological
> spaces (obviously we would want discrete *equivariant*
> topological spaces with trivial group action ;)

I am not a topologist by far, so I am probably going to say some
stupid things. But let me give it a shot at how we could design
this. I assume in general that a Topological Foo (or TFoo for short)
is a foo where the foo-operations are somehow *compatible* with the
topology (continuity I guess).

Assuming, for example, that Foo is a subcategory of groups, the
category hierarchy would look like:


Sets
/ \
Groups TSets
/ \ /
Foos TGroups
\ /
TFoos

I.e., under the usual hierarchy of categories (Sets, Groups, Foos,
...), you would get a parallel hierarchy for their Topological
variants. That is, in theory. In practice it's going to be a much
smaller hierarchy: for most categories Foos, there won't be specific
code implemented for Topological Foos.

Note that automatic intersections would not be appropriate here: a foo
which is also a topological set is not necessarily a topological foo.
That's related to the fact you mentioned: the category of topological
foos is not a full subcategory of the category of foos. Therefore
"Topological" should not be an axiom, but a "covariant functorial
construction" (very much like "Graded").

In practice, one just need to define the construction in the Sets
category, and then, when there is something to be implemented about
topological foos, we create a class Foos.Topological to model this
category. There is no need to hardcode that a topological foo is both
a foo and a topological group: the super category relation for
topological categories is constructed automatically.

Finally one needs to hardcode some facts, in particular about
compatibility with other axioms. For example, commutativity is
compatible with continuity: a commutative topological group is a
topological commutative group. On the other hand the existence of
inverses does not guarantee that those are continuous: a topological
monoid with inverse is not a topological group.


Now to the "Discrete" thing. It's often the case that a given set X
can be equipped with several topologies, and we probably don't want to
handle several topologies at once in the same parent. That is, instead
of having a single parent modeling X, we will have several parents
(possibly facades), each one modeling "X equipped with the YYY
topology". Those parents will be in the category of topological sets
(or some subcategory).

X by itself will be modeled by a parent in the category of sets, and
not endowed by default with it's discrete topology. Instead, there
would be some constructor, say SetWithDiscreteTopology, for building
"X equipped with the discrete topology".

Now what to do with the code for XXX equipped with their discrete
topology? If we have such code for several categories XXX, say Sets,
Groups, ... then it can make sense to model the corresponding
categories like "foos equipped with their discrete topology". Given
that, e.g., a "set equipped with the discrete topology" which is also
a group is a "group equipped with the discrete topology", this would
be a good use case for an axiom "EquippedWithTheDiscreteTopology".
Code would then go in classes Sets.EquippedWithTheDiscreteTopology,
Groups.EquippedWithTheDiscreteTopology.

As above the hierarchy relations will be deduced automatically, and as
a bonus, intersections will happen automatically.

> I've already made my attempt above, but I'll attempt to say
> it more clearly, as I understand it:
> * An "axiom" is a collection of methods that can be called from two or
> more different classes.
> * The methods inherited from an axiom are top-level.
> * Each "axiom" has a mathematical description.
> * If class Cs includes the methods from "axiom" A, then it must be true
> (either as a theorem or by definition) that every instance of "Cs with A"
> satisfies the description of the axiom A.
> The last two seem is especially important as new methods may be added to
> the "axiom", without necessarily testing them on Cs. So there needs to be
> a theoretical guarantee that any (correctly implemented) method in A will
> work correctly for "Cs with A"
> I guess under this definition then any time B is a subclass of A, the
> methods of A not overridden by B are an "axiom" since they're accessible
> by A and B. But we also want to make just some of the methods of A into
> an axiom, without having to move them to a new superclass for both A and
> B.

Err, I am not sure I can connect the above with what I have in mind :-)

For me, an axiom A is just a predicate on the objects of the category
Cs() defining it. Then, for each subcategory Ds() of Cs(), you can
consider the full subcategory Ds().A() of the objects of Ds satisfying
the axiom. And, if relevant, implement that category to provide
methods for those objects and their elements.

Topological is not axiom, because the predicate defining topological
groups is not the same predicate as for defining topological sets.

> I think Nathann Cohen gave a good answer, but I think another point is
> that different mathematical backgrounds are going to give different but
> equivalent ways of describing the same categories. So it should be as
> easy as possible to figure out which description sage is using.

Would you have a concrete example in mind?

[Paul-Olivier Dehaye]

I am confused about what to do. 

Everyone seems to be assuming that one of Volker's or Nicolas' approaches must be right. It might very well be that both approaches are suboptimal compared to a third. Still, Nicolas' has the merit to be implemented. 

Let me explain:

I support Nicolas' initiative and do very much think this is a feature we want in sage. 

I have concerns with Nicolas' nested classes approach but recognize that it has the merit of being the only one implemented (and used already!).

My opinion is that all the objective criteria for a patch have been passed (except for some regression on legacy code, which does not seem to be the core of Volker's objections) and that only fairly subjective concerns are left (paraphrase: "it's confusing to a python developer to have nested classes!"). 

I also have concerns with Volker's approach.

I have my own alternative approach to suggest. 

I am not willing to devote the time to implement my own alternative (just like Volker is not willing to implement his).

I am willing to review any patch submitted by Volker and suggest improvements (these are likely to require substantial work on his part). 

I am *definitely* not trying to be obstructionist but am afraid that in this case the review process for patches is unclearly defined, and just aping the behaviour of others by me would lead to total stalemate. 

At this stage I better outline my own solution so as to open it to more criticism from everyone (more fun!). As I see it, both Nicolas' and Volker's suggestions have the shortcoming that they require a one-time decision now of how to make purely mathematical information fit into python's syntax (object for an axiom versus nested classes). This also prevents any further reuse (short of python introspection of live objects or static analysis of the code) of this formalised-as-code purely mathematical information into any other system, be it the LMFDB, the combinatorial statistics finder, the atlas of categories, another CAS or any other mathematicians' collaborative efforts. Instead, I would advocate using a declarative domain specific language built for semi-formalizing mathematics. Such a language was developed as part of a PhD thesis (Florian Rabe), and is called MMT. It tries to precisely address the needs of a community of mathematicians trying to formalise some part of mathematics. In fact (and very importantly) you don't need to formalise from the ground up. You can just formalise parts of math (i.e. "a finite group has order a prime power", even though you never defined what a group was). This is meant to be used collaboratively, and in fact meant to be used for the whole math community at once in a distributed fashion. You should think of it as "OpenMath (re)done right". One could argue that this would require sage developers to know two languages (python and MMT), but that would only be true for those who want to implement a truly new category. In fact, if this MMT takes off (and sage might have a role in that), it would allow reuse across use cases, and we would conceivably benefit from the work of others, for instance a database of categories developed by others:

http://nforum.mathforge.org/discussion/1754/

Now in my "solution", this declarative language still has to be translated into python code. Of course at that stage mathematicians' interaction and python code have been decoupled, so it would be a pure question of optimisation whether Nicolas or Volker's solution is better. This question would have an entirely objective answer based purely on efficiency. 

Paul

PS: Florian has visited me in Zurich 6 months ago, and is receptive to the idea of using his system as a basis to generate code, even if it was not his original intent.

--
You received this message because you are subscribed to the Google Groups "sage-combinat-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-d...@googlegroups.com.
To post to this group, send email to sage-comb...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-combinat-devel.

[Travis Scrimshaw]

On Tuesday, March 11, 2014 12:40:41 PM UTC-7, Nathann Cohen wrote:

>    On that note, I think reviewers shouldn't hold up tickets because they
> don't like the current implementation without providing a working
> alternative and can demonstrate why it's better. 

Do you think that a patch should automatically be merged when it has been waiting for a reviewer for a long time ?

That's not close to what I said.

From time to time, I think that what a ticket implement is not a good idea. I usually say so on the ticket and refuse to review it (i.e. #13624).

This is a bad practice. Some implementation is better than none, and doing things your way turns people off from reviewing the tickets.

I still think that implementing the reviewer's remarks is the author's job, though. Otherwise you will end up with careless reviews, because the reviewers have no time to improve somebody else's code. And I wouldn't want to see careless patches either, written assuming that the reviewer will fix the problems.

I'm not disagreeing with you when there are errors and specific comments. Yet when the reviewer refuses to let something into Sage because (s)he thinks there is a better way without demonstration slows and sometimes halts progress. Otherwise we should stop improving computers because there's a better way to make them, so let's figure that out first before doing anything more. There are so many examples in history of things we should have done differently, but we only found out by doing the "wrong" thing first. BTW, that is one slipperly slope of logic jumps.

Best,
Travis

[Mark Shimozono]

Paul,

> Instead, I would advocate using a declarative domain specific language built for semi-formalizing
> mathematics

The appeal of this paradigm is evident. It addresses
a fundamentally important issue: how to structure the development process to
encourage the code to reflect the mathematics in the most transparent fashion.

Alas, it appears that this is a fair distance from being implemented
in python, and many important details need to be worked out.
If there was a prototype version with a demonstration-in-principle
then it would be the right time to debate this approach.

In the meantime, we all need a version of sage to work with...

--Mark

[Paul-Olivier Dehaye]

Thanks for the very thoughtful and encouraging comments. A big hurdle to starting any actual work on this has always been to see how well received the category framework would be.

Florian is also struggling with  some issues of canonicity of the definitions, so your insights might be helpful to him. His issues have more to do with presentation though, while yours has to do with dynamic resolution (but it might be that a canonical presentation could be based on your resolution). 

In any case, I do not want to distract the thread away from the topic of Nicolas' ticket, and the fact that he has implemented something that works beyond a prototype, unlike any of us.

Paul

 

On Wed, Mar 12, 2014 at 8:28 AM, Simon King <simon...@uni-jena.de> wrote:

Hi Paul, hi Mark,

On 2014-03-12, Mark Shimozono <msh...@math.vt.edu> wrote:
>> Instead, I would advocate using a declarative domain specific language built for semi-formalizing
>> mathematics
>
> The appeal of this paradigm is evident. It addresses
> a fundamentally important issue: how to structure the development process to
> encourage the code to reflect the mathematics in the most transparent fashion.
>
> Alas, it appears that this is a fair distance from being implemented
> in python, and many important details need to be worked out.

This could actually relate to another suggestion that I made on the
ticket, and like to mention here.

A substantial part of Nicolas' approach deals with the problem
of finding the correct class which Cs().SomeAxiom() will be an instance
of. For instance, mathematical results imply that
Rings().Division().Finite() should be an instance of the FiniteFields
class (resp. Fields.Finite---actually I like nested classes and find
them rather easy to understand and transparent).

All the logic---including the mathematical reasoning---is *implicit* in the
code, by chosing names and by setting some attributes. But in the end of
the day, we have a machinery that returns classes. Hence, what we in
fact have is (close to) a metaclass.

This metaclass must be fairly intelligent, as it implements mathematical
results (main example: Wedderburn's theorem). Part of my criticism of
Nicolas' approach is that the mathematical reasoning somehow happens in
the cloud: There is no single spot that does the reasoning *explicitly*,
but it is a dynamical process that seemingly does the right thing, but
only *implicitly*.

As I have demonstrated (as a proof-of-concept), the required mathematical
reasoning can be done by computations in the "boolean polynomial ring"
(which is implemented in Sage).

Hence, one could think of implementing a metaclass for category classes,
that acts as a database and knows
 - where existing category classes can be found (such as: "the FiniteFields
   class is in sage.categories.finite_fields"),
 - when and how the category class has to be created dynamically,
 - what structure is needed to apply an axiom (for applying the
   "Distributive" axiom, one needs that the category in question both is
   a sub-category of multiplicative and additive magmas),
 - how to find a "normalised" set of axioms that define a category.
   Really, the fact that ring*division*finite is the same as
   ring*division*commutative*finite boils down to normal form
   computations in a boolean polynomial ring.

And again, I have to admit that it isn't close to being implemented.
Thus, I'd prefer to have something that already works, which is Nicolas'
implementation.

> In the meantime, we all need a version of sage to work with...

+1

Best regards,
Simon

[kcrisman]

Preliminary points:

1) It's unreasonable to always ask reviewers to implement their ideas.  For instance, sometimes they may find a problem but not be sufficiently expert to implement it; that doesn't mean there isn't a problem.  Of course, then other reviewers may decide the critique is too trivial/far afield/etc.

2) It's unreasonable to expect authors to always implement reviewer ideas or critiques.  The same reason applies, though others exist.  Of course, then the ticket may (probably will) not be merged.

1-2) Both of the above are also subject to time constraints by author and reviewer, and I think that very few posts in this thread have really recognized that.   The reality is that sometimes good things just don't happen (not just in OSS, by the way) because people have to make choices about what to work on, or whether to work on (say) Sage at all.

1-2) Both of the above have happened zillions of times in Sage.  Just take a look at Trac.

3) Sometimes there will be philosophical differences about structure/implementation, ones which are not about bugs per se, but informed speculation.  There need to be ways to resolve those, but we have to recognize that some decisions will not be a full consensus.  And we have to recognize that those are not necessarily related to 1-2).

I don't know whether any or all of these 1-3) apply to #10963, though it looks likely.  But the philosophy of Sage has *typically* (not universally, not throughout all time) been to give new functionality a chance once technical problems have been worked out, even if not everything is implemented etc.   This thread seems like the canonical place to do it, but let's keep in mind the goal is to see whether, *given* that #10963 has no issues of that kind (which doesn't yet appear to be the case, see #15792 and related discussion), whether it should be given a chance.  

4) I'm not sure how documentation fits into this.  We usually just say "needs doc and doctests" but perhaps in some cases additional documentation is not just desirable (it always is) but actually necessary for things to be merged.  That was the case for the git change, for example.  I have no idea whether that is really the case here, since I don't use this functionality.   This did seem to be part of the motivation behind Volker's original remark, that the un-Pythonic syntax would cause confusion/bad infinite regress/various other things you can read about in this thread; I could imagine very very explicit doc that was "required reading" somehow dealing with this kind of potentially legitimate objection to a ticket's being merged.  And no, I'm not suggesting that any old new syntax is okay; the question is whether this one is different enough to make it inappropriate for Sage.

Main point: 

Raise your hand if you've never seen something implemented in Sage you thought should have been structured/named/organized differently.  I don't think even the BDFL can say that!  We may just end up having slightly suboptimal syntax if other syntax is not implemented in a reasonable period of time - that certainly isn't the first time in Sage, nor in other math software.  The key is whether the change/syntax/concept/whatever is technically sound *enough* and expands the capabilities of Sage enough - keep the mission statement in mind.  And in the end, a decision will have to be made, or the decision not to have this functionality will have been made by default.  Perhaps that's the correct decision, but let's be careful about how often we use that decision.

Good luck resolving this, everyone ;-)

[Nicolas M. Thiery]

Hi Simon!

On Tue, Mar 11, 2014 at 09:36:02PM +0000, Simon King wrote:
> In what location would you put the specification of an axiom and in what
> location would you put the documentation of a concrete implementation of
> the specified axiom, when axioms are lists of strings?

Just as a tiny complement for those who haven't checked the details:

- The specification of an axiom `A` currently goes in the
documentation string of the method Cs().A, where Cs is the category
defining that axiom. For example, the specifications of the axiom
Associative is defined in the documentation string of
Magmas().Associative (which technically is
Magmas.SubcategoryMethods.Associative).

Oops. Actually that's where it ought to have been; but apparently I
took the definition of associative for granted. Let me fix that now.

- Documentation of an implementation of an axiom: I assume you mean
implementation of a category with axiom. The documentation of the
category of finite groups goes in Groups.Finite.

- Axioms are indeed currently modeled by strings; on the user or
category implementer interface side, this shows up in very few spots
(essentially when you ask for the axioms satisfied by a category).
So we can easily change this later.

[Niles Johnson]

On Tuesday, March 11, 2014 6:21:19 PM UTC-4, Nicolas M. Thiéry wrote:

On Mon, Mar 10, 2014 at 11:36:01AM -0700, Niles Johnson wrote:

 

> After scanning the ticket discussion and documentation, I
>    think the purpose is to provide some way of mixing in some (but
>    not all) methods from one class to another.  And it is desirable
>    that these be "top-level" methods -- that makes sense too.

Hmm, I am not sure to see what you mean. See below too.

I think I was trying to understand what practical thing an axiom accomplishes, and I thought it was basically just a way of letting certain methods from instances of class A be called from instances of class B.  But it's more than that . . .

 

For me, an axiom A is just a predicate on the objects of the category
Cs() defining it. Then, for each subcategory Ds() of Cs(), you can
consider the full subcategory Ds().A() of the objects of Ds satisfying
the axiom. And, if relevant, implement that category to provide
methods for those objects and their elements.

Thanks for clarifying.

-Niles

[Nicolas M. Thiery]

On Wed, Mar 12, 2014 at 05:24:08PM +0000, Simon King wrote:
> OK. So, Cs().A?? yields the specification and Cs.A?? the implementation.

Yes, though to get the implementation, it is preferable to use
``Cs().A()?``: it works uniformly, independently of whether this
category is implemented in Cs.A or elsewhere, or is just a join
category.

[Robert Bradshaw]

On Mon, Mar 10, 2014 at 5:48 AM, Nicolas M. Thiery
<Nicolas...@u-psud.fr> wrote:
> On Mon, Mar 10, 2014 at 04:35:25AM -0700, Volker Braun wrote:
>> IMHO the contained-class makes it clear what the base category is, but not
>> what the axiom is. For example:
>> class Cs(Category):
>> class Finite(CategoryWithAxiom):
>> pass
>> class Ds(Category):
>> class Endlich(CategoryWithAxiom):
>> pass
>> Now Cs.Finite implements an axiom, but Ds.Endlich does not (since you
>> haven't added the "Endlich" string to a global list somewhere).
>
> No. What makes Endlich an axiom (or not) is that some super category
> has defined it (or not) [1].

Ah, this was not at all clear in the initial proposal.

> As an analogy, when an abstract class C declares a method foo, and you
> implement this method in some subclass D, you don't write explicitly
> in D.foo that you override the method foo declared in C.
>
> If you need that information, you either use the semantic (here the
> math), or you walk the hierarchy. Of course, since the hierarchy is
> built dynamically here, you need to walk the hierarchy dynamically;
> introspection does that nicely for you (yes, unless Sage does not
> start at all; but in that case we are speaking of the very basic
> axioms and you can still use the math).
>
> Cheers,
> Nicolas
>
> [1] The fact that Sage needs a list of the names of all the axioms
> being defined is an implementation detail. Recall that it is used
> mostly for printing purposes (and allowing for syntactical sugar in
> some spots). We could get rid of it without changing the syntax of how
> axioms are defined and implemented.

>
> --
> Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
> http://Nicolas.Thiery.name/
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.

[Robert Bradshaw]

On Fri, Mar 7, 2014 at 4:14 AM, Nicolas M. Thiery
<Nicolas...@u-psud.fr> wrote:
> Dear Sage developers,
>
> This is a call for vote about the ticket:
>
> #10963: axioms and more functorial constructions [1]
>
> Sorry this post is long; it's trying to give a fair summary of the
> huge discussion on [1]. Here is the table of contents:
>
> - What is the ticket about
> - Relevance
> - History
> - Debate
> - Personal opinion
> - Call for vote
>
>
> What is the ticket about
> ------------------------
>
> The main feature implemented by #10963 is an infrastructure for axioms
> which enriches the current category infrastructure. Namely, if you
> have a category, you can now ask for the (full) subcategory of the
> objects satisfying a given axiom. For example:
>
> sage: Groups().Finite()
> Category of finite groups
>
> Of course, this can be iterated:
>
> sage: Magmas().Associative().Commutative().Unital().Inverse()
> Category of commutative groups
>
> By itself this is a little improvement, since it gives for example a
> single entry point for all the variants of groups (or of magmas, ...);
> in particular this will allow, as a later step, to remove a lot of
> stuff from the global name space. This is in line with what we already
> have for functorial constructions:
>
> sage: Groups().CartesianProducts()
> Category of cartesian products of groups
>
> What's really new is that axioms play nicely with intersection:
>
> sage: Groups() & Magmas().Commutative() & Sets().Finite()
> Category of finite commutative groups
>
> sage: (CommutativeAdditiveGroups() & Monoids()).Distributive()
> Category of rings
>
> sage: Rings().Division() & Sets().Finite() # Wedderburn's theorem
> Category of finite fields
>
> Fine, this is cute, but why is this relevant?
>
> Relevance
> ---------
>
> As time passes, the hierarchy of abstract classes for parents,
> elements, and morphisms in Sage is getting larger and larger. From the
> beginning, the purpose of categories in Sage has been to exploit the
> strong semantic provided by the math to keep such large hierarchies
> maintainable. The key is to fight hard against duplication and toward
> single point of truth. Just as a trivial example, Sage deduces
> automatically from the above that the abstract class for finite groups
> (Groups().Finite().parent_class) should inherit from the abstract
> class of groups and the abstract class of finite monoids.
>
> Axioms are an important step to scale further by controling the
> potential combinatorial explosion of the number of classes and
> categories. For details, see the section on axioms in the primer [2].
>
> History
> -------
>
> The work on this ticket started three years ago, growing up out of
> strong immediate needs for the hierarchies of categories appearing in
> the representation theory of semigroups or Hopf algebras. But the
> infrastructure is also beneficial for non exotic categories like the
> ones we have seen in the examples above. For example, it made it
> practical to implement categories for the many variants of algebras
> (unital or not, associative or not, commutative or not, ...)
>
> For the design of the category infrastructure, we had some strong
> source of inspiration from other systems like Axiom, MuPAD, or
> Magma. As far as I know, no other system has a non trivial axiom
> infrastructre like this one. Which means that it took a long time --
> three years -- to analyse the needs, explore potential idioms, and
> implement and reimplement the infrastructure until it felt right. It
> also took as a prerequisite to fix a bunch of technical hurdles, in
> particular #11935 and #13589 (thank you by the way to all who worked
> and helped on those tickets, and in particular Simon and Nils).
>
> In those three years, this ticket benefited from continuous
> discussions with the other category fans, and in particular Florent
> Hivert and Simon King. I also presented the framework at the occasion
> of several Sage days to see how natural it would feel to new users and
> developers. Most importantly, the framework was put to intensive use
> in actual code to test whether the approach was sensible, viable, and
> practical.
>
> The undesirable consequence of this long history is that the ticket
> has become large: it contains the axiom infrastructure, the
> reorganization of the Sage categories using that infrastructure, a
> bunch of new categories, and quite some related stuff that aggregated
> to it along the way and would be painful to separate away (mostly
> improvements to the existing functorial constructions).
>
> Status
> ------
>
> All test passes. The code is 100% doctested. All dependencies are
> merged. Much time was spent on performance too, in particular checking
> that the Sage startup time did not increase non trivially. Simon King
> completely reviewed the code three months ago (thanks!!!) and gave a
> positive review back then.
>
> Since then, Simon, Nils, and Volker spotted and tracked down a bug
> which is fixed now. I am super grateful to them because it was tricky:
> the code was actually too paranoid! It did an assertion check too
> early during some initialization, which caused a recursion loop
> involving imports, and ended up with a silent garbage collection
> issue.
>
> This triggered a long discussion on trac where basically each and
> every design choice was put again to trial, a good thing in itself!
> Based on this discussion (thanks by the way to all the participants),
> I added a 1500-lines overview text about axioms, including rationale
> for the design decisions, the specification of the infrastructure, and
> a detailed sketch of proof of correctness. This text, and some further
> little improvements, are under review by Simon. There are also a few
> further small suggestions from the reviewer's that I have offered to
> implement, against my own opinion, if they really cared about
> them. It's a matter of days before this is done.
>
> This ticket is holding up a bunch of other tickets that depend on it,
> either because they implement themselves new categories with axioms,
> or use features from the ones implemented here. This includes:
>
> - #14102: Non symmetric Macdonald polynomials. This is one of the main
> outcomes of last year's ICERM semester. Up to one glitch that
> appeared when merging with recent versions of Sage, it has been
> ready to go since last June. It's an important feature and end users
> are waiting for it.
>
> - Work by PhD students (Jean-Baptiste Priez, ...) on Hopf algebras, a
> large piece of which could go in very soon once #10963 is in. This
> code is one of the major pieces of their PhD work, and they deserve
> to get credit for it.
>
> - #11111 and follow-ups on representation theory of finite dimensional
> algebras and semigroups by Saliola, Aladin, Florent, myself, ...
>
> - Further work on categories (support for morphisms, ...)
>
> - A bunch of further stuff in the former Sage-Combinat queue
>
> Taking this in consideration, since last June, basically all the
> energy I could spend on Sage-development was devoted to finalizing
> this ticket and getting it merged.
>
> Debate
> ------
>
> In my view, the most non obvious point about this ticket is whether
> the general approach just makes any sense and is viable. De facto, it
> took me three years to convince myself! Now I am convinced :-) And I
> tried to be convincing in the documentation. Now, if you wonder about
> this point, which is legitimate, actually sane, please read [2] and
> decide for yourself. Comments and opinions are very welcome. Just
> please create a new thread so that we don't mix the discussions.
>
> As far as I can tell from the long trac discussion, there is a
> reasonable consensus about this first point; I might be biased though.
>
> There is also a consensus about a certain number of desirable further
> features (see the TODO's in the documentation) that are best postponed
> to follow-up tickets since this one is already way too large! One such
> feature is to improve and make the overview text more concise, as it
> matures thanks to the feedback from more and more developers reading
> it.
>
> Now, as often, there can be variants on how the approach is
> implemented. And diverging opinions about those variants. Volker
> dislikes some syntactical and implementation choices, and would like
> to change some them. I dislike his counter propositions.
>
> The difficulty in this discussion is that this is more about wild
> guesses about the future; at this point, we lack concrete amo to back
> up clear cut decisions; we just have forged different intuitions about
> the right balances between different principles. That's fine by
> itself: I like that developpers have opinions; it's a richness for the
> Sage community. However we are stuck in an endless discussion with no
> consensus emerging. And a lot of time is wasted on it when there is so
> much else to do.
>
> I think the divergence points are essentially about:
>
> (1) The usage of nested classes: currently if Cs() is a category which
> implements stuff for its subcategory Cs().A() of the objects
> satisfying the axiom A, then that stuff is to be put in a class
> Cs.A
>
> Typically this class can be a nested class, or it can be put
> elsewhere, e.g. in a lazily imported module, with a link from Cs;
> this is very flexible.

This is how Parents and Elements work. I realize this is a
controversial decision (with good reasons on both sides) but IMHO
shouldn't hold up the ticket to continue the convention.

> (2) The "header" for categories with axioms
>
> {{{
> class Blah(Category):
> # The currently implemented syntax
> class Finite(CategoryWithAxiom):
> ...
>
> # Volker's proposal, something like:
> class ChooseYourName(axioms.Finite.Category):
> ...
> }}}
>
> plus possibly some attribute setting.

As I understand it, there are a couple of different proposals here.
Could someone list them (maybe a wiki would be a better format)
according to who finds them ideal, acceptable, and unacceptable? Maybe
short pros and cons of each. Is there any solution that's at least
acceptable to everyone? In that case, is it simply a matter of who's
going to put in the work to change it to this globally-acceptable
form? If it's the latter, I'd say we set a deadline that if anyone
feels strongly enough to make the change they do it, otherwise we go
ahead and try to get this in. Or, if the current solution is plain
unacceptable a solid argument needs to be made. This isn't about
correctness or performance or sufficient testing, it's a question of
aesthetics, right? ( I realize this bleeds into readability,
maintainability, ...) Then more people can weigh in 'cause it seems to
me a shame to hold something like this up.

> (3) Internal implementation points.

Do any of these need fixing now? Or is it good enough, but could be better?

> There is also a preexisting performance issue in some special
> situations which is made worse by this ticket (see #15801). We have a
> plan to fix it, and I am happy to settle this with Simon shortly after
> #10963 is merged.
>
> Personal opinion
> ----------------
>
> About (1): we have been using the same syntax for functorial
> constructions ever since the initial implementation of categories back
> in 2009. It has proven a very practical way to structure the code, we
> never had a problem with it, and developers learned it all right. I
> don't see why we should deviate from this syntax for axioms given that
> axioms are of the same nature as functorial construction, just with
> more properties. If we decide for another syntax, then we should
> change the syntax as well for functorial constructions. I am *not*
> undertaking this change myself.
>
> About (2): I had considered something similar earlier on, and
> practical usage told me to drop it. In any cases, it's a rather small
> change to the interface (one line or two per new category with
> axiom). It can be implemented in a follow up ticket. Even in a couple
> months there won't be hundreds of new categories with axioms
> implemented out there. If we convince ourselves that we should really
> change the interface, the categories with axioms in the Sage library
> will be easy to fix. And we can warn and help the couple developpers
> that will have implemented some in their private code.
>
> About (3): the code is already functional. It certainly can be further
> improved, but this can be postponned at no cost to followup tickets.
>
>
> Conclusion: In such a situation where it comes to opinion against
> opinion, I believe that decisions that have been battlefield tested
> for sanity and practicality over a non trivial period (years) on a non
> trivial body of code (50 categories just in the Sage library itself)
> are as good as any other. The ticket is already way too big (I take my
> share of the blame for it) and this is holding up the work of
> many. Hence further development should go in follow up tickets,
> especially if it's unclear whether they actually improve the
> situation.
>
> Let's not deviate even more from incremental development and the usual
> "release early, release often".
>
> Call for vote
> -------------
>
> [ ] Merge this ticket, and move on with further improvements in follow
> up tickets
> [ ] This ticket should be held up until everybody is happy
> [ ] This ticket does not make any sense
>
> Let me conclude by thanking everybody who has been involved in this
> ticket. And also those who worked so hard on the new Sage development
> workflow! It's been a life saver in the last months, and the previous
> workflow has some responsibilities on the size of this ticket.
>
> Cheers,
> Nicolas
>
> PS: I have tried hard to make a fair account of the situation. Yet
> please read everything with a grain of salt, for this has been the
> most frustrating development experience I ever had.
>
> [1] http://trac.sagemath.org/ticket/10963
> [2] http://sage.math.washington.edu/home/nthiery/sage-6.0/src/doc/output/html/en/reference/categories/sage/categories/primer.html

[Nils Bruin]

On Thursday, March 13, 2014 9:38:35 PM UTC-7, Robert Bradshaw wrote:

This isn't about
correctness or performance or sufficient testing, it's a question of
aesthetics, right?

There are performance issues actually, but probably ones that can be dealt with as long as we make sure that the number of possible categories is uniformly bounded and relatively small. Otherwise, #10963 produces a rather bad memory leak. Giiven that currently we have categories parametrized by base ring, there is such a leak. Even when the design decisions are settled (or at least preliminarily agreed upon) there is more performance/memory use testing to be done.

[Nicolas M. Thiery]

Dear Sage developers,

Here is an update on the progress on #10963.

- About this issue: I implemented in #15801 our plan of allowing
categories like ``Modules`` to be parametrized by the category of
the base ring instead of the base ring. From the benchmarks, it
seems to hold water and fixes all the performance issues made worse
by #10963. It's in needs review.

- About the syntax: Sage Days 57 was the occasion to discuss with
Volker. At the end of the day we still have a difference of taste on
the best syntax for implementing an axiom for a given category;
however Volker agreed that the current syntax has the advantage of
being implemented. So even though he would not endorse it himself,
he would not veto it either if some other reviewer(s) would accept
it.

- Tests: up to double checking with the latest develop version, all
long tests pass.

- Dependencies: there remains a small one, namely #15919, for which I
proposed a possible solution which needs review.

- Documentation review: Darij has made a great job, and it is
basically finished.

- Technical review: someone needs to review the changes made in the
last month(s). I can gather a list of commits if useful.

Cheers,
Nicolas