Sage Explorer

[Nicolas M. Thiery]

Dear Sage developers,

At the AIM/ICMS workshop ``Online databases: from L-functions to
combinatorics'' last week, Jason Bandlow and myself had fun writing a
prototype for a simple LMFDB-style web application allowing to explore
Sage. There it is!

Sage Explorer
-------------

Sage Explorer is a tool for exploring Sage objects and connections
between them. It displays a Sage object, some relevant information
about this object, and links to related objects (those that can be
obtained using a method of the object). One central feature of the
tool is to make it easy to configure which piece of information is
relevant, typically depending on the semantic of the object.

https://github.com/jbandlow/sage-explorer
https://explore.sagemath.org (some day)

You can find some screen shots of Sage Explorer in action at:

https://github.com/jbandlow/sage-explorer/wiki/Screenshots

How to install and run it
-------------------------

You need Sage installed (http://sagemath.org). Then, in any directory
of your choice, run the following shell commands:

git clone https://github.com/jbandlow/sage-explorer.git
cd sage-explorer
sage -python sage_explorer

and connect to the URL that is mentioned (typically http://127.0.0.1:5000/)

WARNING: AT THIS POINT NO STEP HAS BEEN TAKEN TOWARD SECURITY

By choosing appropriate url's, the user can trivially run any sage
command -- and in particular any shell command -- on the server under
the web server's uid. Use locally or at your own risks.

Motivation for this prototype
-----------------------------

The purpose was to evaluate whether such a tool could be written as a
thin view layer above Sage, and how much the semantic information
available in Sage was useful and sufficient for that purpose.

Implementation
--------------

Sage Explorer is currently implemented as a standalone flask
web-site. It indeed a thin layer: 300 lines of infrastructure code,
150 lines of html jinja templates, 60 lines of config. As desired, the
infrastructure is completely generic and allows for exploring any Sage
object. The object-dependent configuration (what are the important
properties that should be displayed) is quite concise, and does not
depend on the rendering detail (no html). It actually could be used as
is to build an heavy weight application instead of a web interface.
The only prerequisite for a contributer to expand it is to know the
mathematics of the object to be displayed and how to call the
appropriate Sage commands to compute its properties.

Future
------

The two authors are unlikely to lead further development of this tool,
since they don't have a strong need for it themselves. If you think
this is useful, patches and take overs are more than welcome! We have
set up developers mailing list:

sage-exp...@googlegroups.com

The most urgent feature would be to use, say, the single-cell server
as backend to handle the security and then run it publicly, typically
on http://explore.sagemath.org/.

For more, see the TODO file in the git repository.

Enjoy your exploration!
Nicolas
--
Nicolas M. Thi�ry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[William Stein]

On Wed, Jan 30, 2013 at 1:04 PM, Nicolas M. Thiery
<Nicolas...@u-psud.fr> wrote:
> Dear Sage developers,
>
> At the AIM/ICMS workshop ``Online databases: from L-functions to
> combinatorics'' last week, Jason Bandlow and myself had fun writing a
> prototype for a simple LMFDB-style web application allowing to explore
> Sage. There it is!
>
> Sage Explorer
> -------------
>
> Sage Explorer is a tool for exploring Sage objects and connections
> between them. It displays a Sage object, some relevant information
> about this object, and links to related objects (those that can be
> obtained using a method of the object). One central feature of the
> tool is to make it easy to configure which piece of information is
> relevant, typically depending on the semantic of the object.

Can I ask one quick question? This is *NOT* meant to be a rhetorical
question (though it could incorrectly be taken that way). Why did you
not write this as an interact? I think with a few small additions to
interact (maybe the kinds I just did in my re-implementation of
interact over the weekend for salvus), this would be pretty easily
done using interact.

I note "The two authors are unlikely to lead further development of
this tool," below, and think perhaps the best next step would be to
take the same idea (and most of the same code), but done in the
context of an enhanced interact command.

William

> Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
> http://Nicolas.Thiery.name/
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Nicolas M. Thiery]

Hi William!

On Wed, Jan 30, 2013 at 05:33:57PM -0800, William Stein wrote:
> Can I ask one quick question? This is *NOT* meant to be a rhetorical
> question (though it could incorrectly be taken that way). Why did you
> not write this as an interact?

The main reason is that we are not serious notebook users; so even if
we did write a couple small interacts, this is not our natural reflex
:-) Another reason is that we wanted something as simple as possible
that would "behave like" the lmfdb; i.e. a standalone web app.

That being said, having explorer cells in the notebook (or even in a
single-cell using the single-cell server) definitely sounds natural,
as well as using interacts for this.

> I think with a few small additions to interact (maybe the kinds I
> just did in my re-implementation of interact over the weekend for
> salvus), this would be pretty easily done using interact.

I just had a deeper look at the current interact code. The main
feature I feel I am missing to implement an "interact-based" Sage
Explorer is how to change the list of controls, and not only the
output zone, each time an interaction occurs (here there is basically
one per method of the object being explored). If this is possible, A
small example would be most welcome!

> I note "The two authors are unlikely to lead further development of
> this tool," below, and think perhaps the best next step would be to
> take the same idea (and most of the same code), but done in the
> context of an enhanced interact command.

Yup. Ideally only the rendering part would change and the rest of the
code base would remain identical, so that it could also be used in a
standalone app like now.

Cheers,

[William Stein]

On Wed, Jan 30, 2013 at 7:49 PM, Nicolas M. Thiery
<Nicolas...@u-psud.fr> wrote:
> Hi William!
>
> On Wed, Jan 30, 2013 at 05:33:57PM -0800, William Stein wrote:
>> Can I ask one quick question? This is *NOT* meant to be a rhetorical
>> question (though it could incorrectly be taken that way). Why did you
>> not write this as an interact?
>
> The main reason is that we are not serious notebook users; so even if
> we did write a couple small interacts, this is not our natural reflex
> :-) Another reason is that we wanted something as simple as possible
> that would "behave like" the lmfdb; i.e. a standalone web app.
>
> That being said, having explorer cells in the notebook (or even in a
> single-cell using the single-cell server) definitely sounds natural,
> as well as using interacts for this.
>
>> I think with a few small additions to interact (maybe the kinds I
>> just did in my re-implementation of interact over the weekend for
>> salvus), this would be pretty easily done using interact.
>
> I just had a deeper look at the current interact code. The main
> feature I feel I am missing to implement an "interact-based" Sage
> Explorer is how to change the list of controls, and not only the
> output zone, each time an interaction occurs (here there is basically
> one per method of the object being explored). If this is possible, A
> small example would be most welcome!

This is not possible with the Sage notebook. However,
coincidentally, it is exactly the main feature I added to interacts
when re-implementing them in Salvus. In Salvus, you'll be able to do, e.g.,:

@interact
def f(n=10):
interact.m = n*2

and (1) it will automatically create a control for m, and (2) set it
to the value 2*n.
In general, you can set control values and create controls by
assigning to interact.[name],
and you can remove controls by doing "del interact.m".

-- William

>
>> I note "The two authors are unlikely to lead further development of
>> this tool," below, and think perhaps the best next step would be to
>> take the same idea (and most of the same code), but done in the
>> context of an enhanced interact command.
>
> Yup. Ideally only the rendering part would change and the rest of the
> code base would remain identical, so that it could also be used in a
> standalone app like now.
>
> Cheers,
> Nicolas
> --

[Jason Grout]

> when re-implementing them in Salvus. In Salvus, you'll be able to do, e.g..,:

>
> @interact
> def f(n=10):
> interact.m = n*2
>
> and (1) it will automatically create a control for m, and (2) set it
> to the value 2*n.
> In general, you can set control values and create controls by
> assigning to interact.[name],
> and you can remove controls by doing "del interact.m".

What if you then try to change the value of the m control (or can you
change the value of the m control)? Or are these automatically created
things more like read-only display widgets than "controls" (i.e.,
controlling something)?

Nicolas, you can also do this in the current sage cell server, though
the notation is a little more difficult than William's proposal.
Basically, you can have nested interacts, and you can give a list of
controls to any interact call, like

@interact(controls=[('controlname', control_default),
('nextcontrolname', next_control_default)])
def f(**kwd):
print kwd

Here's a short example

http://aleph.sagemath.org/?q=3d3b3f6b-a3a9-4619-927d-fd5c1afb526c&lang=sage

The control defaults passed to the controls parameter is processed the
same as control defaults in the function.

Thanks,

Jason

>
> -- William
>
>
>>
>>> I note "The two authors are unlikely to lead further development of
>>> this tool," below, and think perhaps the best next step would be to
>>> take the same idea (and most of the same code), but done in the
>>> context of an enhanced interact command.
>>
>> Yup. Ideally only the rendering part would change and the rest of the
>> code base would remain identical, so that it could also be used in a
>> standalone app like now.
>>
>> Cheers,
>> Nicolas
>> --

>> Nicolas M. Thi�ry "Isil" <nth...@users.sf.net>

[William Stein]

It's a normal control at that point -- it sends a message back to the
server and calls the function f.
One thing is that it does *not* try to pass m into f. The interact
object can tell you which variable(s)
triggered a called, and of course you can get their values with
"interact.<var name>".
To tell what triggered a call, do "interact.changed()".

>Or are these automatically created things more
> like read-only display widgets than "controls" (i.e., controlling
> something)?

No. They are full controls.

>>> Nicolas M. Thiéry "Isil" <nth...@users.sf.net>

>>> http://Nicolas.Thiery.name/
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "sage-devel" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to sage-devel+...@googlegroups.com.
>>> To post to this group, send email to sage-...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/sage-devel?hl=en.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>
>>
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Jason Grout]

On 1/30/13 11:35 PM, William Stein wrote:
> It's a normal control at that point -- it sends a message back to the
> server and calls the function f.

> One thing is that it does*not* try to pass m into f. The interact

> object can tell you which variable(s)
> triggered a called, and of course you can get their values with
> "interact.<var name>".
> To tell what triggered a call, do "interact.changed()".

Ah, okay, that makes more sense.

Jason

[Harald Schilly]

On Wednesday, January 30, 2013 10:04:49 PM UTC+1, Nicolas M. Thiéry wrote:

Future
------

Just my 2 cents:

When I saw it, I liked that it doesn't rely on javascript (ok, well, formula rendering), and had a linear top-down layout (by accident). This led me to the conclusion that it could also be a suitable as simplified interface for mobile devices! Of course, the current features would be just a small part, but it's quite obvious, that an object with an easy-to-scroll list of methods and attributes is much easier to use than having to type in anything. It's also much more accessible to novice users …

What's missing, in Sage, is a decorator for each method, which would specify the input type. Based on that, generic "interact like" forms for inserting values could be created. (Right now, this explorer only shows methods without arguments)

H

[Julien Puydt]

Le 31/01/2013 10:29, Harald Schilly a �crit :

> What's missing, in Sage, is a decorator for each method, which would
> specify the input type.

I thought that wasn't really possible with python?

Snark on #sagemath

[Harald Schilly]

On Thu, Jan 31, 2013 at 10:40 AM, Julien Puydt <julien...@laposte.net> wrote:

I thought that wasn't really possible with python?

Well, not directly, but one could define and add a decorator that just attaches a hidden attribute, like, ".__argument_specification" to the method or function. then, one could read that list and do something useful with it. We could even extend this to include invariants, like, "argument 1 is an Integer and must be >= 0 and <= 10". That's great for randomized testing btw ...

H

[Volker Braun]

I don't think we want to add decorators to each method. In fact, I definitely don't :-)

A saner approach would be to leverage our existing doctests. Just go through the AST and see which methods are being called directly in doctests. Then, as you execute the doctests, save the types of the arguments into a database. Would be a fun undergrad project...

[Travis Scrimshaw]

Hey everyone,

On Thursday, January 31, 2013 4:54:52 AM UTC-5, Volker Braun wrote:

I don't think we want to add decorators to each method. In fact, I definitely don't :-)

I agree. Plus how would this work with optional/variable-length arguments? (Also it gets away from the weakly typed nature of Python.)

A saner approach would be to leverage our existing doctests. Just go through the AST and see which methods are being called directly in doctests. Then, as you execute the doctests, save the types of the arguments into a database. Would be a fun undergrad project...

 

   Hmm...However we should make sure it is after a `__classcall__()`, for example in the crystals code, lots of input is of the form `CrystalOfTableaux(['A', 4], shape=[3,2,2,1]) and what we are after is not two lists, but a CartanType (although the actual datatype you get depends on the letter and ideally we'd want to hide this) and a partition. I guess in these complicated cases a decorator, some kind of docstring hook (such as :inputtype:`Integer`) or manually enter data into a table would be a way to do it...

...Actually what if we did that as a first pass, but from then onwards, we (i.e. whomever would be doing this) would go back and do the (likely small) number of corrections. Plus we would display the data from the database as part of the documentation and when we build the doc, any classes or functions which did not have an entry in the database would run the first pass (unless it used a decorator/datatype hook).

Although this would increase development time (I doubt by much in general), and it would mean we'd have to get doctest coverage for all public functions to 100%...

On Thursday, January 31, 2013 9:29:26 AM UTC, Harald Schilly wrote:

When I saw it, I liked that it doesn't rely on javascript (ok, well, formula rendering), and had a linear top-down layout (by accident). This led me to the conclusion that it could also be a suitable as simplified interface for mobile devices! Of course, the current features would be just a small part, but it's quite obvious, that an object with an easy-to-scroll list of methods and attributes is much easier to use than having to type in anything. It's also much more accessible to novice users …

   If we use the stored database approach, we can also add additional data associated to each type and/or input, such as what type of widget it should be, min/max values, etc. This would make any web/mobile interface much easier/faster to construct and seems fairly flexible to me.

Best,

Travis

[Nicolas M. Thiery]

On Thu, Jan 31, 2013 at 10:40:14AM +0100, Julien Puydt wrote:
> Le 31/01/2013 10:29, Harald Schilly a �crit :

> >What's missing, in Sage, is a decorator for each method, which would
> >specify the input type.
>
> I thought that wasn't really possible with python?

There are a couple existing implementations using decorators:

http://wiki.python.org/moin/PythonDecoratorLibrary#Type_Enforcement_.28accepts.2Freturns.29

As far as I understand, Python 3 will allow to achieve the same using
the natural syntax `def f(x: int, y: float) : ...`, with flexibility
on what "type" can be.

As for whether we want to do it, here is my point of view: MuPAD had
this feature with some configurable policy for controlling whether
dynamic type checking would occur or not:

- Always type check
- Never type check
- Only type check at the interpreter level, but not deeper in the
function call stack; the default.

In practice, that was a great feature. Both as a debugging aid, and
also as added semantic information that could be used by any function
manipulating code (e.g. the Sage Explorer as Harald mentioned, or when
building morphisms, or ...).

That being said, we certainly don't want to decorate every Sage
methods at once. So the database approach, with the databased filled
up using doctests, is sensible indeed. Florent already used that
approach to recover all the implementations of a given category
(e.g. answer to the question: what are all the groups implemented in
Sage).

Cheers,

[William Stein]

On Thu, Jan 31, 2013 at 6:04 AM, Nicolas M. Thiery
<Nicolas...@u-psud.fr> wrote:
> On Thu, Jan 31, 2013 at 10:40:14AM +0100, Julien Puydt wrote:

>> Le 31/01/2013 10:29, Harald Schilly a écrit :
>> >What's missing, in Sage, is a decorator for each method, which would
>> >specify the input type.
>>
>> I thought that wasn't really possible with python?
>
> There are a couple existing implementations using decorators:
>
> http://wiki.python.org/moin/PythonDecoratorLibrary#Type_Enforcement_.28accepts.2Freturns.29
>
> As far as I understand, Python 3 will allow to achieve the same using
> the natural syntax `def f(x: int, y: float) : ...`, with flexibility
> on what "type" can be.

Unfortunately, type declarations, with that syntax, was *proposed* for
Python 3, but then soundly rejected by Guido as "too ugly". See

http://www.python.org/dev/peps/pep-3107/

-- William

>
> As for whether we want to do it, here is my point of view: MuPAD had
> this feature with some configurable policy for controlling whether
> dynamic type checking would occur or not:
>
> - Always type check
> - Never type check
> - Only type check at the interpreter level, but not deeper in the
> function call stack; the default.
>
> In practice, that was a great feature. Both as a debugging aid, and
> also as added semantic information that could be used by any function
> manipulating code (e.g. the Sage Explorer as Harald mentioned, or when
> building morphisms, or ...).
>
> That being said, we certainly don't want to decorate every Sage
> methods at once. So the database approach, with the databased filled
> up using doctests, is sensible indeed. Florent already used that
> approach to recover all the implementations of a given category
> (e.g. answer to the question: what are all the groups implemented in
> Sage).
>
> Cheers,
> Nicolas
> --

> Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
> http://Nicolas.Thiery.name/
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--

[D. S. McNeil]

>> As far as I understand, Python 3 will allow to achieve the same using
>> the natural syntax `def f(x: int, y: float) : ...`, with flexibility
>> on what "type" can be.
>
> Unfortunately, type declarations, with that syntax, was *proposed* for
> Python 3, but then soundly rejected by Guido as "too ugly".

I have good news -- I think you're misreading the current status:

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(a: int, b: float) -> (float, str):
... pass
...
>>> f.__annotations__
{'b': <class 'float'>, 'return': (<class 'float'>, <class 'str'>),
'a': <class 'int'>}


Doug

[William Stein]

Ah, interesting. I guess Guido just didn't like part of the
notation. Note that the declaration doesn't actually do anything:

>>> def f(a: int, b: float) -> (float, str):

... print(a,b)
...
>>> f(2.5,3)
2.5 3

[D. S. McNeil]

> Ah, interesting. I guess Guido just didn't like part of the
> notation. Note that the declaration doesn't actually do anything:
>
>>>> def f(a: int, b: float) -> (float, str):
> ... print(a,b)
> ...
>>>> f(2.5,3)
> 2.5 3

Yeah, IIRC enforcing this was a non-starter, and the decision was to
give a syntax and a storage structure and leave any kind of action up
to the user. This also makes all sorts of creative non-type
annotations possible, although admittedly I haven't really had cause
to use any of them yet.


Doug

[Jason Grout]

On 1/31/13 4:37 PM, William Stein wrote:
> Unfortunately, type declarations, with that syntax, was*proposed* for

> Python 3, but then soundly rejected by Guido as "too ugly".

That was referring to adding annotations about *generators*, according
to the PEP (specifically using 'yields', 'is_sent', etc. in the
annotations). See the email message linked from the PEP:
http://mail.python.org/pipermail/python-3000/2006-May/002103.html

Jason