17 views

Skip to first unread message

Sep 16, 2010, 12:39:51 AM9/16/10

to sage-devel

(Say that subject three times fast.)

I'm looking at a ticket whose main purpose is to apply the

@rename_keyword decorator in lots of places, outside of the plot

module, where it was first employed.

When I build the HTML version of the reference manual, it would appear

these functions are now listed simply and universally as foo(*args,

**kwds) and any automatic information about the exact nature of the

real arguments is lost. If the docstrings were to list this

information carefully, at least it would be there, but the docstrings

are not always so careful.

I believe there is a similar behavior with the @options decorator,

though I have not investigated as thoroughly. Again, these options

are not always listed in the docstring. I think this explains why

sometimes it hard to tell just what plotting options are available.

In the case of @rename_keyword there is automatically a deprecation

warning. So someday, the decorator will be pulled and the

documentation will go back to a more informative version.

I find the generic version of the function definitions less than

satisfactory. I'd guess it would be had to make Sphinx pickup the

more detailed info in these situations? I'd also guess the decorators

could maybe manipulate the docstring and inject some information based

on the arguments of the decorator? Either way, could the effect of

these decorators on the documentation be improved?

Rob

I'm looking at a ticket whose main purpose is to apply the

@rename_keyword decorator in lots of places, outside of the plot

module, where it was first employed.

When I build the HTML version of the reference manual, it would appear

these functions are now listed simply and universally as foo(*args,

**kwds) and any automatic information about the exact nature of the

real arguments is lost. If the docstrings were to list this

information carefully, at least it would be there, but the docstrings

are not always so careful.

I believe there is a similar behavior with the @options decorator,

though I have not investigated as thoroughly. Again, these options

are not always listed in the docstring. I think this explains why

sometimes it hard to tell just what plotting options are available.

In the case of @rename_keyword there is automatically a deprecation

warning. So someday, the decorator will be pulled and the

documentation will go back to a more informative version.

I find the generic version of the function definitions less than

satisfactory. I'd guess it would be had to make Sphinx pickup the

more detailed info in these situations? I'd also guess the decorators

could maybe manipulate the docstring and inject some information based

on the arguments of the decorator? Either way, could the effect of

these decorators on the documentation be improved?

Rob

Sep 16, 2010, 3:10:27 AM9/16/10

to sage-...@googlegroups.com

> I find the generic version of the function definitions less than

> satisfactory. I'd guess it would be had to make Sphinx pickup the

> more detailed info in these situations? I'd also guess the decorators

> could maybe manipulate the docstring and inject some information based

> on the arguments of the decorator? Either way, could the effect of

> these decorators on the documentation be improved?

> satisfactory. I'd guess it would be had to make Sphinx pickup the

> more detailed info in these situations? I'd also guess the decorators

> could maybe manipulate the docstring and inject some information based

> on the arguments of the decorator? Either way, could the effect of

> these decorators on the documentation be improved?

Yes, that is possible -- decorators already have to "steal"

documentation from whatever they decorate. I think that we should

require decorators to document their existence and effect.

--tom

Sep 16, 2010, 11:06:16 AM9/16/10

to sage-devel

@CachedFunction decorator hide documentation: if you have

@CachedFunction

def bozo(...):

Then bozo doesn't appear in the reference manual. I've considered

doing

def bozo_(...):

bozo = CachedFunction(bozo_)

Then bozo_ appears in the ref manual and bozo is what you actually

call. But it's annoying.

--

John

Sep 16, 2010, 11:11:06 AM9/16/10

to sage-...@googlegroups.com

> To post to this group, send an email to sage-...@googlegroups.com

> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com

> For more options, visit this group at http://groups.google.com/group/sage-devel

> URL: http://www.sagemath.org

>

Perhaps changing the decorators to make use of the decorator package

(http://micheles.googlecode.com/hg/decoratior/documentation.html and

http://pypi.python.org/pypi/decorator), which preserve call signatures

and docstrings, would help?

--

Tim Joseph Dumol <tim (at) timdumol (dot) com>

http://timdumol.com

Sep 16, 2010, 1:16:32 PM9/16/10

to sage-devel

Thanks for the pointer, Tim. That looks very promising - I'll try

some experiments when I get a chance.

And I'd forgotten all about John's struggles with @CachedFunction.

The documentation for the decorator package has an example that is a

"memoize" decorator that seems to have an identical purpose as Sage's

caching decorators. Maybe the example would provide some insight, or

be a simple verbatim test case to see if it comes through into the

documentation.

Rob

On Sep 16, 8:11 am, Tim Joseph Dumol <t...@timdumol.com> wrote:

> Perhaps changing the decorators to make use of the decorator package

> (http://micheles.googlecode.com/hg/decoratior/documentation.htmlandhttp://pypi.python.org/pypi/decorator), which preserve call signatures

> and docstrings, would help?

some experiments when I get a chance.

And I'd forgotten all about John's struggles with @CachedFunction.

The documentation for the decorator package has an example that is a

"memoize" decorator that seems to have an identical purpose as Sage's

caching decorators. Maybe the example would provide some insight, or

be a simple verbatim test case to see if it comes through into the

documentation.

Rob

On Sep 16, 8:11 am, Tim Joseph Dumol <t...@timdumol.com> wrote:

> Perhaps changing the decorators to make use of the decorator package

> and docstrings, would help?

Sep 17, 2010, 3:51:28 PM9/17/10

to sage-devel

On 16 sep, 17:06, John H Palmieri <jhpalmier...@gmail.com> wrote:

> I don't know if this is the same issue, but I think I've also seen the

> @CachedFunction decorator hide documentation: if you have

>

> @CachedFunction

> def bozo(...):

>

> Then bozo doesn't appear in the reference manual. I've considered

> doing

>

> def bozo_(...):

>

> bozo = CachedFunction(bozo_)

>

> Then bozo_ appears in the ref manual and bozo is what you actually

> call. But it's annoying.

It's just a workaround, but what about
> I don't know if this is the same issue, but I think I've also seen the

> @CachedFunction decorator hide documentation: if you have

>

> @CachedFunction

> def bozo(...):

>

> Then bozo doesn't appear in the reference manual. I've considered

> doing

>

> def bozo_(...):

>

> bozo = CachedFunction(bozo_)

>

> Then bozo_ appears in the ref manual and bozo is what you actually

> call. But it's annoying.

def bozo(...):

...

bozo = CachedFunction(bozo)

This is, you define a function and assign it the "bozo" identifier,

and then set the "bozo" identifier to point to the cached version of

the old "bozo".

Sep 17, 2010, 6:00:01 PM9/17/10

to sage-devel

reference manual.

--

John

Sep 18, 2010, 2:44:42 AM9/18/10

to sage-devel

Hi!

On 18 Sep., 00:00, John H Palmieri <jhpalmier...@gmail.com> wrote:

> ...

"temporary_result" that caches methods of a ring approximation (i.e.,

generators and relations are only known out to a certain degree), but

clears the cache if more generators or relations are found in higher

degree.

So, in the code, I have, e.g.:

@temporary_result

def poincare_series(self):

"""

Return the Poincar\\'e series...

...

"""

...

"poincare_series" DOES appear in the package's manual (see

http://sage.math.washington.edu/home/SimonKing/Cohomology/cohomology.html#pGroupCohomology.cohomology.COHO.poincare_series).

Moreover, the decorator also manages to decorate the given docstring,

so that it becomes

Temporarily cached method: Return the Poincar’e series...

The decorator is a callable class, and a decorated method is an

instance of this class. This instance can apparently borrow the name

and the docstring (even adding a modification) from the method.

Since the decorators in Sage (more precisely: the cached method

decorator) were my source of inspiration, I am surprised that

@CachedFunction is problematic for the documentation.

Cheers,

Simon

On 18 Sep., 00:00, John H Palmieri <jhpalmier...@gmail.com> wrote:

> ...

> Unfortunately, this makes any entry for "bozo" disappear from the

> reference manual.

Why is this? For my group cohomology spkg, I wrote some decorator
> reference manual.

"temporary_result" that caches methods of a ring approximation (i.e.,

generators and relations are only known out to a certain degree), but

clears the cache if more generators or relations are found in higher

degree.

So, in the code, I have, e.g.:

@temporary_result

def poincare_series(self):

"""

Return the Poincar\\'e series...

...

"""

...

"poincare_series" DOES appear in the package's manual (see

http://sage.math.washington.edu/home/SimonKing/Cohomology/cohomology.html#pGroupCohomology.cohomology.COHO.poincare_series).

Moreover, the decorator also manages to decorate the given docstring,

so that it becomes

Temporarily cached method: Return the Poincar’e series...

The decorator is a callable class, and a decorated method is an

instance of this class. This instance can apparently borrow the name

and the docstring (even adding a modification) from the method.

Since the decorators in Sage (more precisely: the cached method

decorator) were my source of inspiration, I am surprised that

@CachedFunction is problematic for the documentation.

Cheers,

Simon

Sep 23, 2010, 8:13:26 AM9/23/10

to sage-devel

Hi

> I find the generic version of the function definitions less than

> satisfactory. I'd guess it would be had to make Sphinx pickup the

> more detailed info in these situations? I'd also guess the decorators

> could maybe manipulate the docstring and inject some information based

> on the arguments of the decorator? Either way, could the effect of

> these decorators on the documentation be improved?

I agree that this is completely unacceptable. I have created Trac 9976

and

written a patch for Sage's custom version of Sphinx. This essentially

checks if

a function/method to be documented is a decorator by looking for the

attribute

"_sage_decorating". If a function/method is actually a decorated

version of

another callable, then this callable should be referenced to in

_sage_decorating. With my patch, Sphinx then constructs the signature

from this

callable instead.

All decorators (on documented functions/methods) should then set this

attribute; in particular, sage_wraps does so (with my patch), and I

guess that

every decorator aimed at documented functions/methods should use

sage_wraps.

Ideally, noone should therefore again have to know about this

workaround.

This seems to work for every function and method I have looked at.

However, it

is a bit tedious to force Sphinx to do the rebuild. It seems that you

have to

manually touch all py-files, do a complete rebuild (sage -ba) and then

rebuild

the documentation.

I hope that you agree this solution is sensible.

Simon King wrote:

> Why is this? For my group cohomology spkg, I wrote some decorator

> "temporary_result" that caches methods of a ring approximation (i.e.,

> generators and relations are only known out to a certain degree), but

> clears the cache if more generators or relations are found in higher

> degree.

>

> So, in the code, I have, e.g.:

> @temporary_result

> def poincare_series(self):

> """

> Return the Poincar\\'e series...

> ...

> """

> ...

>

> "poincare_series" DOES appear in the package's manual (see

> http://sage.math.washington.edu/home/SimonKing/Cohomology/cohomology....).

The problem is not that the functions won't appear but that they will

have the

signature (argument list) of the decorator; for decorators which can

decorate a

function with any argument, this will often be the generic signature

"<function-name>(*args, **kwargs)". In your case, your method does not

take any

arguments (except for self), and neither does the decorator, so you

will see no

difference in the documentation.

> Moreover, the decorator also manages to decorate the given docstring,

> so that it becomes

> Temporarily cached method: Return the Poincar’e series...

>

> The decorator is a callable class, and a decorated method is an

> instance of this class. This instance can apparently borrow the name

> and the docstring (even adding a modification) from the method.

>

> Since the decorators in Sage (more precisely: the cached method

> decorator) were my source of inspiration, I am surprised that

> @CachedFunction is problematic for the documentation.

The docstring and name is inherited by the decorator by using Python's

"wraps"

decorator inside the decorator; isn't that what you did? We need a

similar

mechanism for inheriting the signature. My patch does this by

requiring

decorators to set the attribute _sage_decorating, which is

automatically done

by using Sage's "sage.misc.decorators.sage_wraps" decorator (see also

Trac

#9907).

Cheers,

Johan

> I find the generic version of the function definitions less than

> satisfactory. I'd guess it would be had to make Sphinx pickup the

> more detailed info in these situations? I'd also guess the decorators

> could maybe manipulate the docstring and inject some information based

> on the arguments of the decorator? Either way, could the effect of

> these decorators on the documentation be improved?

and

written a patch for Sage's custom version of Sphinx. This essentially

checks if

a function/method to be documented is a decorator by looking for the

attribute

"_sage_decorating". If a function/method is actually a decorated

version of

another callable, then this callable should be referenced to in

_sage_decorating. With my patch, Sphinx then constructs the signature

from this

callable instead.

All decorators (on documented functions/methods) should then set this

attribute; in particular, sage_wraps does so (with my patch), and I

guess that

every decorator aimed at documented functions/methods should use

sage_wraps.

Ideally, noone should therefore again have to know about this

workaround.

This seems to work for every function and method I have looked at.

However, it

is a bit tedious to force Sphinx to do the rebuild. It seems that you

have to

manually touch all py-files, do a complete rebuild (sage -ba) and then

rebuild

the documentation.

I hope that you agree this solution is sensible.

Simon King wrote:

> Why is this? For my group cohomology spkg, I wrote some decorator

> "temporary_result" that caches methods of a ring approximation (i.e.,

> generators and relations are only known out to a certain degree), but

> clears the cache if more generators or relations are found in higher

> degree.

>

> So, in the code, I have, e.g.:

> @temporary_result

> def poincare_series(self):

> """

> Return the Poincar\\'e series...

> ...

> """

> ...

>

> "poincare_series" DOES appear in the package's manual (see

The problem is not that the functions won't appear but that they will

have the

signature (argument list) of the decorator; for decorators which can

decorate a

function with any argument, this will often be the generic signature

"<function-name>(*args, **kwargs)". In your case, your method does not

take any

arguments (except for self), and neither does the decorator, so you

will see no

difference in the documentation.

> Moreover, the decorator also manages to decorate the given docstring,

> so that it becomes

> Temporarily cached method: Return the Poincar’e series...

>

> The decorator is a callable class, and a decorated method is an

> instance of this class. This instance can apparently borrow the name

> and the docstring (even adding a modification) from the method.

>

> Since the decorators in Sage (more precisely: the cached method

> decorator) were my source of inspiration, I am surprised that

> @CachedFunction is problematic for the documentation.

"wraps"

decorator inside the decorator; isn't that what you did? We need a

similar

mechanism for inheriting the signature. My patch does this by

requiring

decorators to set the attribute _sage_decorating, which is

automatically done

by using Sage's "sage.misc.decorators.sage_wraps" decorator (see also

Trac

#9907).

Cheers,

Johan

Sep 23, 2010, 8:57:39 AM9/23/10

to sage-...@googlegroups.com

On 9/23/10 7:13 AM, Johan S. R. Nielsen wrote:

> Hi

>

>> I find the generic version of the function definitions less than

>> satisfactory. I'd guess it would be had to make Sphinx pickup the

>> more detailed info in these situations? I'd also guess the decorators

>> could maybe manipulate the docstring and inject some information based

>> on the arguments of the decorator? Either way, could the effect of

>> these decorators on the documentation be improved?

>

> I agree that this is completely unacceptable. I have created Trac 9976

> and

> written a patch for Sage's custom version of Sphinx. This essentially

> checks if

> a function/method to be documented is a decorator by looking for the

> attribute

> "_sage_decorating". If a function/method is actually a decorated

> version of

> another callable, then this callable should be referenced to in

> _sage_decorating. With my patch, Sphinx then constructs the signature

> from this

> callable instead.

> Hi

>

>> I find the generic version of the function definitions less than

>> satisfactory. I'd guess it would be had to make Sphinx pickup the

>> more detailed info in these situations? I'd also guess the decorators

>> could maybe manipulate the docstring and inject some information based

>> on the arguments of the decorator? Either way, could the effect of

>> these decorators on the documentation be improved?

>

> I agree that this is completely unacceptable. I have created Trac 9976

> and

> written a patch for Sage's custom version of Sphinx. This essentially

> checks if

> a function/method to be documented is a decorator by looking for the

> attribute

> "_sage_decorating". If a function/method is actually a decorated

> version of

> another callable, then this callable should be referenced to in

> _sage_decorating. With my patch, Sphinx then constructs the signature

> from this

> callable instead.

In the case of an @options decorator, it would be really cool if the

function signature could be modified to show the options, or in the case

of a @suboptions decorator, show the suboptions. Is there an easy way

to have a decorator change the signature in the docs? For example,

could we make a _sage_signature attribute that the decorator could

change, and use that for the docs?

Thanks,

Jason

Sep 24, 2010, 2:40:31 AM9/24/10

to sage-devel

First of all, sorry for the terrible word-wrapping in my previous

post; it seems Google groups wraps at less than 80 characters.

Thats a very nice extension; I guess all sorts of uses for this could

be imagined.

With the current patch, there is no easy way for the signature to be

changed. One would have to define a new function with the appropriate

signature inside the decorator and assign to _sage_decorating, as

_sage_decorating contains only a reference to a callable. Originally,

I wanted to use a _sage_signature like you suggest instead, but the

problem was in constructing such a signature. The most practical thing

would probably be to construct a signature as a named tuple such as

that returned by inspect.getargspec and then let Sphinx format this to

text. However, as can be seen by the code in

doc.common.sage_autodoc.py surrounding my patches, it takes Sphinx

quite a bit of logic to extract the signature, and I wanted to avoid

having to copy that. Especially references to custom C-function

signature-extractors scared me a bit.

There is, of course, the possibility to have both custom attributes,

and let _sage_signature override _sage_decorating in case both are

defined. It still leaves the problem of having to construct these

signatures in the general case.

I think the idea is good, but I'm not completely sure how to do it

elegantly.

Cheers,

Johan

post; it seems Google groups wraps at less than 80 characters.

be imagined.

With the current patch, there is no easy way for the signature to be

changed. One would have to define a new function with the appropriate

signature inside the decorator and assign to _sage_decorating, as

_sage_decorating contains only a reference to a callable. Originally,

I wanted to use a _sage_signature like you suggest instead, but the

problem was in constructing such a signature. The most practical thing

would probably be to construct a signature as a named tuple such as

that returned by inspect.getargspec and then let Sphinx format this to

text. However, as can be seen by the code in

doc.common.sage_autodoc.py surrounding my patches, it takes Sphinx

quite a bit of logic to extract the signature, and I wanted to avoid

having to copy that. Especially references to custom C-function

signature-extractors scared me a bit.

There is, of course, the possibility to have both custom attributes,

and let _sage_signature override _sage_decorating in case both are

defined. It still leaves the problem of having to construct these

signatures in the general case.

I think the idea is good, but I'm not completely sure how to do it

elegantly.

Cheers,

Johan

Oct 5, 2010, 7:48:49 AM10/5/10

to sage-devel, jason...@creativetrax.com

On Sep 23, 2:57 pm, Jason Grout <jason-s...@creativetrax.com> wrote:

Ok, I succeeded in writing a slightly less intuitive version of the

patch which allows for something like the suggested for the @options

and @suboptions decorators. For example, the Sphinx-built argument

line for the function sage.plot.contour_plot.contour_plot becomes

sage.plot.contour_plot.contour_plot(f, xrange, yrange, axes=False,

linestyles=None, frame=True, labels=False, plot_points=100,

linewidths=None, colorbar=False, contours=None, fill=True, **options)

However, this does not fix everything at once. For example, the

function sage.plot.contour_plot.implicit_plot accepts two arguments

which are defaulted to None (linewidth and linestyle), so the author

did not care to add them to the options-decorator. Therefore, they

will not show up in any documentation.

If this sounds like what you wanted, I can clean up the code, test it

some more and add a new patch to #9976.

Cheers, Johan

Oct 5, 2010, 10:02:42 AM10/5/10

to Johan S. R. Nielsen, sage-devel

That sounds great! I've been looking at the decorator patches over the

last few days, but haven't had time to do a complete review yet. I hope

to review them by the deadline for 4.6.

Thanks,

Jason

Reply all

Reply to author

Forward

0 new messages

Search

Clear search

Close search

Google apps

Main menu