Martin Albrecht pointed out to me that there is a way to markup
citations and references in ReST (and that I should be using that in
my patches). I was so happy to see this that I immediately fixed my
patch and added a few words to the developer guide to this effect (see
http://trac.sagemath.org/sage_trac/ticket/7456 for this).
It would seem that it's a fairly simple habit to get into: instead of
def do_this():
"""
Return really great thing, based on [ABC].
REFERENCES:
- [ABC] The ABC of great things
"""
you would write
def do_this():
"""
Return really great thing, based on [ABC]_.
REFERENCES:
.. [ABC] The ABC of great things
"""
However, there are some issues in doing this on the large scale that
is the Sage library.
1. These references used to be local to the docstring they appear in.
As soon as we ReST-ify them, they become global in the reference
manual. Therefore if there is already a reference labeled [ABC],
Sphinx will rightfully complain. That's easy to fix, just use a
different label.
2. What happens if we cite the same reference from two different
docstrings? We have to pick which one docstring will contain the
"definition" of the reference, otherwise Sphinx will complain about
duplicates. The effect of this is:
a) in the html output, the citation itself is a link to the reference,
so it's easy to get to it
b) in the pdf output, all the references are *only* listed in the
bibliography section toward the end of the document. This means that
there are a bunch of lines that just say "REFERENCES" and nothing else
in the docstrings. Not very useful or pretty, so it would be great if
we could somehow remove them.
c) in the docstring itself (obtained by introspection or by reading
the code directly), the citation will be there saying "see [ABC]" but
the actual reference could be nontrivial to find. In fact, the only
way to find it that's sure to work is to do search_src(".. [ABC]"),
because the definition could be in a different file, for instance.
I'm not sure I like c), but maybe that's the price that we have to pay
for having beautiful and consistent-looking documentation?
Alex
--
Alex Ghitza -- Lecturer in Mathematics -- The University of Melbourne
-- Australia -- http://www.ms.unimelb.edu.au/~aghitza/
This is one of the many, many cases where I think it would be easier
to fix the tool rather than change people's behavior. Why not have our
cake and eat it too? [ABC] can automatically be turned into [ABC]_
(perhaps only if it matches something below). Global name conflicts
can automatically be resolved, either by picking the first (if they're
"close enough") or making them unique. The REFERENCES section can be
stripped if its empty.
I reallly like ReST, and it's a huge step ahead of what we used to
have, but (and I've said this before) in many ways using raw ReST
seems like a step backwards (double ::'s for doctests, ` for math
mode, double-spacing function argument specs, etc.)
- Robert
> 1. These references used to be local to the docstring they appear in.
> As soon as we ReST-ify them, they become global in the reference
> manual. Therefore if there is already a reference labeled [ABC],
> Sphinx will rightfully complain. That's easy to fix, just use a
> different label.
>
> 2. What happens if we cite the same reference from two different
> docstrings? We have to pick which one docstring will contain the
> "definition" of the reference, otherwise Sphinx will complain about
> duplicates. The effect of this is:
[...]
One good habits which will probably solve a large part of the duplicate it to
put the reference not in the doctring of the methods or function but in that
of the module = file. It is very likely that several methods/function or
related to the same paper will appear. It is as well likely that they appear
in the same class/file. Also, it's coherent to put them close to the AUTHOR:
part since both are some kinds of acknowledgment (one for the mathematical
idea, one for the implantation).
[...]
Florent
I strongly agree with this. In fact, last week when I was hanging out
with Jarrod Millman (of scipy/numpy), he baited me -- "so, are you
guys using Sphinx a lot for Sage yet?" When I said "yes!", he
responded, "so how did you guys solve the references problem, since
references are a total mess in Sphinx?" And said that we hadn't yet.
Of course, it was inevitable that we would run head first into this
problem eventually, and it's amusing that we did this week.
Anyway, I agree with Robert -- to solve this particular problem we
will have to go creatively beyond what Sphinx offers "out of the box".
William
I'm not sure this will be a good thing--it might make a cleaner
reference manual but I think docstrings are most used via the powerful
introspection that's available in Sage, and we want to put as much as
possible (e.g. references) right at the user's fingertips.
- Robert
> > One good habits which will probably solve a large part of the
> > duplicate it to
> > put the reference not in the doctring of the methods or function but
> > in that
> > of the module = file. It is very likely that several methods/
> > function or
> > related to the same paper will appear. It is as well likely that
> > they appear
> > in the same class/file. Also, it's coherent to put them close to the
> > AUTHOR:
> > part since both are some kinds of acknowledgment (one for the
> > mathematical
> > idea, one for the implantation).
>
> I'm not sure this will be a good thing--it might make a cleaner
> reference manual but I think docstrings are most used via the powerful
> introspection that's available in Sage, and we want to put as much as
> possible (e.g. references) right at the user's fingertips.
>
I strongly agree with Robert here, I wouldn't want to break
introspection. Moreover, this doesn't solve the problem of the same
reference being cited from different files (and this is not just a
hypothetical situation, I've run into it with steenrod_algebra.py and
steenrod_algebra_element.py, and I think it also comes up in the
graphs code).
Best,
Since we're not the only ones having this sort of problem, I wonder if
we can convince the Sphinx developers to do something about it. We
would probably have a better chance of doing this if we had a good
idea of how we would like references to work in an ideal world.
(A quick look at Sphinx' trac didn't yield any relevant tickets. I'll
try to have a look at their mailing list as well to see if anything
was brought up.)
Best,
(1) Almost all of the docstrings in Sage look like this:
INPUT:
- `n` - an integer
- `m` - another integer
- `k` - a third integer, with a longer description
with spaces. I'm not sure why, but I think Mike Hansen argued that he
had to do things this way when he was transition from our old
non-Sphinx docs. Evidently Sphinx has changed since then (or he was
wrong).
(2) Regarding your example:
"""
INPUT:
- n - an integer
- m - another integer
- k - a third integer, with a
longer description
"""
I just want to point out that the space before the "longer" above is
*critical*. Moreover, it has to be at least *two* spaces, i.e., the
start of the word "longer" has to line up with the "k" above (your
example maybe didn't).
>
> Or you can write
>
> INPUT:
>
> :param n: an integer
> :param m: another integer
> :param k: a third integer, with a
> longer description
>
>
>> I strongly agree with this. In fact, last week when I was hanging out
>> with Jarrod Millman (of scipy/numpy), he baited me -- "so, are you
>> guys using Sphinx a lot for Sage yet?" When I said "yes!", he
>> responded, "so how did you guys solve the references problem, since
>> references are a total mess in Sphinx?" And said that we hadn't yet.
>
> In what way are they a total mess? I'm curious.
My impression is that it was in exactly the way that is being
discussed in this thread. The way Jarrod put it was "we need a Bibtex
for Sphinx", i.e., a way to have a database of references that is
separate from the individual docstrings. That's basically the same
problem being discussed in this thread.
>
>> Of course, it was inevitable that we would run head first into this
>> problem eventually, and it's amusing that we did this week.
>>
>> Anyway, I agree with Robert -- to solve this particular problem we
>> will have to go creatively beyond what Sphinx offers "out of the box".
>
> Which particular problem?
Problem = the lack of a "Bibtex for Sphinx".
> Alex just pointed out that we haven't been
> using standard Sphinx/reST for citations, and we should. That's easy
> to solve "in the box".
>
> More generally, we don't use pure Python in Sage -- we've changed it
> so that ^ means exponentation -- but it's pretty close to pure, and it
> seems that we're reluctant to change it much. I feel the same way
> about Sphinx or reST: it seems like a bad idea to just change it when
> we happen to not like the syntax. Sage is a computer program, and it
> is reasonable to ask people to learn Python syntax to write Sage
> programs, and I think it is also reasonable to ask people to learn
> reST to write docstrings. It's not that hard, and it makes our
> docstrings portable.
My understanding is that the problem is that nobody has yet written
something like Bibtex for Sphinx. That's a bit of a different problem
than just changing syntax. How would you write large latex
papers/books without Bibtex? Ick.
-- William
Here's an example (I'm not picking on you, John, this was just the
first instance I ran into this after doing search_src("REFERENCE") to
look for un-Sphinx-ified references.)
In the module docstring of steenrod_algebra.py, there is a reference
to Milnor's Annals paper [Mil]. I eagerly put this in proper Sphinx
syntax. Moving on to steenrod_algebra_element.py, I had to remove the
same reference from the module docstring because Sphinx complained.
So you can have something like "see [Mil]_" but then the actual
reference is nowhere to be found in that file.
This means that someone reading only this docstring does not have this
information at hand any more. Then we get to
steenrod_algebra_bases.py, where the Monks and Wood papers are now [M]
and [W], whereas in steenrod_algebra_element.py they were [Mon] and
[Woo]. Sphinxify these and you get duplicate references
with different names and labels.
Once again, I'm not picking on you or anybody else. But I think this
shows that it's hard to keep consistency alive even in cases where a
single author wrote a bunch of files, never mind what happens when ten
different people work on the same code. It adds to the author's
workload and to the reviewer's workload. And as we pointed out
already, it cripples docstrings introspection.
Best,
There is an enhancement ticket for making Sphinx read Bibtex files:
http://bitbucket.org/birkenfeld/sphinx/issue/63/make-sphinx-read-bibtex-files-for
There seems to have been some activity on it, but the latest was 10
months ago.
Interesting. I imagined more that one would create an _analogue_ of
Bibtex for Sphinx, rather than have Sphinx actually read Bibtex.
After all, Bibtex entries can contain fairly complicated LaTeX, and
there are subtle rules (e.g., incollection...). Of course, those
rules are all important for actual applications.
-- William