Shortcoming in Jupyter Notebook w.r.t. latex in help

[Erik Bray]

This question raised an issue I was not aware of (and didn't see any
existing tickets for):

https://ask.sagemath.org/question/41592/inside-help-not-processed/#

When viewing docs for Sage objects in the Jupyter Notebook, they are
just displayed as plain text--this is a major shortcoming over SageNB
which renders the latex in the online help.

What's worse, in the Notebook the docs are passed through the same
"de-texification" that is used on the command-line to make some tex
readable in plain ASCII. There are two issues here:

1) The de-texification is obviously imperfect. The particular example
contained \longrightarrow which it didn't know what to do with.
Easily fixable, but beside the point. It will never be 100% perfect
(it also looks like it could benefit tremendously from Unicode support
which has never been added to it).

2) It goes without saying that in the Notebook help should be rendered
by MathJax. It's not clear to me that the Jupyter Notebook actually
knows that it can/should run MathJax over the help window, but
currently it does not appear to run MathJax over the help panel (even
if I explicitly wrap equations in dollar signs or something).

Has anyone else looked into this?

Thanks,
E

[Emmanuel Charpentier]

Le lundi 19 mars 2018 11:44:33 UTC+1, Erik Bray a écrit :

This question raised an issue I was not aware of (and didn't see any
existing tickets for):

https://ask.sagemath.org/question/41592/inside-help-not-processed/#

When viewing docs for Sage objects in the Jupyter Notebook, they are
just displayed as plain text--this is a major shortcoming over SageNB
which renders the latex in the online help.

What's worse, in the Notebook the docs are passed through the same
"de-texification" that is used on the command-line to make some tex
readable in plain ASCII.  There are two issues here:

1) The de-texification is obviously imperfect.  The particular example
contained \longrightarrow which it didn't know what to do with.
Easily fixable, but beside the point.  It will never be 100% perfect
(it also looks like it could benefit tremendously from Unicode support
which has never been added to it).

De-texification is *hard* : see the various solutions enumerated here...

2) It goes without saying that in the Notebook help should be rendered
by MathJax.  It's not clear to me that the Jupyter Notebook actually
knows that it can/should run MathJax over the help window, but
currently it does not appear to run MathJax over the help panel (even
if I explicitly wrap equations in dollar signs or something).

It can be done. See for an (approximate) example what do the two available R Jupyter kernels (respectively IRkernel and Juniper). I'ts only approximate, since R help pahges use their own sui generis markup (Rd).

A possibly easier alternative would go the route \LaTeX ==> DVI ==> HTML, as proposed by the (many) "solutions" proposed there.

[Erik Bray]

On Mon, Mar 19, 2018 at 12:16 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
>
>
> Le lundi 19 mars 2018 11:44:33 UTC+1, Erik Bray a écrit :
>>
>> This question raised an issue I was not aware of (and didn't see any
>> existing tickets for):
>>
>> https://ask.sagemath.org/question/41592/inside-help-not-processed/#
>>
>> When viewing docs for Sage objects in the Jupyter Notebook, they are
>> just displayed as plain text--this is a major shortcoming over SageNB
>> which renders the latex in the online help.
>>
>> What's worse, in the Notebook the docs are passed through the same
>> "de-texification" that is used on the command-line to make some tex
>> readable in plain ASCII. There are two issues here:
>>
>> 1) The de-texification is obviously imperfect. The particular example
>> contained \longrightarrow which it didn't know what to do with.
>> Easily fixable, but beside the point. It will never be 100% perfect
>> (it also looks like it could benefit tremendously from Unicode support
>> which has never been added to it).
>
>
> De-texification is *hard* : see the various solutions enumerated here...

Yes... The solution we have is useful; could be improved; but that's
a never-ended process.

>> 2) It goes without saying that in the Notebook help should be rendered
>> by MathJax. It's not clear to me that the Jupyter Notebook actually
>> knows that it can/should run MathJax over the help window, but
>> currently it does not appear to run MathJax over the help panel (even
>> if I explicitly wrap equations in dollar signs or something).
>
>
> It can be done. See for an (approximate) example what do the two available R
> Jupyter kernels (respectively IRkernel and Juniper). I'ts only approximate,
> since R help pahges use their own sui generis markup (Rd).
>
> A possibly easier alternative would go the route \LaTeX ==> DVI ==> HTML, as
> proposed by the (many) "solutions" proposed there.

I'm not sure what you mean here. We already *have* HTML versions of
the Sage docs installed with Sage. For example if something like
Manifold.diff_form? could return a path to
http://doc.sagemath.org/html/en/reference/manifolds/sage/manifolds/differentiable/manifold.html?highlight=diff_form#sage.manifolds.differentiable.manifold.DifferentiableManifold.diff_form
and display that in the Notebook's help window, then our job is done
:)

I'm just not sure if it's possible to do that through Jupyter's API,
but I'll have to look at what the R kernels you pointed me to do.

[Erik Bray]

It looks like, just from reading the Jupyter docs, this where we can
control what '?' returns at the kernel level:
http://jupyter-client.readthedocs.io/en/stable/messaging.html#introspection

So if Sage could just return a snippet of the appropriate HTML docs
(with MathJax enabled, etc.) then that should do the trick. I'm not
sure about the details though.

I wonder if CoCalc has already done this...

[Erik Bray]

On Mon, Mar 19, 2018 at 12:16 PM, Emmanuel Charpentier
<emanuel.c...@gmail.com> wrote:
>
>

Thanks for pointing those out. I looked at the source code for
Juniper and indeed found the part where it handles introspection:

https://github.com/JuniperKernel/JuniperKernel/blob/master/R/inspect_request.R

So it basically returns the help as HTML. It seems we can actually
return a multi-part MIME message with all sorts of data in it (images,
etc.) as well.

[Dima Pasechnik]

Have you tried asking Jupyter people/github how this can be done---I cannot imagine Sage is the only Python library

which has docstrings with MathJax ready to be used in them...

 

[Eric Gourgoulhon]

Hi,

Another issue involving the Jupyter notebook and LaTeX is that typeset outputs, which are correctly rendered with MathJax in the browser, are no longer typeset when exporting the notebook to pdf:
https://trac.sagemath.org/ticket/23330

Probably the two issues are not related (or maybe they are?), but I am taking the opportunity of this thread to recall the above problem, which is a serious limitation of pdf export IMHO.

Eric.

[William Stein]

On Mon, Mar 19, 2018 at 4:42 AM, Erik Bray <erik....@gmail.com> wrote:
> It looks like, just from reading the Jupyter docs, this where we can
> control what '?' returns at the kernel level:
> http://jupyter-client.readthedocs.io/en/stable/messaging.html#introspection
>
> So if Sage could just return a snippet of the appropriate HTML docs
> (with MathJax enabled, etc.) then that should do the trick. I'm not
> sure about the details though.
>
> I wonder if CoCalc has already done this...

No, it's equally bad in CoCalc.

-- William

--
William (http://wstein.org)

[Erik Bray]

Ah, too bad. I've been reading the Jupyter kernel documentation and
looking at some examples, and I think I have some ideas of how this
can be vastly improved (in short, mainly by looking up the Sphinx docs
for objects in Sage (sometimes extracting just a section from the
page) and sending those over. I have a rough idea how this can be
done but it will take some tinkering. I've opened an issue for it
here:

https://trac.sagemath.org/ticket/25015