options to <sage> elements?

[Randall Pruim]

The documentation on <sage> at https://pretextbook.org/doc/guide/html/basics-s-sage-cell.html is pretty minimal.  I discovered that language="python" does what I would expect, and cells in a section are linked, but autoeval="true" does not appear to work.  Is there a way to get a cell to auto-evaluate on page load?

Here is an example use case.  I have a an appendix covering python code used throughout the text.  Many (most) of the examples in that appendix require that a few python packages be loaded (numpy, scipy.linalg, ...).  I'd like to have a cell or two at the top of the section to do the necessary imports and auto-runs.  These can (and should) be visible, but I'd also be interested in other sagemath options like @hide.  My concern is that for people jumping to a particular item for reference (perhaps from the index), they might not see those cells and wonder why the code doesn't work, or they might remember that those cells are there, but not want to scroll to the top to execute them and then back down to the one command they are interested in at the moment.

WIP at https://rpruim.github.io/ula-ds/subsec-frequently-used-python.html

PS.  I see some older threads from 2 and more years ago that are somewhat related (and mostly about hiding), but I can't tell if things resolved.

[Rob Beezer]

There are some basic problems in how the Sage Cell Server configures itself. If
I remember right, something like linking and hiding and auto-evaluation have
mutually exclusive portions.

But your request is a long-standing one:

https://github.com/PreTeXtBook/pretext/issues/484

Taking an existing page of PreTeXt HTML output, and modifying it slightly to add
a new feature can be a great way to get new code. Hint, hint. ;-) Post the
old and the new pages so a diff makes it clear what changes are needed.

Rob

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-suppo...@googlegroups.com
> <mailto:pretext-suppo...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pretext-support/35b6af93-39bc-4f22-9ce4-52508ebec06an%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/35b6af93-39bc-4f22-9ce4-52508ebec06an%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Randall Pruim]

Full example is at https://github.com/rpruim/pretext-poc

The README explains what I did. 

Note: This is likely exactly how this should be handled.  In particular, it would be nicer to handle a number of sagemath options systematically rather than to create classes for various combinations of options.  In particular, it should be possible to auto-evaluate a sage cell of any language type, whether hidden or visible.  But somehow one needs to be able to identify which sage cells should be set up with which options. I've not dug into the pretext xml internals enough to know what the best mechanism is for handling this.

You may also be interested in this PR to imathas which enabled similar things for sage cells embedded in MyOpenMath: https://github.com/drlippman/IMathAS/pull/399

Here's the diff between the output pretext currently generates and a hand-edited version that auto-evaluates the first cell (as tested on my localhost).

> diff output/web/section-title.html output/web-alt/section-title.html                                                (base)
67a68,76
>
> // NB: auto-eval must be handled first because it is a "sub-type" of python as currently implemented
> // NB: linkKey is used to link the python cells (but not other sage cells that might be on the same page)
> sagecell.makeSagecell({inputLocation: 'pre.sagecell-autoeval',
>                        linked: true,
>                        linkKey: 'python',
>                        languages: ['python'],
>                        autoeval: true,
>                        evalButtonText: 'Evaluate (Python)'});
69a79
>                        linkKey: 'python',
71a82
>
227c238
< <pre class="ptx-sagecell sagecell-python" id="sage-1"><script type="text/x-sage">import numpy as np
---
> <pre class="ptx-sagecell sagecell-python sagecell-autoeval" id="sage-1"><script type="text/x-sage">import numpy as np

[Randall Pruim]

Oops.  Third sentence should read like NOT how this should be handled.

[kcri...@gmail.com]

The README explains what I did. 

Note: This is likely exactly how this should be handled.  In particular, it would be nicer to handle a number of sagemath options systematically rather than to create classes for various combinations of options.  In particular, it should be possible to auto-evaluate a sage cell of any language type, whether hidden or visible.  But somehow one needs to be able to identify which sage cells should be set up with which options. I've not dug into the pretext xml internals enough to know what the best mechanism is for handling this.

I agree with you in https://groups.google.com/g/pretext-support/c/hS2Mi0bk3N0/m/04KJ-4HgAgAJ that attributes are the way to go on this.  

In fact, currently we have

    <xsl:if test=".//sage[@type='display']">
        <xsl:call-template name="sagecell-display" />
    </xsl:if>

    <xsl:if test=".//sage[@type='practice']">
        <xsl:call-template name="sagecell-practice" />
    </xsl:if>

I don't know if this is well documented, though.  The only reference I could find is 

https://pretextbook.org/examples/sample-article/annotated/section-sage-cells.html but you have to read the code, the text refers only to type "invisible" explicitly as an attribute.

The types supported in actual code seem to be:

* invisible - doctest only

* display - uneditable, but you can evaluate it

* practice - apparently just an empty cell to practice in, for ease of coding

* full - default.  This *used* to be documented in the code (see https://github.com/PreTeXtBook/pretext/commit/bb74d11786fedbfc3850fcb349da12c5a48a23bd) but was apparently deleted in a commit I'm not going to search for now.

In addition, in the comments of the sample article, you will find (https://github.com/PreTeXtBook/pretext/blob/master/examples/sample-article/sample-article.xml#L1569C1-L1581C16)

            <!-- This is not implemented
            This Sage code will be (a) hidden and (b) automatically executed
            <sage type="init">
                <input>
                def square(m):
                    return m*m

                aaa= 78
                print aaa
                </input>
            </sage>
            -->

so at least in principle this is the right way to go.  More "type"s, and more documented.

On a related note, some comments:

* There also used to be a "copy" attribute which is no longer available, I think.

* https://github.com/PreTeXtBook/pretext/issues/501 suggests removing the "practice" type, though this wasn't implemented

* Output? https://github.com/PreTeXtBook/pretext/issues/123

[Rob Beezer]

Having "sage" elements was one of the motivations for PreTeXt in the first
place, so the code is pretty old. Good evidence for that is the @type attribute
having a lame name. So it could stand a refactor. I may be into it in the
morning for braille, so will take a close look then.

Rob

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-suppo...@googlegroups.com
> <mailto:pretext-suppo...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pretext-support/5c9e40f9-7593-44dd-87fc-8d7a751d6246n%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/5c9e40f9-7593-44dd-87fc-8d7a751d6246n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Randall Pruim]

Rob,

This is great news.  Let me know if you need anything more from me. 

I assume you are aware of https://github.com/sagemath/sagecell/blob/master/doc/embedding.rst#customization.  It would be good to support as many of the options there as make sense in the context of a pretext document.  As for naming attributes, I'd suggest mimicking the names used there as much as possible.

[Randall Pruim]

I'm nudging this again because late August is getting close and I'd like to be able to finish up my text before classes start.

Any more thoughts on passing more options to <sage>?  

As a reminder, I'd like the option to

* have a cell auto-execute when the page loads

* have cells share an environment

Both of these (and some other things, like changing the text on the execute button) are supported by the sage API.

Example use case: An initial cell loads a couple python packages used throughout a section so these don't need to be included in every cell.  Also the ability to build up to a solution across several cells without re-entering the code from previous cells.

[Juan Carlos Bustamante]

Hello to everyone,

Perhaps this is superficial, but let me point it anyway.

The font size used in the sage-cell is, in my opinion, too large compared with the text. As a consequence, if one has a not-so-complex sage entry, the cell cannot be seen without scrolling. It also forces break lines. See the attached screenshot

Perhaps nothing can be done about this, but perhaps there is some hope also.

Kind regards,

JC

[Rob Beezer]

The Sage Cell server uses CodeMirror to format and display the code itself. I suspect this can be controlled with CSS. We may even do some of this already, but I am not in a good position to check just now at 40,000 feet.

Rob

On July 31, 2023 6:55:40 AM PDT, Juan Carlos Bustamante <jotac...@gmail.com> wrote:
>Hello to everyone,
>
>Perhaps this is superficial, but let me point it anyway.
>
>The font size used in the sage-cell is, in my opinion, too large compared with the text. As a consequence, if one has a not-so-complex sage entry, the cell cannot be seen without scrolling. It also forces break lines. See the attached screenshot
>

[Rob Beezer]

Dear Randall,

Just back from two weeks away.

Bug fixes are top-priority. Then pull requests. Working through several of
each right now.

I won't bore you with this month's deadlines for a grant proposal, a final
report for a current grant, and a deadline for developer recruitment. All part
of running an active project.

Feature requests rarely come with deadlines. Except a current one for a project
sponsor, that had a 6-month lead time.

I'm well aware of your request and your upcoming course. Before adding more
technical debt, this one will also require a careful study of the potential for
a refactor.

Rob

On 7/28/23 09:29, Randall Pruim wrote:
> I'm nudging this again because late August is getting close and I'd like to be
> able to finish up my text before classes start.
>
> Any more thoughts on passing more options to <sage>?
>
> As a reminder, I'd like the option to
>
> * have a cell auto-execute when the page loads
> * have cells share an environment
>
> Both of these (and some other things, like changing the text on the execute
> button) are supported by the sage API.
>
> Example use case: An initial cell loads a couple python packages used throughout
> a section so these don't need to be included in every cell.  Also the ability to
> build up to a solution across several cells without re-entering the code from
> previous cells.
>
>
> On Friday, June 30, 2023 at 7:43:37 AM UTC-4 Randall Pruim wrote:
>
> Rob,
>
> This is great news.  Let me know if you need anything more from me.
>
> I assume you are aware of

> https://github.com/sagemath/sagecell/blob/master/doc/embedding.rst#customization <https://github.com/sagemath/sagecell/blob/master/doc/embedding.rst#customization>.  It would be good to support as many of the options there as make sense in the context of a pretext document.  As for naming attributes, I'd suggest mimicking the names used there as much as possible.

> https://pretextbook.org/examples/sample-article/annotated/section-sage-cells.html <https://pretextbook.org/examples/sample-article/annotated/section-sage-cells.html> but you have to read the code, the text refers only to type "invisible" explicitly as an attribute.

> >
> > The types supported in actual code seem to be:
> > * invisible - doctest only
> > * display - uneditable, but you can evaluate it
> > * practice - apparently just an empty cell to practice in, for ease
> of coding
> > * full - default.  This *used* to be documented in the code
> > (see

> https://github.com/PreTeXtBook/pretext/commit/bb74d11786fedbfc3850fcb349da12c5a48a23bd <https://github.com/PreTeXtBook/pretext/commit/bb74d11786fedbfc3850fcb349da12c5a48a23bd>) but was apparently deleted in a commit I'm not going to search for now.

> >
> > In addition, in the comments of the sample article, you will find
> >

> (https://github.com/PreTeXtBook/pretext/blob/master/examples/sample-article/sample-article.xml#L1569C1-L1581C16 <https://github.com/PreTeXtBook/pretext/blob/master/examples/sample-article/sample-article.xml#L1569C1-L1581C16>)

> https://groups.google.com/d/msgid/pretext-support/5c9e40f9-7593-44dd-87fc-8d7a751d6246n%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/5c9e40f9-7593-44dd-87fc-8d7a751d6246n%40googlegroups.com> <https://groups.google.com/d/msgid/pretext-support/5c9e40f9-7593-44dd-87fc-8d7a751d6246n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pretext-support/5c9e40f9-7593-44dd-87fc-8d7a751d6246n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-suppo...@googlegroups.com
> <mailto:pretext-suppo...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pretext-support/7794e8f5-2f09-425b-bb5e-73415ca0b2fcn%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/7794e8f5-2f09-425b-bb5e-73415ca0b2fcn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Rob Beezer]

Dear JC,

You have hijacked a totally unrelated thread (other than being about Sage).

It is less likely to get lost if you were to make an issue on GitHub. Thanks.

Rob

[Randall Pruim]

Rob,

Thanks for the update and I hope it was a good couple of weeks away.  I didn't necessarily expect a fix in time for my course if it was a challenging fix, I mostly wanted to know what I should be expecting and also if there is any way I can be of help.

[kcri...@gmail.com]

Thanks for the update and I hope it was a good couple of weeks away.  I didn't necessarily expect a fix in time for my course if it was a challenging fix, I mostly wanted to know what I should be expecting and also if there is any way I can be of help.

A way to do that, in this case, might be to make a branch of pretext with a *non*-refactored version that captures the type of cell you'd like to make, and then to put this as a "draft" pull request on GH that could be commented on by developers (once it's working, which I don't think will be very much trouble for you) without it having the priority factor of an "actual" pull request (unsure on the exact terminology).  That gives you something that works, and gives Rob et al. something to factor out from.

[Randall Pruim]

I finally spent a little time digging into the pretext xsl internals.  (Super handy that it is easy to replace the core xsl with local versions using the CLI just by creating a new build target.). 

Yesterday a created modified versions of pretext-html.xsl and pretext-common.xsl that implement autoevaluation of sage cells using @autoeval='true'.  I may make another pass before submitting a pull request for Rob et al to take a look at.  But I already have something that minimally does what I need for my upcoming course, so this isn't urgent.  (If anyone else is interested in this in the short-term, let me know.)

My current implementation works by modifying the class assigned to <pre> elements.  The current class includes something like sagecell-python or sagecell-octave.  My version amends this to sagecell-python-true or sagecell-python-false (boolean indicator for whether or not to autoevaluate).  That was quick and easy for a POC.  Slightly nicer looking might be sagecell-python-autoeval and sagecell-python-noautoeval, but that probably makes the xsl messier.  Another option would be to separate this out into separate classes: sagecell-python sagecell-autoeval, etc.

Are their any guiding principles here? Should I just submit a PR with a working POC and let others on the team clean things up or provide feedback?

So far I've only looked at HTML and no other formats.

[Randall Pruim]

One other note, in case someone has information to contribute.  I noticed in my first experiments that if there are multiple auto-evaluated cells, the output for all of them shows up in the same output cell.  I'm not sure if that is a sagecell issue or a problem with my xsl.  For my use case, this isn't a bit deal since (a) I'll typically be using this to load packages, so there isnt' any ouput or (b) I'll have a single auto-evaluated cell to get things rolling in a section.  But it is unexpected behavior, and I can imagine use cases where this would not be desirable.  So ideally, I'd like to improve this.

[Rob Beezer]

Dear Randall,

Thanks very much for the exploratory work - I've been thunking about this one
the past few days.

> I'm not sure if that is a sagecell issue or a problem with my xsl.

Hard to say without seeing the XSL in this case. I could speculate, but...
Instead, an in-progress or proof-of-concept pull request is always welcome.
Pile-on commits or force pushes don't bother me there.

> So far I've only looked at HTML and no other formats.

There's the dilemma of whether or not this code should be visible in a static
format. If a few imports, then I'd say yes. A whole library of "helper"
routines to make life easier for a reader, then I'd say no. Do we need another
attribute? @show-autoeval? Or maybe @autoeval becomes "no", "hide", "show"? Or?

Another aspect I have not explored is this "linkKey" option. We've had trouble
through the years with Sage cells buried in knowls - either at birth or as the
target of a cross-reference. So this seems a valuable (new) option, but it
comes with some trepidation.

Suppose a Sage cell is part of a proof, and earlier on the page is some setup
code. Now, somewhere else (on another page) you cross-reference the theorem
that has the proof. The reader follows the cross-reference by clicking on it
and the theorem opens up, and the proof is in a knowl, and the reader opens
that. Do we expect the setup code to be active for that cell in the proof? We
have a similar problem with linked cells, but if the linked-ness is more local
and the reader learns to execute in order, then it is workable.

Rob

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-suppo...@googlegroups.com
> <mailto:pretext-suppo...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pretext-support/759ee1c4-1482-4998-bb98-15d7a6fc4458n%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/759ee1c4-1482-4998-bb98-15d7a6fc4458n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Randall Pruim]

Since you don't seem opposed to iterating on this a bit, I'll get a PR issued, perhaps after one more round of edits.  Right now, I've only implemented the autoeval for python chunks.  There is a lot of redundant code for the different languages, and I wondered if some refactoring might be able to clean that up.

BTW, I implemented linkedKey as well to explicitly link cells using the same language.  It wouldn't be too hard to turn this into an option that the user could select to control how things are linked.

I'll admit to not fully understanding all the infrastructure around knowls, but perhaps I can at least create a test case by putting a sage cell into a proof or into something that gets indexed.

Sage allows for cells that can select a language.  The current pretext wrappers seem to be targeting single-language use cases.  Is the intent to keep it that way?

Eventually, I'm hoping something like this will work and do what I think is pretty clearly intended.

<sage language='python' autoeval='true' linkKey = 'my-linked-python-stuff' editor='codemirror-readonly' evalButtonText='Re-run this code!'>

I'm not sure about the hide option.  Any thoughts on that?

[Rob Beezer]

Dear Randall,

> There is a lot of redundant code for the different
> languages, and I wondered if some refactoring might be able to clean that up.

Yes, that is the sort of thing that is going to make getting into this more
complicated than getting one course ready to go.

> I'll admit to not fully understanding all the infrastructure around knowls

Me too. Which is why I used the word "trepidation". ;-)

> <sage language='python' autoeval='true' linkKey = 'my-linked-python-stuff'
> editor='codemirror-readonly' evalButtonText='Re-run this code!'>

A fundamental goal is to shield authors from the complexity of setting up
add-ins like this. While still retaining much of the power and flexibility.
Its a balancing act.

So if @editor is to make a cell un-editable, then maybe we just have a flag for
that. But doesn't this go in the setup/head and influence multiple cells? (I
haven't looked.)

Superficially, giving authors the linkKey sounds like it is over the line and
would create a lot of confusion. I think my approach has been to link
everything on the page that has the same language. So multiple linked sets, but
the @language is defining the partition. And then the language in effect has
been expressed by the text on the button, so a reader can see the partition. Of
course, this requires the author to impose constraints on how granular chunking
into pages can go.

Rob

> https://groups.google.com/d/msgid/pretext-support/759ee1c4-1482-4998-bb98-15d7a6fc4458n%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/759ee1c4-1482-4998-bb98-15d7a6fc4458n%40googlegroups.com> <https://groups.google.com/d/msgid/pretext-support/759ee1c4-1482-4998-bb98-15d7a6fc4458n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pretext-support/759ee1c4-1482-4998-bb98-15d7a6fc4458n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-suppo...@googlegroups.com
> <mailto:pretext-suppo...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pretext-support/0a0a2edf-4407-49bb-ab9d-4fe02eac952en%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/0a0a2edf-4407-49bb-ab9d-4fe02eac952en%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Randall Pruim]

Given what you say here, I think I'll issue a PR with a fairly minimal POC.  I did do a bit more work, but I think it would be better to have some more discussion before I continue down that path.  I've stashed that on a local branch for now.

The PR only does Python, but it would be straightforward to replicate this for the other languages.  (Nicer still if we could come up with a way to do this with less redundant code.)

I think a flag for uneditable cells could be a fine solution here.  Sage actually has two editors, each with editable and non-editable versions. But I don't know how important it is to allow for all four.

My original thought was to simply pass things along using the same names the sage uses, but I can see why a simpler interface in pretext could make sense.

| But doesn't this go in the setup/head and influence multiple cells? (I haven't looked.)

The sagecell api works by using an xpath to select elements and then assigning all the same options on those matching elements.  So you can have different options for different cells, but you need to be able to distinguish them somehow.

| I think my approach has been to link everything on the page that has the same language. So multiple linked sets, but the @language is defining the partition.

That certainly seems like a good default, and perhaps it the only reasonable use case.  Also, I'll have to look more carefully to see how you were linking by language. It may be that what I was thinking to do and what you are already doing aren't compatible.

Regarding non-editable cells -- that seems like it could be useful for autoevaluated cells. But I don't know if that's always what one would want.  And I could imagine cells that are not autoevaluated that shouldn't be edited.  It would be nice if there was a way to restore the original contents of a cell (without refreshing the page and resetting all the cells). But the sagecell API doesn't have that feature, so we'd have to implement on the pretext side...

I'll issue the PR in a couple minutes.  Do you want to move subsequent discussion to GitHub?

Thanks again for thinking about this.

[Rob Beezer]

On 8/12/23 09:20, Randall Pruim wrote:
> I'll issue the PR in a couple minutes.  Do you want to move subsequent
> discussion to GitHub?

Thanks, Randall!

Follow us to

https://github.com/PreTeXtBook/pretext/pull/2041

for more discussion.

Rob