webwork-id modal template

[Alex Jordan]

In pretext-assembly, there is this template:

<xsl:template match="webwork" mode="webwork-id">
<xsl:choose>
<xsl:when test="@xml:id">
<xsl:value-of select="@xml:id"/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="local-name(.)" />
<xsl:text>-</xsl:text>
<xsl:number from="book|article|letter|memo" level="any" />
</xsl:otherwise>
</xsl:choose>
</xsl:template>


In researching how to speed up some things, I've discovered this one
is a hog. That "xsl:number" is the problem.

Does anyone (well, I'm mainly asking Rob, but anyone really) have
insight for how to automatically assign a unique, repeatable id to
each "webwork" in a faster way?

This all happens within the mode="labeling" template. I considered
giving that template a parameter that can be tunneled down (in the
imprecise XSLT 1.0 sense) and used more directly to make an id, but I
don't want to mess with that template without consulting with Rob
first.

[Alex Jordan]

I see the "DELAY" note in the mode="labeling" template. I suspect that
once that delay is lifted, I'll have what I'm after. Is that more of a
near horizon delay or a distant horizon delay at this point?

[Rob Beezer]

Yes, I know this ID is a hog. I can't tell you how much time I've spent on IDs
and pre-processing. And how many commits that have been tossed. WeBWorK being
a two-pass thing is part of it, iirc. Anyway, I had to just move on and leave
that be as it is.

I literally just added YAID this week. Commit:
00b10586c261a7c8f59f5ac885ef9774e4b72eeb. I particularly like it, though it is
not immune to source changes (neither is the one being discussed presently). It
is serving a very particular purpose, but maybe it could have broader applicability?

Maybe a fresh look, the passage of time, and an additional sympathetic set of
eyes, would yield improvements. Can we discuss the code together on Friday?
That would be a small delay. ;-)

[Alex Jordan]

I might not make it this week. Veterans Day and kids out of school and all. But the following week or month  would still be a small delay.

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/0bd6a8c8-2f9e-3cc5-b86f-1d7dc4fe087a%40ups.edu.

[Rob Beezer]

On 11/9/22 20:16, Alex Jordan wrote:
> Veterans Day

Forgot about that! I'll try to survey the sitauation sometime soon and be ready
when you are.

Rob

[Alex Jordan]

I'll outline an idea I have, that is mostly your idea (as I understood
it from an earlier thread). We could discuss here, or wait until we
next meet.

-assembly has all these phases:
version
assembly
representations
repair
identification
language
augment
exercise

Right now "labeling" is inserted after "version". But suppose
"labeling" is done to the tree after every phase. As the tree is
walked, every element gets two attributes:
@phase-when-address-was-applied = "version" (or "assembly",
"representations", etc.)
@address-at-time-of-labeling = "mychapter-s3t5"
("mychapter" is the nearest self-or-ancestor @xml:id, and s3t5 is
encoding the 3rd section, 5th theorem)

Labeling like this only happens to elements that do not already have
these attributes.

In the end when a tree is passed to a main stylesheet, we have a fast
way to know:
* at what phase each element was introduced
* what its "address" was at that phase
The combination of these could make a relatively stable identifier. For example:
"mychapter-s3t5-version"
or
"mysection-ch2wr1-representations"
(the 1st webwork-representations from the second checkpoint in
"mysection", which was introduced to the tree during "representations"
phase of -assembly.)

> --
> You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/d7b084d8-f582-9d48-69e9-aead4183a2f3%40ups.edu.

[Rob Beezer]

Alex and I had a very long and very technical and very helpful joint reading of
the code during Drop-In Friday. So this is mostly for Alex.

The "visible-id" template is the big hog, PreTeXt-wide, not just for "webwork".

So I bugged it to get the new (temporary-ish) @original-id before doing the
(slow) document-wide counting of the old scheme. That caught almost everything.
One big exception is the "fn" automatically added after a "url". Just a few
other items to chase down.

This suggests that adding "fn" in the "repair" phase is too late. Suppose we
move it sooner, before any sort of ids are built. Then a static webwork
representation will be located in the representations file (via an id), after
the manufacture of a automatic "fn" on a "url".

(I've rewritten much of the above several times already.)

Maybe we need a fast ID just for "webwork"? Employed like the slow one, but
fast to build? It could be the same scheme as for the "original-id", but saved
in a special-purpose attribute (@ww-id!). It could be set after big chunks of
source are processed by the pre-processor (versions), and *right before*
"webwork" is processed (both Pass 1 and Pass 2). Then more general-purpose id
construction could wait until much later (after various representations enter -
WW and Runestone) and other manipulations. Perhaps last.

Any red flags?

[Alex Jordan]

> Maybe we need a fast ID just for "webwork"?

> Any red flags?

It could be OK, but I hesitate because it feels uncomfortable to run
with a webwork-specific solution.

What are your thoughts on adding id's more than one time during asssemmbly?
Goal: end the assembly gauntlet with every instance of certain
elements having an id.
Constraint: At least one phase of -assembly (representations) needs
the id~s to already be present. So there must be an early labeling.
Constraint: Some phases of assembly add new elements to the tree.
Including the representations phase. So there must be a late labeling.

So it's a Catch 22 if you are only labeling once. But you could label
once, early, and then make a point of labeling each new thing that
gets added. After each phase that may have added new elements, you
walk the entire tree and add id's to those elements that don't have an
id.

Say we have author source:
<p>
text
<url href="foo"/>
text
<m>x</m>
</p>

After labeling, we have:
<p label="mychapter-p1>
text
<url href="foo" label="mychapter-p1u1"/>
text
<m label="mychapter-p1m2">x</m>
</p>

After repair, we have:
<p label="mychapter-p1>
text
<url href="foo" label="mychapter-p1u1"/>
<fn pi:url="bar"/>
text
<m label="mychapter-p1m2">x</m>
</p>

But after "repair labeling" we could have:
<p label="mychapter-p1>
text
<url href="foo" label="mychapter-p1u1"/>
<fn pi:url="bar" label="mychapter-p1f2-repair"/>
text
<m label="mychapter-p1m2">x</m>
</p>

Everything gets a unique label. Now the "2" in "mychapter-p1m2" is a
little misleading, if that matters. But it's not inaccurate if you
exclude things that have @pi:url from the preceding-sibling axis.

Also if that "fn" was added during repair, would we get anything out
of having an @pi:assembly="repair"? Other things could have
@pi:assembly="representations". It would clearly indicate that
something was added, and at which phase.

> To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/8d26ba9b-9f0e-f439-4d45-0eb1cdb0c15a%40ups.edu.

[Rob Beezer]

On 11/12/22 23:30, Alex Jordan wrote:
>> Maybe we need a fast ID just for "webwork"?
>> Any red flags?
>
> It could be OK, but I hesitate because it feels uncomfortable to run
> with a webwork-specific solution.

We get static MyOpenMath from a server, but each problem has a MOM id, so there
is no trouble coordinating things. And no translating between markup languages.

We build static versions of Runestone problems, but it is totally a
PreTeXT-internal-XSL process, translation, but no server communication.

WeBWorK has PreTeXt markup, which we translate to PGML, which we sent to a
server to get back static. And no natural sort of id connected to the server
(unless PTX does it). And I think we want to "repair" what comes back from old
servers that do not produce un-deprecated PreTeXt. (We contol static versions
of Runestone, so shame on us if we make old PreTeXt.)

So I don't think this is a WW thing - I think WW is just our only instance of
several moving parts all rolled into one.

Your discussion below is spot on. But the solution feels too brittle to me. I
loved the addition of the "fn" to a "url" when I did it, but it is exposing lots
of delicate matters, only partially because it can show up almost anywhere. To
make it exceptional almost defeats the purpose.

Thought du jour. One "webwork" pass/mode. Early. Compute an id as discussed.
Always happens the same place in the process, so consistent. If "extracting"
(Pass 1), then make any @copy. Perhaps make repairs later in process. Convert
to PGML, send to server, get back a representations file. But, if not
"extracting", then retrieve desired representation from the representations file
(ids are in sync!), and perhaps repair static later. Then do "real" ids last,
during what I call "augmentation" -- adding attributes that save later repeated
computation (like numbering and tagging the many categories of "exercise").

We could have made (or still could) extraction a standalone process. Put an id
on each "webwork" as part of "original" source (first!) and make PGML and send
to the server and get it back into a file of its own. Then insert
representations from the right file (based on an id computed at the same early
time as before). I'm thinking the process *could* look more like MOM, except
that we extract and make PGML (more work), and we ignore any notion of versions,
which in some extreme cases might mean more WW processing than necessary. I'm
not proposing this - only something to think about down the line.

[PreTeXt development]

Status report, mostly for Alex.

Consolidated all the WW passes into one early template as part of the "assembly" pass, with a bespoke identifier, matching current identifier.   Then moved all general-purpose labeling/identification late, and recycled a no-op early template to just do the webwork manipulations.  No changes to output.  In a position to "simply" switch the identifier to something new and fast.

All on a branch.  Need to work back in some error messages before it can be pushed.  But should be a no-change refactor for authors and publishers.

Then, in a happy accident, I noticed a fast identifier I just added for the "View Source" knowls, being created right before the WW pass.  So as an experiment I replaced the slow identifier with the fast one already being computed.  Total chaos in diffs for intermediate products (eg representations file) but final products (HTML, LaTeX) just show piles of name changes in the right places.  Needs more checking, and would need to be released at a good time for folks to adjust, but I'm optimistic we can eliminate *all* identifier generation for WW.  Much like your suggestion - an early identifier and a late identifier.

Rob