Validation improvements

[Rob Beezer]

I am working a lot on the schema, and now on ways we can encourage authors to
apply it more often.

I would really appreciate feedback on the latest. Can you take some time to
take this for a spin for me? I'm interested in how the tool works - specific
questions about the schema and your project can wait a bit.

pretext/pretext script, "-V" argument will validate. There are three methods
("-M").

* local (the default)
* local-dev (uses "dev" version of the schema)
* server (not usable right now, plan to work on it)

You need to have a "jing" executable. I'm not going to say anymore about
getting started, so if

Subsection E.1.1: Validation via the pretext Script
https://pretextbook.org/doc/guide/html/jing-trang.html#jing-pretext-script

is not helpful or is confusing, say so.

The Python has a method called "agent" which a tool could use in automated way
to refine its production of valid PreTeXt. Hmmmmmmmmm.

Subsection E.1.2: Validation for Programs
https://pretextbook.org/doc/guide/html/jing-trang.html#validation-agent

@Mark F - I'd be real curious if you can get this to go with your very long jing
invocation.

@Oscar - the signature for "validate()" has changed. I suspect it will not
change further. Please do not integrate all this into the CLI just yet, I
consider it still "in progress". We can plan more at Drop-In.

Rob

[Jeremy Sylvestre]

Nice! Here are a few quick impressions:

1) IMHO the -validation.txt and -assembled.xml files should not be created in the current working directory from which the script was invoked. Instead they should be deposited at the same directory path as the root source file that was passed to the script on the command line.

2) In the path info line for a specific error, it's nice that it tells you the index of that element in the tree, but all of the [1]'s create a lot of visual clutter. (I don't really need to know that the leading /pretext element is the first of its kind, as there is only one.) Is it possible to omit the [1] in the case where that element is the only one of its kind at that path?

3) I'm getting a lot of false positives of the following error:

PTX:WARNING: A run of text contains Unicode characters for double quotation marks (U+201C, decimal 8220; U+201D, decimal 8221). Likely this was introduced in a conversion of source material authored in a word-processor. A matching pair U+201C, U+201D should be replaced by the "<q>" element enclosing content. In rare cases, U+201C might be replaced by the empty element "<lq/>". In rare cases, U+201D might be replaced by the empty element "<rq/>".

From a quick scan, it seems to be triggering mostly on the quotation marks around the following attributes:

image/@width

latex-image/@label

In at least one case it triggered on some quotation marks inside a <!-- comment -->.

In all of the above cases, these were *not* unicode double quotation marks, but just plain old ASCII hex code 0x22 (verified with a hex editor).

4) I'm also getting some

PTX:WARNING: A run of text contains a Unicode character for an en-dash (U+2013, decimal 8211). Likely this was introduced in a conversion of source material authored in a word-processor. The en-dash will behave better as the empty element "<ndash/>". Understand the difference between an en-dash and an em-dash before editing. An en-dash is used for a range, such as the years 2013-22.

but the only place my source contains en-dash characters is in the copyright notice headers in each file (within comments). The text included in the error message in the -validation.txt file does not contain a dash of any kind.

Maybe the validation is getting confused by the en-dash in the copyright notice? Maybe this is also what's happening with the double-quotes error messages, as my copyright notice also contains unicode quote symbols:

A copy of the license is included in the appendix entitled “GNU Free Documentation License” that appears in the output document of this PreTeXt source code.

But again this is all within an XML comment. (Or, in the case of a tikz image, within a LaTeX comment.)

Cheers,

Jeremy S

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/MTAwMDAxOC5iZWV6ZXI.1783275402%40pnsh.

[Jeremy Sylvestre]

Just following up: I'm pretty confident the source of both the unicode quotation marks warnings and the unicode en-dash warnings are in fact from copyright notices inside LaTeX comments from included .tex files. (I was initially thrown off by the text: line in the warning message, which only contained the line of the parent element.)

In the case of a <latex-image> file that xi:includes a .tex file containing tikz source, in the assembled file all of this ends up inside <![CDATA[ ...

But I've also got some xi:includes inside docinfo/macros to include .tex files containing macros, and inside the assembled file this does *not* end up inside <![CDATA[.  But again the offending characters are inside LaTeX comments.

So I guess it would be nice to ignore LaTeX comment lines, if possible. If not, maybe include a switch to suppress the checks for unicode characters.

[David Austin]

I ran ULA through the new validation process.  In spite of the very large number of messages (gulp), I found the output easy to understand and usable.  I can usually see what the problem is just from the message, and the filename and line number in the assembled version helps me navigate there.  I only had three occurrences of the Unicode character message, all three about quotation marks.

You said you weren't ready for specific comments on the schema, but I found a number of places where my valid PreTeXt was marked invalid.  I have a list of these when you're ready.

David

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CALMYOncU3PbjW2ArXW%2B2i1LutS4PZ8mPNp3FJMJugkbrC7%2B3fQ%40mail.gmail.com.

[Andrew Scholer]

Agree, easy to use, possibly intimidating amount of output (schema for RS interactives is in rough shape), but easy enough to read.

For integration purposes, is there a way to restrict processing to a certain subtree (other than filtering after generating all of it)? i.e. if I am working in source/pointers-aggregation/pointers-null.ptx that contains a section pointers-aggrecation_pointers-null, is there a way to only scan that part of the document?

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CANXmVMDk5G3hjfGLK1LvowbXdLDBFHDsW0_v%3Dj5XX88UwoqRqw%40mail.gmail.com.

[Sean Fitzpatrick]

Validation fails for me due to the use of components.

I get an lxml.etree.XMLSyntaxError due to a duplicate xml:id. It's a bit
of a false positive, because the duplicate ids are in different
components, so there are no duplicates post-assembly.

Let me see if I can cludge my way around this. I'll have to use a
different ID whenever I have the same item in different components.
(This usually arises because a figure needs to be placed at a different
location in the book in different versions.)

The main issue is going to be that when that duplicate id gets xref-ed I
may need to also duplicate the xrefs, and put those in components as well!

[Sean Fitzpatrick]

OK. Only two duplicate IDs to deal with, so not so bad. I had to repeat
a paragraph in two components to use different xml:ids in each.

A general note: line numbers seem to be off. Maybe jing does not count
empty lines, but my text editor does?

I get a lot of errors that I think should not be errors. This might
point to places where the schema needs to be updated?

- In my docinfo, I am told that "document-id" and "blurb" are not
allowed, but these are needed for Runestone. Maybe I need to put these
in a Runestone component so they don't show up for non-Runestone builds?

- It says that "component" is not allowed as an attribute on
latex-image-preamble and asymptote-preamble. But this is necessary if
you want to maintain both colour and black-and-white versions.

- It objects to the use of @label on the titlepage element, but this is
needed for file-naming if I understand correctly.

- Lots of complaints because figure/tabular is not allowed. I think this
is correct, but to be honest, I don't remember how this one is resolved.
APEX has many tabulars that don't meet the threshold of a proper table.

- It flags a lot of the interactive exercise stuff as invalid:
<response/> is not allowed. It does not allow @width or @mode or @answer
on a fillin. <evaluation> is not allowed. The attribute @correct is not
allowed on a <statement>. <feedback> is not allowed, cardsort is not
allowed, etc.

- solution and hint are not allowed as a child of webwork, but they
should be

- the schema allows @label on a latex-image, but *not* when it is a
descendant of webwork

- pg-macros is not allowed as a child of webwork, but it should be,
unless something has changed

- Validation-plus is throwing a *lot* of false positives about image
short descriptions being over 125 characters long


I didn't get through all of validation plus. There was a lot.

[David Austin]

I think that #description is being confused with #shortdescription in a couple of places.  I have an #image with a #shortdescription but no #description, and the report says there is no alt-text.

Also, #directories (my favorite!) is not allowed anywhere and lots of problem types, like query, cardsort, choices, and blocks.

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/101b0155-dc46-4228-b29c-49945e1897de%40gmail.com.

[Mark Fitch]

I'd love to report on this but all I get is an error message.

$ ~/pretext/pretext/pretext -vv -V -p publication/publication.ptx
source/geometries_ibl.ptx
PTX:DEBUG   : Parsed CLI args {'verbose': 2, 'config_file': None,
'component': None, 'format': None, 'publication_file':
'publication/publication.ptx', 'stringparams': [], 'extra_stylesheet':
None, 'method': '', 'xmlid': '', 'server': None, 'data_dir': None,
'out': None, 'dir': None, 'abort': False, 'tgz': False, 'validate':
True, 'xml_file': 'source/geometries_ibl.ptx'}
PTX:DEBUG   : Python version: 3.12 (expecting 3.10 or newer)
PTX:DEBUG   : PreTeXt distribution and xsl directories:
C:\Users\drmaf\pretext, C:\Users\drmaf\pretext\xsl
PTX:INFO    : parsing possible configuration files:
['C:\\Users\\drmaf\\pretext\\pretext\\pretext.cfg',
'C:\\Users\\drmaf\\pretext\\user\\mbx.cfg',
'C:\\Users\\drmaf\\pretext\\user\\pretext.cfg']
PTX:DEBUG   : configuration files actually used/read:
['C:\\Users\\drmaf\\pretext\\pretext\\pretext.cfg']
PTX:INFO    : using default configuration only
PTX:INFO    : custom configuration file not used at
C:\Users\drmaf\pretext\user\pretext.cfg
PTX:INFO    : dictionary of executables/commands: {'latex': 'latex',
'pdflatex': 'pdflatex', 'xelatex': 'xelatex', 'asy': 'asy', 'mermaid':
'mmdc', 'sage': 'sage', 'pdfeps': 'pdftops', 'node': 'node', 'liblouis':
'file2brl', 'perl': 'perl', 'fop': 'fop', 'jing': 'jing'}
PTX:INFO    : verifying and expanding XML source file:
source/geometries_ibl.ptx
PTX:INFO    : input XML source file expanded to absolute path:
C:\DATA\Documents\texts\geometries_ibl\source\geometries_ibl.ptx
PTX:INFO    : verifying and expanding publisher file:
publication/publication.ptx
PTX:INFO    : input publisher file expanded to absolute path:
C:/DATA/Documents/texts/geometries_ibl/publication/publication.ptx
PTX:INFO    : discovering source file's directory name:
C:\DATA\Documents\texts\geometries_ibl\source
PTX:INFO    : verifying and expanding input directory:
C:\DATA\Documents\texts\geometries_ibl\source\../generated-assets
PTX:INFO    : input directory expanded to absolute path:
C:\DATA\Documents\texts\geometries_ibl\generated-assets
PTX:INFO    : verifying and expanding input directory:
C:\DATA\Documents\texts\geometries_ibl\source\../assets
PTX:INFO    : input directory expanded to absolute path:
C:\DATA\Documents\texts\geometries_ibl\assets
PTX:INFO    : Done examining environment and initializing setup info
PTX:DEBUG   : locating "jing" in [executables] section of configuration file
PTX:DEBUG   : jing executable: jing, options:
PTX:INFO    : converting
C:\Users\drmaf\AppData\Local\Temp\ptx-85zr4y7z\merged.xml to enhanced
(pre-processed) PreTeXt source as
C:\Users\drmaf\AppData\Local\Temp\ptx-85zr4y7z\assembled-attributed.xml
PTX:INFO    : XSL conversion of
C:\Users\drmaf\AppData\Local\Temp\ptx-85zr4y7z\merged.xml by
C:\Users\drmaf\pretext\xsl\utilities\pretext-enhanced-source.xsl
PTX:DEBUG   : XSL conversion via
C:\Users\drmaf\pretext\xsl\utilities\pretext-enhanced-source.xsl of
C:\Users\drmaf\AppData\Local\Temp\ptx-85zr4y7z\merged.xml to
C:\Users\drmaf\AppData\Local\Temp\ptx-85zr4y7z\assembled-attributed.xml
and/or into directory None with parameters {'assembly.file-attribution':
'yes', 'publisher':
'C:/DATA/Documents/texts/geometries_ibl/publication/publication.ptx',
'assembly.version-only': 'yes'}
PTX:INFO    : successful application of
C:\Users\drmaf\pretext\xsl\utilities\pretext-enhanced-source.xsl
PTX:DEBUG   : jing command: jing
C:\Users\drmaf\pretext\schema\pretext.rng
C:\DATA\Documents\texts\geometries_ibl\geometries_ibl-assembled.xml
Traceback (most recent call last):
  File "C:\Users\drmaf\pretext\pretext\pretext", line 986, in <module>
    main()
  File "C:\Users\drmaf\pretext\pretext\pretext", line 672, in main
    ptx.validate(xml_source, publication_file, stringparams, out_file,
dest_dir, method)
  File "C:\Users\drmaf\pretext\pretext\lib\pretext.py", line 4549, in
validate
    _validate_local(
  File "C:\Users\drmaf\pretext\pretext\lib\pretext.py", line 4662, in
_validate_local
    result = subprocess.run(full_cmd, capture_output=True, text=True)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program
Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2800.0_x64__qbz5n2kfra8p0\Lib\subprocess.py",
line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program
Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2800.0_x64__qbz5n2kfra8p0\Lib\subprocess.py",
line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Program
Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2800.0_x64__qbz5n2kfra8p0\Lib\subprocess.py",
line 1538, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified

[Rob Beezer]

On 7/5/26 13:10, Jeremy Sylvestre wrote:
> Nice! Here are a few quick impressions:
>
> 1) IMHO the -validation.txt and -assembled.xml files should not be created in
> the current working directory from which the script was invoked. Instead they
> should be deposited at the same directory path as the root source file that was
> passed to the script on the command line.

That beahvior is part of the simple interface to the Python routines and will
not be rearranged. The default is current directory, and the "-d" flag lets you
specify wherever you want. Once integrated into the CLI, it may be more to your
liking.

> 2) In the path info line for a specific error, it's nice that it tells you the
> index of that element in the tree, but all of the [1]'s create a lot of visual
> clutter. (I don't really need to know that the leading /pretext element is the
> first of its kind, as there is only one.) Is it possible to omit the [1] in the
> case where that element is the only one of its kind at that path?

Yes, that could be a nice improvement for teh human-readable output.

> 3) I'm getting a lot of false positives of the following error:

Well, I got only a few true positives for the sample article (a cut/paste job
gone bad, as predicted). Possible we are looking at comments, if so, that
should be stopped. Will investigate.

Thanks for the comments, that is exactly what I was looking for.

> /A copy of the license is included in the appendix entitled “GNU

> Free Documentation License” that appears in the output document of this PreTeXt

> source code./

>
> But again this is all within an XML comment. (Or, in the case of a tikz image,
> within a LaTeX comment.)
>
> Cheers,
> Jeremy S
>

> email to pretext-dev...@googlegroups.com <mailto:pretext-
> dev%2Bunsu...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/

> MTAwMDAxOC5iZWV6ZXI.1783275402%40pnsh <https://groups.google.com/d/msgid/
> pretext-dev/MTAwMDAxOC5iZWV6ZXI.1783275402%40pnsh>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to pretext-dev...@googlegroups.com <mailto:pretext-
> dev+uns...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/

> CALMYOnd8H9hYYGR_MycjPO53J49gtbM6WbpeCsR53xNQFfqO3g%40mail.gmail.com <https://
> groups.google.com/d/msgid/pretext-dev/
> CALMYOnd8H9hYYGR_MycjPO53J49gtbM6WbpeCsR53xNQFfqO3g%40mail.gmail.com?
> utm_medium=email&utm_source=footer>.

[Jeremy Sylvestre]

On Mon, Jul 6, 2026, 4:12 p.m. 'Rob Beezer' via PreTeXt development <prete...@googlegroups.com> wrote:

On 7/5/26 13:10, Jeremy Sylvestre wrote:
> Nice! Here are a few quick impressions:
>
> 1) IMHO the -validation.txt and -assembled.xml files should not be created in
> the current working directory from which the script was invoked. Instead they
> should be deposited at the same directory path as the root source file that was
> passed to the script on the command line.

That beahvior is part of the simple interface to the Python routines and will
not be rearranged.  The default is current directory, and the "-d" flag lets you
specify wherever you want.

My bad, didn't realize the "-d" flag could be applied to the output of this process. That will definitely work for me.

[Sean Fitzpatrick]

One of the places where I get complaints about the use of keyboard quotation marks is inside of shortdescription, but surely that's a place where it needs to be allowed

To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/MTAwMDAzMS5iZWV6ZXI.1783375944%40pnsh.

[Rob Beezer]

Thanks, David. Just what I needed, including holding the schema comments. ;-)
But I would be able to talk about them at Drop-In perhaps.

Rob

> /A copy of the license is included in the appendix entitled “GNU

> Free Documentation License” that appears in the output document of

> this PreTeXt source code./

> script <https://pretextbook.org/doc/guide/html/jing-trang.html#jing-
> pretext-script>

>
> is not helpful or is confusing, say so.
>
> The Python has a method called "agent" which a tool could use in
> automated way
> to refine its production of valid PreTeXt.  Hmmmmmmmmm.
>
> Subsection E.1.2: Validation for Programs
> https://pretextbook.org/doc/guide/html/jing-trang.html#validation-

> agent <https://pretextbook.org/doc/guide/html/jing-
> trang.html#validation-agent>

>
> @Mark F - I'd be real curious if you can get this to go with your
> very long jing
> invocation.
>
> @Oscar - the signature for "validate()" has changed.  I suspect it
> will not
> change further.  Please do not integrate all this into the CLI just
> yet, I
> consider it still "in progress".  We can plan more at Drop-In.
>
> Rob
>
> --
> You received this message because you are subscribed to the Google
> Groups "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pretext-dev...@googlegroups.com

> <mailto:pretext-dev%2Bunsu...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/

> pretext-dev/MTAwMDAxOC5iZWV6ZXI.1783275402%40pnsh <https://
> groups.google.com/d/msgid/pretext-dev/
> MTAwMDAxOC5iZWV6ZXI.1783275402%40pnsh>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to pretext-dev...@googlegroups.com <mailto:pretext-
> dev+uns...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/

> CALMYOncU3PbjW2ArXW%2B2i1LutS4PZ8mPNp3FJMJugkbrC7%2B3fQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/pretext-dev/
> CALMYOncU3PbjW2ArXW%2B2i1LutS4PZ8mPNp3FJMJugkbrC7%2B3fQ%40mail.gmail.com?
> utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to pretext-dev...@googlegroups.com <mailto:pretext-
> dev+uns...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/

> CANXmVMDk5G3hjfGLK1LvowbXdLDBFHDsW0_v%3Dj5XX88UwoqRqw%40mail.gmail.com <https://
> groups.google.com/d/msgid/pretext-dev/
> CANXmVMDk5G3hjfGLK1LvowbXdLDBFHDsW0_v%3Dj5XX88UwoqRqw%40mail.gmail.com?
> utm_medium=email&utm_source=footer>.

[Rob Beezer]

On 7/6/26 07:34, Andrew Scholer wrote:
> Agree, easy to use, possibly intimidating amount of output (schema for RS
> interactives is in rough shape), but easy enough to read.

"Possibly intimidating" is maybe good, I think. ;-)

> For integration purposes, is there a way to restrict processing to a certain
> subtree (other than filtering after generating all of it)? i.e. if I am working
> in source/pointers-aggregation/pointers-null.ptx that contains a section
> pointers-aggrecation_pointers-null, is there a way to only scan that part of the
> document?

Haven't looked, but I'm not optimistic. You can start a grammar at some lower
portion of the tree to make a new grammar, but I'm not sure how that would
happen on-the-fly. Your goal is zero errors all the time anyway. ;-) The "-r"
flag has been perverted for things it was never meant to do, so more of that is
not real high on my list.

Rob

[Sean Fitzpatrick]

Overall I thought it worked well. Each individual message was ready enough to understand,

but the sheer length of the file was a bit scary.

I used regex to delete, one by one, the things that were flagged as schema errors, but that I knew were ok. 

This took maybe 15-20 minutes, and removed some 10,000 lines from the file. That left me with about a dozen legitimate schema errors that I was able to fix!

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/MTAwMDAzMy5iZWV6ZXI.1783394150%40pnsh.

[Oscar Levin]

I haven't had a chance to try this out, but a couple of possibly related things for consideration as this moves forward.

1. Completely independently, I implemented a `pretext validate` command for the cli.  This was brought on my the recent new failure of lxml to validate (we have always done a validation step as part of a built, and just dumped a log file with the errors if there were any).  To fix that, I wired up a jing validation if jing was available, and added the cli command as an easy extra win.  But the real point of all this is that I've added jing to the default codespace/docker image, so it should be available for more people to use

2. Taking advantage of the availability of fable, I've finally implemented schema checking in pretext-tools (without relying on the redhat xml extension).  The big win with this is that we have complete control of how we report errors.  For example, since #me is no long valid, but still "works", I switched to reporting that just as a warning with a custom message that we now prefer #md.  Haven't done much customization yet, but the potential is there.  Having validation errors show up in the editor is likely much easier for people to use than reading through thousands of lines of a log file.  

It should also be possible to use the javascript schema validation instead of jing; it is more likely that someone has node installed than java and jing, so perhaps this would make sense as an additional target for the pretext validate command.

[Sean Fitzpatrick]

It would be nice, whatever the validation scheme is, if we can give a list of attributes etc. to omit from the report. 

Eg. for all the PROTEUS projects using interactive exercises that aren't yet in the schema, being able to ignore those would make it easier to find the errors that really need fixing.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/39a5d6dc-0605-43a3-8d47-15e68fd2ea27n%40googlegroups.com.

[Rob Beezer]

On 7/6/26 20:21, Sean Fitzpatrick wrote:
> Overall I thought it worked well. Each individual message was ready enough to
> understand,
> but the sheer length of the file was a bit scary.

Thanks for the report.

> left me with about a dozen legitimate schema errors that I was able to fix!

Me too! ;-)

Rob

[Rob Beezer]

Dear Mark,

Skimpy documentation at

Section 47.6: Configuring External Helper Programs
https://pretextbook.org/doc/guide/html/pretext-helpers.html#pretext-helpers

There is a new "jing" entry which right now defaiults to "jing". In your copy,
replace the entrey by your (very long!) command to execute jing as a Java program.

Since you have that all configured, I thought you would be a good test case.

Thanks,
Rob

[Rob Beezer]

> But the
> real point of all this is that I've added jing to the default codespace/docker
> image

Excellent - I was hoping so.

> we have complete control of how we report errors.

Right - the "validation-plus" stylesheet is like this. Since it reports more
complicated scenarios, it can more fully describe what is at issue. For
example, I just added a chack that tables have the same number of cells in each row.

> Having validation errors show up in the editor is likely much easier for people

Agreed. Though we need to be careful, the schema is "relaxed" in a few places
and will rely on "validation-plus" to cover. Like - you can put a WeBWorK #var
lots of places according to the schema, but it really only belongs as a
descendant of #webwork. So we should think about how an editor is going to
present that to an author.

> it is more likely that someone has node installed than java and jing,

I'm hoping to resurrect a server-based option (zero install) that Mitch has been
helping me with. In the queue.

Rob

> dev/MTAwMDAzMy5iZWV6ZXI.1783394150%40pnsh <https://groups.google.com/d/
> msgid/pretext-dev/MTAwMDAzMy5iZWV6ZXI.1783394150%40pnsh>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to pretext-dev...@googlegroups.com <mailto:pretext-
> dev+uns...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-

> dev/39a5d6dc-0605-43a3-8d47-15e68fd2ea27n%40googlegroups.com <https://
> groups.google.com/d/msgid/pretext-
> dev/39a5d6dc-0605-43a3-8d47-15e68fd2ea27n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

[Rob Beezer]

Did you try the "local-dev" option? That will do a passable job of covering
lots of the "experimental" stuff while it is evolving.

I don't plan to offer any filtering - folks will need to do there own grep/sed/awk.

I could open the "agent" view, which is terse, one line per error, and it is
tab-delimited, so I suspect susceptible to command-line (or editor) "sort".

Or have your assistant cluster the results?

Rob

> pretext-dev/MTAwMDAzMy5iZWV6ZXI.1783394150%40pnsh <https://
> groups.google.com/d/msgid/pretext-dev/
> MTAwMDAzMy5iZWV6ZXI.1783394150%40pnsh>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to pretext-dev...@googlegroups.com <mailto:pretext-
> dev+uns...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/pretext-
> dev/39a5d6dc-0605-43a3-8d47-15e68fd2ea27n%40googlegroups.com <https://
> groups.google.com/d/msgid/pretext-
> dev/39a5d6dc-0605-43a3-8d47-15e68fd2ea27n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.
>

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to pretext-dev...@googlegroups.com <mailto:pretext-
> dev+uns...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/

> CAH%2BNcPYwy4XxxHD7-t3dHcRqzSapvt9nDFZJ_YZzkn2MuZhQ%3DA%40mail.gmail.com
> <https://groups.google.com/d/msgid/pretext-dev/CAH%2BNcPYwy4XxxHD7-
> t3dHcRqzSapvt9nDFZJ_YZzkn2MuZhQ%3DA%40mail.gmail.com?
> utm_medium=email&utm_source=footer>.

[Mark Fitch]

We have success. pretext scripts runs with no errors and the output is a
list of needed changes.

user/pretext.cfg entery:

jing = java -classpath ~/jing-trang/build -jar ~/jing-trang/build/jing.jar

command:

~/pretext/pretext/pretext -V -p publication/publication.ptx
source/geometries_ibl.ptx

[Mark Fitch]

Improvement the --classpath was extraneous. Now using just the line
documented in

https://pretextbook.org/doc/guide/html/jing-trang.html#jing-pretext-script

[Sean Fitzpatrick]

With local-dev, once I remove errors due to attributes that are defined in some custom xsl for PDF builds, the validation.txt file for APEX is still over 22,000 lines long. Around 15,000 if I ignore the validation-plus stuff. 

It's a lot, but manageable if you know how to remove the errors that can be ignored. 

One interesting note: I think local-dev caught a few real errors that local missed. The main one I noticed was description missing a p.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/MTAwMDA0Mi5iZWV6ZXI.1783612296%40pnsh.

[Rob Beezer]

Excellent! Thanks, Mark. Exactly what I needed.

I'll need to remember to get that into the documentation.

[Rob Beezer]

Thanks to everyone for the testing and the suggestions. Really appreciated.
With some general comments below, I think I have addressed everybody's post. Maybe.

* I liked Jeremy's idea to squelch the [1] when it is not necessary, but we
will keep it in the machine-readable version. "only children" as my assistant
calls them.

* The "agent" output is now called the "terse" output and is exposed via a
method of the same name (-M).

* I just now had the idea to give names to the messages from "validation-plus",
so you will see those. (Eg, "tabular-ragged-columns".) The jing messages will
just say "schema", for a consistent output format.

* Sean wants to filter, and I know making clusters is a nice device. The IDs
for "validation-plus" will make filtering easy, and you can cluster with
command-line tools like "sort" on the terse output. Yes, all you early adopters
are overwhelmed, but the goal should be almost no errors, so we should expect
that to be the norm.

* The #description/#shortdescription message was really stale. That should
be fixed now. Mea culpa. And I am hoping than now my assistant will do a much
better job of helping me address *everything else* that needs to change when
just *one* thing changes.

Details: https://github.com/PreTeXtBook/pretext/pull/3027
Details: https://github.com/PreTeXtBook/pretext/pull/3028

I'm going to work on a "no-install" version, and then it may be open season on
schema adjustments. Stay tuned.

Thanks,
Rob

[Mark Fitch]

Possibly the easier solution is to use Python.

pip install jingtrang

jing = pyjing (in user/pretext.cfg)

I used this on my work computer and then then new validation works nicely.

[William Haynes]

My source contains all my interactives in single file, interactives.ptx, and then I insert individual interactives into my book using xi:include with xpointers.

<xi:include href="interactives.ptx" xpointer="ggb-rectangular-coordinates_interactive"/>

This works perfectly when I build using the CLI,  but the new validation tool doesn't honor the xpointers, so the entire interactives.ptx file gets included multiple times, leading to duplicate xml:ids errors in the assembled xml, and no validation report.  Is this a bug, a limitation of the tool, or a user error?

Thanks,

WIll Haynes

[Rob Beezer]

Dear Will,

xpointer() is frequently mooted as a solution to various jams. But it is an old
standard that I think failed (never left "draft" status). So adoption is
uneven, as you have discovered.

The only place it is mentioned in The Guide says "we do not support or recommend
these techniques." But the organization of your source is beyond our grasp, so
we cannot warn or suppress.

Maybe it will be inconvenient for your workflow to have one file per
#interactive, but that would be my suggestion. And big tedious reorganizations
like this do not scare me anymore. An AI assistant will make short work of
this, do it more accurately than you would, and would be a good first experiment
if you have not been doing this sort of thing. And it will do bulk edits across
many files just as well, so one-per-file may not be an impediment. $20/month
for Claude Code will change your life.

Rob

On 7/12/26 12:41, 'William Haynes' via PreTeXt development wrote:
>
> My source contains all my interactives in single file, interactives.ptx, and
> then I insert individual interactives into my book using xi:include with xpointers.
>
> <xi:include href="interactives.ptx" xpointer="ggb-rectangular-
> coordinates_interactive"/>
>
> This works perfectly when I build using the CLI,  but the new validation tool
> doesn't honor the xpointers, so the entire interactives.ptx file gets included
> multiple times, leading to duplicate xml:ids errors in the assembled xml, and no
> validation report.  Is this a bug, a limitation of the tool, or a user error?
>
> Thanks,
>
> WIll Haynes
> On Friday, July 10, 2026 at 6:38:28 PM UTC-4 maf...@alaska.edu wrote:
>
> Possibly the easier solution is to use Python.
> pip install jingtrang
> jing = pyjing (in user/pretext.cfg)
>
> I used this on my work computer and then then new validation works nicely.
>
> On Friday, July 10, 2026 at 8:04:21 AM UTC-8 Rob Beezer wrote:
>
> Excellent! Thanks, Mark. Exactly what I needed.
>
> I'll need to remember to get that into the documentation.
>
> On 7/9/26 11:48, 'Mark Fitch' via PreTeXt development wrote:
> > Improvement the --classpath was extraneous. Now using just the line
> documented in
> >

> > https://pretextbook.org/doc/guide/html/jing-trang.html#jing-pretext-
> script <https://pretextbook.org/doc/guide/html/jing-trang.html#jing-
> pretext-script>
> >

> > On 7/9/2026 7:39 AM, 'Rob Beezer' via PreTeXt development wrote:
> >> Dear Mark,
> >>
> >> Skimpy documentation at
> >>
> >> Section 47.6: Configuring External Helper Programs
> >> https://pretextbook.org/doc/guide/html/pretext-helpers.html#pretext-

> helpers <https://pretextbook.org/doc/guide/html/pretext-
> helpers.html#pretext-helpers>

> >>>> https://pretextbook.org/doc/guide/html/jing-trang.html#jing-
> pretext-script <https://pretextbook.org/doc/guide/html/jing-

> trang.html#jing-pretext-script>

> >>>>
> >>>> is not helpful or is confusing, say so.
> >>>>
> >>>> The Python has a method called "agent" which a tool could use in
> automated
> >>>> way to refine its production of valid PreTeXt. Hmmmmmmmmm.
> >>>>
> >>>> Subsection E.1.2: Validation for Programs

> >>>> https://pretextbook.org/doc/guide/html/jing-trang.html#validation-
> agent <https://pretextbook.org/doc/guide/html/jing-
> trang.html#validation-agent>
> >>>>

> >>>> @Mark F - I'd be real curious if you can get this to go with your
> very long
> >>>> jing invocation.
> >>>>
> >>>> @Oscar - the signature for "validate()" has changed.  I suspect it
> will not
> >>>> change further.  Please do not integrate all this into the CLI
> just yet, I
> >>>> consider it still "in progress".  We can plan more at Drop-In.
> >>>>
> >>>> Rob
> >>>>
> >>>
> >>
> >
>

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-dev...@googlegroups.com <mailto:pretext-
> dev+uns...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/pretext-

> dev/99769975-9a8a-4d05-90c4-1805e0ae5428n%40googlegroups.com <https://
> groups.google.com/d/msgid/pretext-
> dev/99769975-9a8a-4d05-90c4-1805e0ae5428n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

[William Haynes]

Dear Rob,

Thanks for your response. It's about what I expected based on the tepid language in the guide. 

 It turns out that it's trivial to assemble the entire book into a single file before validation, so this won't be an issue for me and I won't need to reorganize my source either.

xmllint --xinclude source/ptx/main.ptx > source/ptx/combined.ptx 

Will

[Oscar Levin]

Are you talking about the new pretext script validation or the vs code validation?  I'm surprised the pretext script doesn't assemble the source before validating, or that assembly doesn't resolve valid xpointers.

--

You received this message because you are subscribed to the Google Groups "PreTeXt development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/71bc61f0-2cd8-4424-8429-73e65bbd9393n%40googlegroups.com.

[William Haynes]

I am using the pretext -V script.   I've been using oXygen for xml editing since at least 2013 so I'm comfortable there, and haven't used vs code much.

I'm getting a warning from the CLI that  "Schema validation could not be completed: no validator was available (jing is not installed and lxml could not compile the schema)," but I haven't figured out how to get it to find jing on my computer.  Can you point me in the right direction?  

I'm on a mac, and jing is located at /usr/local/jing-trang/build/jing.jar.  I also ran  pip install jingtrang but that didn't help either.

Thanks

Will

[Rob Beezer]

On 7/13/26 10:10, Oscar Levin wrote:
> I'm surprised the pretext script doesn't assemble the source before
> validating, or that assembly doesn't resolve valid xpointers.

Thinking more carefully, I am surprised as well. The "-V" argument to the
pretext/pretext script will first go as far through the assembly process as to
resolve the "version" support an author may be using. We need to do that in
order that we have something the author thinks will be legal PreTeXt. Of
course, for most all builds, the same process, and more, always happens. So
what is different?

The only thing I can think of right now is that we are using a "custom" XInclude
loader, so that we can track file names from modularized source. In other
words, each schema error is able to report the modular file where it originated,
even thougn resolving version support gets us one big grand XML file to
validate. (That file is bugged with filenames.)

What would happen in Will's situation? His one file full of #interactive would
need to have this file's name all over the place - on each #interactive, since a
schema error on any one #interactive (Will wouldn't do that!) would need to
report that one big collection file. I have not checked, but I suspect our
bugging right now is on the root of the subtree that comes from the file. So
any one schema error needs to just find a bugged ancestor to augment the report.

Thinking as I write, but maybe I can investigate.

@Will:

<xi:include href="interactives.ptx"
xpointer="ggb-rectangular-coordinates_interactive"/>

suggests to me that "interactives.ptx" is structured like

foo
interactive
interactive
.
.
interactive xml:id="ggb-rectangular-coordinates_interactive"
.
.
interactive
interactive

Is there one overall XML element in that file? If so, what did you use (nee "foo").

Rob

[William Haynes]

My interactives.ptx is set up as a PreTeXt book with chapters and sections.  Each section contains a figure containing an interactives.  The root element is <pretext>