Pre and post-processing

[William Haynes]

Steven,

I don't have a specific example for you, but what I would expect is if a shell script was specified in project.ptx it would get called at the appropriate point either before or after the build.  

I'm only a hack programmer, and not very familiar with python, but am willing to take a stab at this with some guidance.  I see that the subprocess module is available for calling shell scripts.     I use a mac and have python 3.10.5 installed using macPorts.  

Will

> Could you add hooks to run  -pre and -post processes?

Possibly - do you have an example of another Python application that does this in the way you expect? I haven't tried something like this. PRs are also welcome of course (maybe wait until 1.0 is out)

[David Farmer]

Moving files from one place to another is a common issue
and perhaps should be treated as a special case.

Allowing publishers to run arbitrary scripts to pre- or post-process
the document is a recipe for long-term unhappiness. (For both the
publishers and the developers.)

> --
> You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> pretext-dev...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pretext-dev/af20dcb1-59fa-4a23-8a34-6de6314333a6n%40googlegroups.com.
>
>

[Oscar Levin]

Hi Will,

Can you say a little more about what sort of thing you would want to do pre- or post- processing?  One example I can think of is that previously the WeBWorK representations file labeled parts in a strange way, so I used sed to replace lettered lists with numerical ones (or the reverse?).  But this sort of thing can be done now: run `pretext generate webwork` and then do the post processing.  If I wanted it automated, I could use a bash script or makefile or python script that did both of those steps.  

Is there a use case you have in mind that couldn't be accomplished this way?  

Oscar.

[Steven Clontz]

In addition to David's note, some other reasons to be hesitant about this.

Right now project.ptx is rather platform-agnostic (modulo the <executables> which I often think should be factored out to a different file). Adding in scripts would likely be platform (or machine) specific.

I'm also thinking a lot about the security of PreTeXt-CLI projects. Long-term we want anyone to feel completely safe running `pretext new -u ARBITRARY_URL && pretext build` without pulling down and executing any nasty code. We probably need a "safe mode" that skips things like sageplots unless the user explicitly trusts the project (unless the core routine is more sandboxed than I expect).

Finally I'm not sure it's worth the trouble of letting folks put arbitrary shell scripts in project.ptx when it's essentially as easy to write a build.sh that invokes the PreTeXt-CLI itself.

Steven Clontz

https://clontz.org - steven...@gmail.com

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/ac9d1478-1c8b-41e5-980a-9d954eda8af8n%40googlegroups.com.

[Bradley Miller]

Indeed it is just as easy to write a shell script that calls pretext.

But… In terms of real world issues that cause publishers to write pre/post processing scripts here is what I had to do for publishing the books on Runestone Academy

1. Make sure the book has a document-id defined in docinfo
2. Make sure the name specified in the document-id is in the database and registered as a course
3. Make sure that the destination in project.ptx is correctly specified as “./published"

If all is verified then go ahead and build the book.  After a successful build I have to:

1. Process the Runestone-manifest.xml file produced by the build.
2. Read the metadata Book Title and subtitle, shelf section and blurb and update the library table so the book properly shows up in the library.

Having pre/post processing slots built into a pretext build would make my weekly build scripts and my manual build script and my Author UI page must simpler.

Brad

Bradley Miller, PhD
Professor Emeritus Computer Science
Luther College
Founder, Runestone Academy
Blog: A Reputable Journal

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/CAOWT6TjoO6ERzVK6mwXkjkxtSf-n5TkNkBLWsEGjnj1Me_JmAA%40mail.gmail.com.

[William Haynes]

Can you say a little more about what sort of thing you would want to do pre- or post- processing?  

The illustrations in my book are drawn using tools such as CAD or Affinity Designer, then exported as .svg and .pdf for use by PreTeXt.  For my own sanity these files are all stored in a ‘resources’ directory organized somewhat hierarchically with related images together and source files located alongside their derived image files.   Geogebra, css, and javascript files are organized in their own folders here too.  

My preprocessing script walks the ‘resources’ folder, flattens the hierarchy, copies all the [ .svg .png, .pdf, and .ggb] files into the "external/images’’ folder while filtering out files with other extensions.  GGB, CSS and JS files folders just are simply copied,  and folders with names beginning with underscore are ignored.  This way all the images are in the same directory and files not needed for the output aren't published.  In a sense, I am generating the external folder from my source documents.

I can certainly do this with a makefile -- that’s how I was doing it before -- but the pretext-cli does such a nice job moving files to the right places and running the right scripts that my makefile got very simple, and I hoped to be able to get rid of it completely.

Will

You received this message because you are subscribed to a topic in the Google Groups "PreTeXt development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pretext-dev/wIpo2FZdOJE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pretext-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/119c1fe6-a78d-489b-b8c6-8ad9d7c2cfb4%40Spark.