Using MathJax with Perl

[Phil Perry]

Let me start by explaining my situation. I maintain a Perl-based PDF generator (called PDF::Builder on CPAN and PhilterPaper/Perl-PDF-Builder on GitHub). It takes user-supplied text and creates a normal PDF file. I would like to add equation support, and MathJax looks like a good format to use.

Be aware that there is no browser necessarily available, and no Javascript either. There doesn't seem to be a capable enough JS engine available for Perl, but I'm open to using such a thing. To run from a Perl program, I could invoke an external command-line MathJax-based utility (my program writing out an equation to a file, and receiving an HTML or SVG file in return). An alternative is to compile/link a C or C++ source MathJax engine into a "Perl extension" with some Perl glue to let my program talk to it. The CPAN build/install process permits C/C++ source modules, in a modified source file called "XS format".

I'm not sure how cleanly and capably (if at all), various PDF Readers (e.g., Adobe Acrobat, or a PDF plug-in for Firefox) can handle Javascript directly. That would certainly be the simplest solution -- just bundle the markup into the PDF and let the Reader pull in the JS scripts and do the rendering. Has anyone done something like that, or even seen it? The PDF spec vaguely talks about "Javascript actions" to take for various form actions, but with no browser DOM I'm not sure how much it can handle.

I would very much want to avoid requiring the manual installation of external libraries etc. that can't be bundled up into a CPAN-style installation package. Most Perl users are unhappy at the thought of having to install random stuff on their system, so even a prebuilt binary command-line utility might be a problem for many, unless it comes from a reputable-looking source. Requiring something like NodeJS to be installed on their system (a wide range of OS's) may be a deal-breaker for many.

As for output from whatever MathJax creates, there is no DOM or other browser internals. I understand that it can output some flavor of HTML or SVG, and two of the things on my plate right now are HTML markup support for text input to PDF::Builder, and SVG (image file) support. As long as I know the expected dimensions of the MathJax when rendered, and whether it's in-line or display, I can place it properly.

So, given these requirements, is there any hope in pursuing a MathJax-based solution to getting equations into a PDF? I really, really don't want to write my own Javascript engine or to transpile the Javascript into C or Perl. A Javascript-input, HTML or SVG-output utility would also be useful for various graphic/plotting packages out there, so bonus points for that. Suggestions? Thanks!

[Phil Perry]

Looking around more, I see this conversation https://groups.google.com/g/mathjax-users/c/51PeC3HKuQM "Equations converted to images" asking for markup-to-SVG image files. Looking into the GitHub entry, it appears that NodeJS (?) needs to be installed, and a headless Chrome browser is used. Is this the best I'm going to be able to do? It's a lot to ask of users who want equation support when creating PDFs, but as long as it's well-supported programs being used, they may be willing to go through this. I presume that I would feed either a file containing the TeX markup, or export as a file, to a command line tex2svg and a little later get back an SVG file?

Is any external command-line utility going to be NodeJS or other Javascript-based? I can understand a preference for that from the MathJax end, as they only have to concentrate on their JS implementation, and not have to translate/maintain other systems, but from the Perl community standpoint it's a bit awkward. Still, if that's the best way to have cross-platform support...

[Davide Cervone]

Looking around more, I see this conversation https://groups.google.com/g/mathjax-users/c/51PeC3HKuQM "Equations converted to images" asking for markup-to-SVG image files. Looking into the GitHub entry, it appears that NodeJS (?) needs to be installed, and a headless Chrome browser is used. Is this the best I'm going to be able to do?

Since MathJax is written in Javascript, the most direct solution is to use node.js to run it as an external program.  You don't actually need the headless Chrome, however, as MathJax can run with its own internal DOM implementation, but if you will be using characters outside of MathJax's fonts, however, the results will be better if you do use headless Chrome.

I did some poking around, and there seems to be a very old Javascript package for perl (https://metacpan.org/dist/JavaScript) that seems to be based on libjs, in turn based on mozilla's engine, and a more recent one that works with the V8 engine (https://metacpan.org/dist/JavaScript-V8).  I have not tried either of these, but the later seems that it might have a chance at running MathJax.  But both still require external library installations, and in either case, you would still need to have a MathJax installation.  This can be done easily via npm (node package manager), or via a ZIP file from GitHub.

It's a lot to ask of users who want equation support when creating PDFs, but as long as it's well-supported programs being used, they may be willing to go through this. I presume that I would feed either a file containing the TeX markup, or export as a file, to a command line tex2svg and a little later get back an SVG file?

You could do that, but it may be more efficient to use pipes to do it rather than intermediate files.  It is not clear whether you want to process each equation separately, or process an entire page at once (either is possible).  If you are doing individual equations, you may want to use a single program that starts up once (rather than incurring the overhead of starting a separate process that needs to load and compile MathJax for each equation), where you send each equation to it via a pipe, followed by some "end-equation" marker, it processes the expression and sends the SVG back on a return pipe (followed by an "end of output" marker).  Your program sends are receives the expressions and their output using the same sub-process for all of them.

Is any external command-line utility going to be NodeJS or other Javascript-based?

Most likely.

I can understand a preference for that from the MathJax end, as they only have to concentrate on their JS implementation, and not have to translate/maintain other systems, but from the Perl community standpoint it's a bit awkward. 

MathJax's main use-case is in web pages in a browser, and that is naturally javascript-based.  It's other main use case is preprocessing web pages on a server, and that is readily done via javascript as well.  Using the same language in both settings is a significant advantage, as it means the same code works in both places, maintaining consistency and reducing maintenance costs.  The MathJax code is both large and complex, and makes significant use of Javascript prototype-based inheritance mechanism, which is significantly different from Perl's package-based class structure.  A conversion to Perl would be a major undertaking, and is not something we have the resources to provide.

Davide

[Phil Perry]

Hi Davide, thanks for getting back to me with detailed information.

It sounds like the use of Node.js is going to be my best bet. I have never used it before, and reading the Node.js pages they talk a lot about things "on the server". I'm hoping that by this they mean just the server side of the system, and not that they require a full-fledged web server. In other words, this whole thing should run on my Windows PC, or any other box, where I'm creating a PDF with equations? Is there any guidance on what additional or optional pieces are needed to be able to use MathJax? It's a bit overwhelming, and I don't want to have to do a lot of trial and error finding out what I need to install from Node.js in order to use MathJax, as well as what I need to install from MathJax.

You say that a headless Chrome browser is not necessary in most cases. What sort of things might someone do that would come out much better using Chrome? Would the built-in DOM do for 99% of equation setting, or should I seriously look at Chrome? Of course, I already have the browser, but I have no idea if I need to obtain or install something else to do the headless bit. For the benefit of those users who want to do some really advanced equation work, are there instructions somewhere on how to use the headless Chrome? At some point I should try it out myself (before telling users to do it), after getting the rest of the system running, so I would need some examples that show off the difference. Are there some such examples?

I looked around CPAN, and there are the two JS engines you mentioned, but they both appear to require manual installation of various libraries and subsystems, which is a big turn-off for most users (more so on Windows). There is also a claimed pure Perl JS engine (JE), but its development seems to have stalled some time ago. I think I'm better off using a nice clean Node.js + MathJax "name brand" system. Worst case, I can always switch over to one of the three JS engines at a later date.

Regarding the use of files to send markup out, and get HTML or SVG back, that's a last resort if I can't get pipes running. I think it can be done, but I don't have a lot of experience in that area. I like the idea of starting up the whole thing when a MathJax equation is found, and leaving it running until the PDF creation program is finished, dropping markup into the outbound pipe and grabbing the result from the inbound pipe. From the MathJax end, is this well documented, and maybe some examples?

I'm guessing that the 'tex2svg' example might be a good starting point (I'll have to try it first), and maybe an HTML-output one too. Rather than having my users install the MathJax-demos package, would I just supply a program derived from tex2svg? Is that all that's needed, once Node.js is installed? Of course, it will likely have to be modified to use pipes.

I wouldn't ask the MathJax people to rewrite in Perl (not that popular anymore, anyway)! A C or C++ version might be useful, including as a basis for a Perl extension (XS), as well as a standalone utility or library. I think there is a JS-to-CPP converter out there, but I don't know how good it is. Something to keep in mind, if manual labor on your part can be minimized (mostly automated conversion from JS).

[Davide Cervone]

Phil:

Sorry for the delay in getting back to you.  I had to replace my laptop due to power issues, and since the new MacBooks use a new CPU, everything had to be upgraded/recompiled/replaced/tested, and that was quite a process (some of the code goes back decades, and needed code adjustments in order to work with new versions of compilers/interpreters).  Just finished all that this morning.

It sounds like the use of Node.js is going to be my best bet. I have never used it before, and reading the Node.js pages they talk a lot about things "on the server". I'm hoping that by this they mean just the server side of the system, and not that they require a full-fledged web server.

You do not need a server.  Node.js runs as a command-line tool, so you can think of it in the same terms as you do your perl programs.  But its original use-case was to run browser code on the server side, and having the same code in both places has its advantages (rather than having to use javascript in the browser but php in the server, for example).  That's probably why there is still so much of that in the documentation.

In other words, this whole thing should run on my Windows PC, or any other box, where I'm creating a PDF with equations?

Yes, that is not a problem.

Is there any guidance on what additional or optional pieces are needed to be able to use MathJax? It's a bit overwhelming, and I don't want to have to do a lot of trial and error finding out what I need to install from Node.js in order to use MathJax, as well as what I need to install from MathJax.

All you need is node.js, which you can get from

https://nodejs.org/en/download/

to install by hand.  It is also possible to use a package manager to load it, depending on the environment.  E.g., apt install nodejs on ubuntu linux, or port install nodejs18 on MacOS.  For Windows, you can install either the standard system or for Windows Subsystem for Linux.  See their documentation for details.  There is a version manager for node called nvm (node version manager) that you could use, though it is not necessary, as node can update itself once you have it.

Once installed, you can use

node install mathjax

to obtain the MathJax code ready to run in node.

You say that a headless Chrome browser is not necessary in most cases. What sort of things might someone do that would come out much better using Chrome?

There are two places where it can be beneficial.  First, if you used characters that are not in the MathJax fonts, MathJax won't know how big the characters are, and so will have to guess about how much space the characters will take up.  MathJax usually asks the browser to measure the character's width, which is usually the most important dimension, but without a browser (i.e., in node), that can't be done, so MathJax guesses, and usually doesn't do a great job.  With headless Chrome, it would be able to ask Chrome to do the measurements as usual.

Second, MathJax scales its output to match the surrounding font, but to do so it needs to ask the browser what that size is.  Without the browser, it can't do that, and it either uses a default value for the font size, or you can give it one.  With headless Chrome, it can ask for the size of the surrounding font.  If you are planning to pipe individual expressions to MathJax, then it would be up to you to tell it the font size anyway, so there is no advantage there; but if you are processing complete pages with MathJax, then it would help to get the sizes right.  But with SVG output, the SVG is sized using width and height attributes using ex units, so the matching is done automatically, and again Chrome is not needed for this.  The only place that the font size is used in that case is to handle conversion of measurements in fixed units like inches and centimeters, and to properly scale characters that aren't in the MathJax fonts.

So the real use for you would be if you are planning to support characters that are outside of the MathJax fonts.  The main place this comes up is if people use text embedded in the math and that text uses accents, or non-European characters (like Chinese or Bengali, etc.)

Of course, I already have the browser, but I have no idea if I need to obtain or install something else to do the headless bit.

There is an example at

https://github.com/mathjax/MathJax-demos-node/tree/master/puppeteer

that shows how it is done.  There are instructions there, but everything you need should be installed automatically when you do npm install in the example directory.  If you were to go that route, you would make a package.json file like the one in that directory and run npm install as part of your installation (or installation instructions), and that would be all that is needed.

I looked around CPAN, and there are the two JS engines you mentioned, but they both appear to require manual installation of various libraries and subsystems, which is a big turn-off for most users (more so on Windows).

I understand.

There is also a claimed pure Perl JS engine (JE), but its development seems to have stalled some time ago. I think I'm better off using a nice clean Node.js + MathJax "name brand" system. Worst case, I can always switch over to one of the three JS engines at a later date.

I suspect that, unless these are very good implementations, they may not be up to handling MathJax, which is a complicates system, especially when loading extensions and such is needed.  You might have to do some coding to make that work in any of these extensions.  I doubt a pure Perl version will have implemented enough to do it, but I may be surprised.

Regarding the use of files to send markup out, and get HTML or SVG back, that's a last resort if I can't get pipes running. I think it can be done, but I don't have a lot of experience in that area.

I've done it before, but it has been a while.  I'll see if I can cut out some time to whip up an example.

I like the idea of starting up the whole thing when a MathJax equation is found, and leaving it running until the PDF creation program is finished, dropping markup into the outbound pipe and grabbing the result from the inbound pipe.

Yes, that is the slickest (and most efficient) way to handle it.

From the MathJax end, is this well documented, and maybe some examples?

Ha ha, you are funny.  "Well documented" is almost never the case.  There is no example of MathJax running through pipes that I know of, but I'll try to put one together for you when I get the chance.

I'm guessing that the 'tex2svg' example might be a good starting point (I'll have to try it first), and maybe an HTML-output one too.

SVG is probably your best choice, here.  The HTML is more complicated as it relies critically on CSS as well.

Rather than having my users install the MathJax-demos package, would I just supply a program derived from tex2svg?

Yes, that is the idea.

Is that all that's needed, once Node.js is installed?

Yes, as described above.  Your package.json will tell node.js what it needs to install, and then npm install will do that for your users.

Of course, it will likely have to be modified to use pipes.

Indeed.

I wouldn't ask the MathJax people to rewrite in Perl (not that popular anymore, anyway)!

I used perl extensively for a while, but not so much since I started using javascript.

A C or C++ version might be useful,

Even longer since I used that.  Not much chance we will be converting to C.

Anyway, I think you will not find it that onerous to do the node.js approach.  I will try to get something to start you off.

Davide

On Monday, June 6, 2022 at 9:49:39 AM UTC-4 Davide Cervone wrote:

Looking around more, I see this conversation https://groups.google.com/g/mathjax-users/c/51PeC3HKuQM "Equations converted to images" asking for markup-to-SVG image files. Looking into the GitHub entry, it appears that NodeJS (?) needs to be installed, and a headless Chrome browser is used. Is this the best I'm going to be able to do?

Since MathJax is written in Javascript, the most direct solution is to use node.js to run it as an external program.  You don't actually need the headless Chrome, however, as MathJax can run with its own internal DOM implementation, but if you will be using characters outside of MathJax's fonts, however, the results will be better if you do use headless Chrome.

I did some poking around, and there seems to be a very old Javascript package for perl (https://metacpan.org/dist/JavaScript) that seems to be based on libjs, in turn based on mozilla's engine, and a more recent one that works with the V8 engine (https://metacpan.org/dist/JavaScript-V8).  I have not tried either of these, but the later seems that it might have a chance at running MathJax.  But both still require external library installations, and in either case, you would still need to have a MathJax installation.  This can be done easily via npm (node package manager), or via a ZIP file from GitHub.

It's a lot to ask of users who want equation support when creating PDFs, but as long as it's well-supported programs being used, they may be willing to go through this. I presume that I would feed either a file containing the TeX markup, or export as a file, to a command line tex2svg and a little later get back an SVG file?

You could do that, but it may be more efficient to use pipes to do it rather than intermediate files.  It is not clear whether you want to process each equation separately, or process an entire page at once (either is possible).  If you are doing individual equations, you may want to use a single program that starts up once (rather than incurring the overhead of starting a separate process that needs to load and compile MathJax for each equation), where you send each equation to it via a pipe, followed by some "end-equation" marker, it processes the expression and sends the SVG back on a return pipe (followed by an "end of output" marker).  Your program sends are receives the expressions and their output using the same sub-process for all of them.

Is any external command-line utility going to be NodeJS or other Javascript-based?

Most likely.

I can understand a preference for that from the MathJax end, as they only have to concentrate on their JS implementation, and not have to translate/maintain other systems, but from the Perl community standpoint it's a bit awkward. 

MathJax's main use-case is in web pages in a browser, and that is naturally javascript-based.  It's other main use case is preprocessing web pages on a server, and that is readily done via javascript as well.  Using the same language in both settings is a significant advantage, as it means the same code works in both places, maintaining consistency and reducing maintenance costs.  The MathJax code is both large and complex, and makes significant use of Javascript prototype-based inheritance mechanism, which is significantly different from Perl's package-based class structure.  A conversion to Perl would be a major undertaking, and is not something we have the resources to provide.

Davide

-- 
You received this message because you are subscribed to the Google Groups "MathJax Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mathjax-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mathjax-users/a7e4b1f4-37cc-4534-a24f-9ce5feb96131n%40googlegroups.com.

[Phil Perry]

Thanks much for getting back. Yeah, life happens and systems die.

It sounds like I've got enough information here to soon give it a try. Where I'm trying to end up is to not only be able to set myself up correctly, but more importantly, be able to give accurate, concise instructions to all PDF::Builder users on how to set up their systems (without fear) to be able to do equations with MathJax markup. I hope that the pipe-usage code will be platform-agnostic, but as I can only test with Windows, I'll have to cross my fingers and hope that someone on Linux etc. can test for me (I know a Dutch user on Linux who will probably be willing to try it).

I'm looking forward to any pipe-usage code you can provide! Not a tremendous rush -- I probably won't get to the SVG reader until late summer, but at least I can test Node.js and file-based (or pipe) transfer by then, to see if it is returning good-looking SVG.

Yeah, I wouldn't be surprised if HTML code was heavily CSS-dependent, which I'm not sure I will be able to handle (advanced CSS) for a while, so SVG may be "it" for the time being. At some point, I'd like to test out "with Chrome" to see what difference that makes for non-standard fonts, and include instructions. I hope there's an example around somewhere so I can test it out. That also applies to SVG output, and not just HTML?

By the way, does the Node.js-based processor know about inline versus display equations? For the former, I'm guessing I would have to pass some information about the available line height? For the latter, I may or may not make use of any equation numbering that MathJax provides -- I'll have to see what's provided.

[Phil Perry]

Something else, not worth a whole new thread: the MathJax site has recommended HTML code (https://docs.mathjax.org/en/latest/misc/badges.html) for adding the "Powered by MathJax" button. Could you please pass the word to the appropriate party to consider changing border="0" in it to style="border: 0;" so that the W3C Validator will stop complaining about it (the border attribute is deprecated). Thanks!

[Davide Cervone]

I put together some sample code for you to try out.  See

https://gist.github.com/dpvc/e34f45cee6fb85dc40ea5a38ecba2754

There is a README will a little bit of documentation, and there are comments in the code, though you may have to ask questions if you have any.  There are still some things you may need to do yourself, like adjust the MathJax configuration to your liking, and making it configurable by your users, if you wish, but the basic typesetting is there.

I hope that the pipe-usage code will be platform-agnostic, but as I can only test with Windows, I'll have to cross my fingers and hope that someone on Linux etc. can test for me (I know a Dutch user on Linux who will probably be willing to try it).

I used IPC::Open2, which is appears should work on Windows as well as linux and MacOS, so I think you should be OK.

At some point, I'd like to test out "with Chrome" to see what difference that makes for non-standard fonts, and include instructions. I hope there's an example around somewhere so I can test it out. That also applies to SVG output, and not just HTML?

The

https://github.com/mathjax/MathJax-demos-node/tree/master/puppeteer

that I pointed you to earlier is such an example.  You should be able to use the tex2svg script as a guide to update the mathjax.js file in the gist I link to above, and the package.json will tell you what needs to be installed.

By the way, does the Node.js-based processor know about inline versus display equations? For the former, I'm guessing I would have to pass some information about the available line height? For the latter, I may or may not make use of any equation numbering that MathJax provides -- I'll have to see what's provided.

The MathJax::Typeset() function in my example gist allows you to specify options that control the display mode as well as the font sizing, etc.  Use

MathJax::Typeset("tex code", display => 1, em => 16, ex => 8);

to get display-mode typesetting where an em is 16 pixels and an ex is 8.  Leave off the display option (or use display => 0) to get in-line typesetting.

Hope that gets you started.

Davide

To view this discussion on the web visit https://groups.google.com/d/msgid/mathjax-users/98c501d0-f5cc-46be-8f87-2c66fa6d98a4n%40googlegroups.com.

[Davide Cervone]

Thanks.  Those snippets are over a decade old, and should be updated, as you suggest.

Davide

To view this discussion on the web visit https://groups.google.com/d/msgid/mathjax-users/31a4eaa8-86fc-47fe-819d-4eb77c838787n%40googlegroups.com.

[Phil Perry]

Hi Davide,

Just letting you know I've had a chance to get back to this. I installed Node.js (18.4.0) from nodejs.org and ran a "Hello World" sample from W3schools to show that Node.js was up and working. Then I tried installing MathJax itself. Your instructions said "node install mathjax" but it complained it couldn't find the module "install". Did you mean "npm install mathjax"? That seemed to work. Just for the benefit of anyone following this discussion...

Then I installed the MathJax demos: "npm install https://github.com/mathjax/MathJax-demos-node" seemed to do the trick. Then I had to find where tex2svg had been hidden, and I could run (command line, per Node.js page instructions) "node -r esm .\node_modules\MathJax-demos-node\component\tex2svg "A = B + C" >abc.svg". This produced an SVG file which works (yea!). I looked in the SVG file and was a bit distressed to see that it included paths (with ABSOLUTE COORDINATES! ... maybe the translate() takes care of relocation?) to draw all the glyphs, rather than text instructions, but I guess that the MathJax developers felt that was the better way. I haven't checked yet, but I'm hoping that if I use "A" twice, it doesn't put the glyph path in twice! Note that for use in PDF::Builder, users won't be asked to install MathJax-demos-node (unless they want to), but I'll be providing whatever module(s) are necessary.

I still need to

1. write some Perl code to send an arbitrary equation from PDF::Builder to MathJax and retrieve the SVG file, via tex2svg (as proof of concept)
2. write a tex2svg replacement module sending either a file or a command-line string, and retrieving a file
3. implement your piped interface (I haven't had a chance to look at it yet) in the tex2svg replacement (keep file interface if user has trouble with the pipe)
   
4. take a look at inline vs. display (per your note), and equation numbers for display (at least, leave space for me to write them)
   
5. take a look at headless vs browser for certain cases, and implement such support for users who need it
   
6. finish implementing SVG support for PDF (now that I know what sort of support MathJax will be expecting)
7. consider HTML support, as I need at least limited HTML support for some other stuff (see how complex the CSS turns out to be)
   

I think my plate is full for this summer and fall! Thanks much for your support and work on this. I'll let you know here how things turn out.

[Davide Cervone]

Your instructions said "node install mathjax" but it complained it couldn't find the module "install". Did you mean "npm install mathjax"? That seemed to work. 

Yes, it should have been 'npm install mathjax'.  My error.  Glad you were able to work it out.

Then I installed the MathJax demos: "npm install https://github.com/mathjax/MathJax-demos-node" seemed to do the trick.

You don't need to install the demos, though you certainly can if you wish.  (They were intended only as examples, not as modules to install directly.)

Then I had to find where tex2svg had been hidden,

There are several different copies of tex2svg illustrating the various ways that MathJax can be included in node applications.  The components version that you used is one of them.

I looked in the SVG file and was a bit distressed to see that it included paths (with ABSOLUTE COORDINATES! ... maybe the translate() takes care of relocation?) to draw all the glyphs, rather than text instructions, but I guess that the MathJax developers felt that was the better way.

Yes, the characters are included as paths, not text, which means you don't need to have special fonts installed on your user's computers (or have access to web fonts).

I haven't checked yet, but I'm hoping that if I use "A" twice, it doesn't put the glyph path in twice!

You can control whether MathJax uses shared paths or separate paths for each instance.  The shared paths use a <defs> node to define the paths, and then <use> nodes to access the common paths.  That requires the use of element IDs which is a bit more complicated, so the option of using individual paths is available.  For full-page processing, the cache can be global to the page, otherwise it is local to each expression.  See the fontCache option

https://docs.mathjax.org/en/latest/options/output/svg.html?highlight=fontCache

I still need to 

1. write some Perl code to send an arbitrary equation from PDF::Builder to MathJax and retrieve the SVG file, via tex2svg (as proof of concept)
2. write a tex2svg replacement module sending either a file or a command-line string, and retrieving a file
3. implement your piped interface (I haven't had a chance to look at it yet) in the tex2svg replacement (keep file interface if user has trouble with the pipe)
   
4. take a look at inline vs. display (per your note), and equation numbers for display (at least, leave space for me to write them)
   
5. take a look at headless vs browser for certain cases, and implement such support for users who need it
   
6. finish implementing SVG support for PDF (now that I know what sort of support MathJax will be expecting)
7. consider HTML support, as I need at least limited HTML support for some other stuff (see how complex the CSS turns out to be)
   

The gist that I linked in my previous message already does most of your 1 through 4, so I don't think you really need to use the MathJax-demos-node repository, but can go directly to the gist, which includes that functionality already.  You might want to look at that for the file-based version, if you need that.  The pipe interface already includes a replacement for tex2svg that handles individual expressions, including inline and display mode, so there is nothing more that needs to be done there.  Your 5 to 7 are really what is left.

Davide

[Phil Perry]

I installed your gist and tried running it, but had problems:

-------------------------------------------------------------------------------------------------------------

C:\Users\Phil\Desktop\MathJax.pipe>main.pl
C:\Users\Phil\Desktop\MathJax.pipe\mathjax.js:1
use IPC::Open2;
    ^^^

SyntaxError: Unexpected identifier
←[90m    at Object.compileFunction (node:vm:353:18)←[39m
←[90m    at wrapSafe (node:internal/modules/cjs/loader:1040:15)←[39m
←[90m    at Module._compile (node:internal/modules/cjs/loader:1076:27)←[39m
←[90m    at Module._extensions..js (node:internal/modules/cjs/loader:1166:10)←[39m
←[90m    at Module.load (node:internal/modules/cjs/loader:988:32)←[39m
←[90m    at Module._load (node:internal/modules/cjs/loader:834:12)←[39m
←[90m    at Module.require (node:internal/modules/cjs/loader:1012:19)←[39m
←[90m    at require (node:internal/modules/cjs/helpers:102:18)←[39m
    at Object.<anonymous> (C:\Users\Phil\Desktop\MathJax.pipe\pipe-handler.js:19:33)
←[90m    at Module._compile (node:internal/modules/cjs/loader:1112:14)←[39m

Node.js v18.4.0
Pipe-handler didn't send START command () at ./MathJax.pl line 20.
Compilation failed in require at C:\Users\Phil\Desktop\MathJax.pipe\main.pl line 1.

-----------------------------------------------------------------------------------------------------------

It looks like it's trying to run "mathjax.js" as JS code, yet the code appears to be Perl. My system is set up to assume a file ending in ".pl" is to be run with Perl (no explicit "perl xxxx.pl" needed), if that has any bearing on it. I'm not sure if it automatically handles ".js" as JavaScript. Should this file be renamed to mathjax.js.pl? Will it be properly run from NodeJS, or will it need an explicit "perl " in the command?

Also, I see that "npm install" put the node_modules/ directory under this Desktop test directory, rather than where I installed MathJax (under a different directory). Is that going to cause any problems? I presume that I will need to tell users to be careful where they do their "install" and create node_modules/.

Desktop/

  MathJax.pipe/

      node_modules/

      main.pl

      MathJax.pl

      mathjax.js

      package.json

      package-lock.json

      pipe-handler.js

      README.md

I'm a noob at NodeJS stuff, so please bear with me as I try to figure this out! At this point, I want to make sure your sample code (from the gist) is working properly, before I go anywhere else. Thanks!

[Phil Perry]

By the way, I do have IPC::Open2 installed (v1.04). Regarding where the install is made, I guess that a user can always update the Perl code to point to wherever the module lives (node_modules/ path), since there doesn't seem to be a single standard repository for installed modules.

Is there any way to reply on this group without copying the entire previous post, or at least, allow editing of it to show selected parts? You're doing it, so I take it there's a way...

[Davide Cervone]

Apparently I copied the MathJax.pl code into the mathjax.js file accidentally rather than the needed javascript.  I've updated the gist, so give it another try (you can just download the mathjax.js file, as that is the only one changed).

Sorry about that!

Davide

By the way, I do have IPC::Open2 installed (v1.04). Regarding where the install is made, I guess that a user can always update the Perl code to point to wherever the module lives (node_modules/ path), since there doesn't seem to be a single standard repository for installed modules.

The problem was that node was trying to run the perl code (since I pasted the wrong file into mathjax.js), and was complaining about the syntax of "use IPC::Open2", since that is not value javascript.  It has nothing to do with whether IPC::Open2 is installed in perl or where.  That will be in the user's perl installation, not the node_modules directory.   Where you place the files from the gist is up to you, but once you have it where you want it to be, you will need to do "npm install" in that directory, and that will create the node_modules subdirectory there.  That is the correct place for it, and should not be changed, or node won't be able to find its needed modules.

The perl code from the gist will find IPC::Open2 in the main perl system library (and if it is no installed, then "cpan install IPC::Open2" should do the trick.

Is there any way to reply on this group without copying the entire previous post, or at least, allow editing of it to show selected parts? You're doing it, so I take it there's a way...

I get the messages via email and reply to them that was as well, so can edit out the piece that I don't want to include.  I have not used the on-line interface, but I suspect you can edit out the original message there as well.

Davide

[Phil M Perry]

Sigh. My reply appears to have gone directly to Davide and not to here. Try again.

OK, let's try a mail reply rather than using the webpage interface.

On 6/29/2022 1:36 PM, Davide Cervone wrote:

Apparently I copied the MathJax.pl code into the mathjax.js file accidentally rather than the needed javascript.  I've updated the gist, so give it another try (you can just download the mathjax.js file, as that is the only one changed).

I d/l the zip file, unzipped, and copied the new mathjax.js over the bad one. Is that all I needed to do? Or did I need to do a new install?

By the way, I do have IPC::Open2 installed (v1.04). Regarding where the install is made, I guess that a user can always update the Perl code to point to wherever the module lives (node_modules/ path), since there doesn't seem to be a single standard repository for installed modules.

The problem was that node was trying to run the perl code (since I pasted the wrong file into mathjax.js), and was complaining about the syntax of "use IPC::Open2", since that is not value javascript.  It has nothing to do with whether IPC::Open2 is installed in perl or where.  That will be in the user's perl installation, not the node_modules directory.

I was just trying to reassure you, if you saw the error on "use IPC::Open2;" and thought it was because I didn't have IPC::Open2 installed on my Perl. Didn't want to send you down the wrong path and all that.

Where you place the files from the gist is up to you, but once you have it where you want it to be, you will need to do "npm install" in that directory, and that will create the node_modules subdirectory there.  That is the correct place for it, and should not be changed, or node won't be able to find its needed modules.

I'll probably have the user (who installs and configures PDF::Builder) check where it's looking for the modules (where they installed it), and update the configuration that PDF::Builder looks in to find where MathJax is. Is MathJax itself installed as part of your gist, or does it matter that it's in a different place?

Is there any way to reply on this group without copying the entire previous post, or at least, allow editing of it to show selected parts? You're doing it, so I take it there's a way...

I get the messages via email and reply to them that was as well, so can edit out the piece that I don't want to include.  I have not used the on-line interface, but I suspect you can edit out the original message there as well.

That's what I'm trying here.

OK, here are the results, with some added print debugs:

    C:\Users\Phil\Desktop\MathJax.pipe>main.pl

    START  <=== print OPIPE returned $line from pipe-handler start

    \frac{x^2}{2}   <=== print $tex LaTeX to convert
    ERROR Unexpected end of JSON input  <=== print OPIPE returned $line after TYPESET command
    MathJax Error: Unexpected end of JSON input at ./MathJax.pl line 41, <OPIPE> line 2.

It doesn't look happy. 3 debug lines were added, so the original error message was line 38.

Two further questions:

1. Is it sending the "display" flag over to MathJax, some place?

2. Does anything need to be sent to shut down MathJax when I'm done with it? The main.pl sends two examples, so I'm guessing that you did implement  keeping the pipe open for multiple uses, but does it shut down at the end (such as when Perl goes away)?

By the way, my last message to you is "marked as abuse" when I'm not signed in to Google Groups, but I didn't see that message when I logged in. Did you accidentally hit a "report as abuse" button when looking at it?

[Davide Cervone]

OK, I figured out what happened:  I had added the support for display vs. in-line math just before posting the gist, and had to update the files there to include it.  I accidentally pasted the changes that were intended for MathJax.pl into mathjax.js (which was the initial problem you were reporting).  I fixed that in the gist, but that error also meant that the MathJax.pl file was not updated (since the updated version was pasted into the wrong file).  So your current error is because the MathJax.pl file doesn't include the changes to support the options (in particular, the display option) being passed to MathJax, and that is causing the error you are currently seeing (the options are passed as a JSON structure, but the old MathJax.pl doesn't pass them, so the pipe-handler doesn't have them, leading to the JSON error you are seeing.

So the solution is now to update the MathJax.pl file to the one in the current gist.  So sorry about the error (one paste mistake has ripple effects).

Apparently I copied the MathJax.pl code into the mathjax.js file accidentally rather than the needed javascript.  I've updated the gist, so give it another try (you can just download the mathjax.js file, as that is the only one changed).

I d/l the zip file, unzipped, and copied the new mathjax.js over the bad one. Is that all I needed to do? Or did I need to do a new install?

You don't need to do another install.  You only need to update the files from the gist.

The problem was that node was trying to run the perl code (since I pasted the wrong file into mathjax.js), and was complaining about the syntax of "use IPC::Open2", since that is not value javascript.  It has nothing to do with whether IPC::Open2 is installed in perl or where.  That will be in the user's perl installation, not the node_modules directory.

I was just trying to reassure you, if you saw the error on "use IPC::Open2;" and thought it was because I didn't have IPC::Open2 installed on my Perl. Didn't want to send you down the wrong path and all that.

The error message was not about a missing module, but a syntax error, and it was being reported by node, not perl, so I wasn't sent down the wrong path (but thanks).

Where you place the files from the gist is up to you, but once you have it where you want it to be, you will need to do "npm install" in that directory, and that will create the node_modules subdirectory there.  That is the correct place for it, and should not be changed, or node won't be able to find its needed modules.

I'll probably have the user (who installs and configures PDF::Builder) check where it's looking for the modules (where they installed it), and update the configuration that PDF::Builder looks in to find where MathJax is. Is MathJax itself installed as part of your gist, or does it matter that it's in a different place?

I would assume that your PDF::Builder will include your versions of the files from my gist (once you modify them to your liking), and that your installation process will trigger the "npm install" command in the directory containing them (or else ask the user to do that).  That will cause the node_modules directory to be created and MathJax to be installed (you should NOT include the node_modules directory in your package directly).  That is the correct place for MathJax, and it will be found by the pipe-handler and mathjax.js files automatically.   Since the (updated) gist files will be in your PDF::Builder distribution in the location you have placed them, PDF::Builder should know where there are and be able to call them from that location.  I don't see why there should be a need for user configuration for that (but I'm not all that familiar with perl package distributions).

1. Is it sending the "display" flag over to MathJax, some place?

Yes, but that was missing in the MathJax.pl file, since I pasted that version that did that into mathjax.js accidentally.  The new MathJax.pl does it.

2. Does anything need to be sent to shut down MathJax when I'm done with it? The main.pl sends two examples, so I'm guessing that you did implement  keeping the pipe open for multiple uses, but does it shut down at the end (such as when Perl goes away)?

On Unix, the mathjax sub-process should end when the perl program ends.  But if you want to be explicit about it, you can add an End() command to MathJax.pl that sends the END command to the MathJax pipe.  The pipe-handler already has code to handle the END command on the pipe.

By the way, my last message to you is "marked as abuse" when I'm not signed in to Google Groups, but I didn't see that message when I logged in. Did you accidentally hit a "report as abuse" button when looking at it?

I have removed the mark.  Not sure whey that happened.

Davide

[Phil M Perry]

Great news! Your example (with pipes, from Perl) appears to work well. Thank you for putting the effort into this. Of course, feel free to add it to the examples for your next MathJax release.

On 6/30/2022 7:26 AM, Davide Cervone wrote:

So the solution is now to update the MathJax.pl file to the one in the current gist.  So sorry about the error (one paste mistake has ripple effects).

Been there, done that.

I'll probably have the user (who installs and configures PDF::Builder) check where it's looking for the modules (where they installed it), and update the configuration that PDF::Builder looks in to find where MathJax is. Is MathJax itself installed as part of your gist, or does it matter that it's in a different place?

I would assume that your PDF::Builder will include your versions of the files from my gist (once you modify them to your liking), and that your installation process will trigger the "npm install" command in the directory containing them (or else ask the user to do that).  That will cause the node_modules directory to be created and MathJax to be installed (you should NOT include the node_modules directory in your package directly).  That is the correct place for MathJax, and it will be found by the pipe-handler and mathjax.js files automatically.   Since the (updated) gist files will be in your PDF::Builder distribution in the location you have placed them, PDF::Builder should know where there are and be able to call them from that location.  I don't see why there should be a need for user configuration for that (but I'm not all that familiar with perl package distributions).

I'll have to think this one through, as exactly what instructions to give to my users. They'll need to install NodeJS before the MathJax-related stuff can be installed, so it may be cleaner (Perl-wise) to make it a fully manual installation process. That way I won't have to do hand-holding for someone who tried installing PDF::Builder before installing NodeJS. The use of MathJax will be an optional package, so I'll probably have them install NodeJS and then MathJax manually, as well as your piped version of tex2svg. I'm assuming that the full MathJax still needs to be installed, even with your package -- is that correct? Or does the code from your gist contain what's needed of MathJax, and users don't need to install the full MathJax (not talking about the -demos-node package)? I'm a little unclear on that. They may have to manually set a path to where MathJax things are, in Perl code (or a config file) -- I don't know yet where things will sift out to. To Be Seen.

2. Does anything need to be sent to shut down MathJax when I'm done with it? The main.pl sends two examples, so I'm guessing that you did implement  keeping the pipe open for multiple uses, but does it shut down at the end (such as when Perl goes away)?

On Unix, the mathjax sub-process should end when the perl program ends.  But if you want to be explicit about it, you can add an End() command to MathJax.pl that sends the END command to the MathJax pipe.  The pipe-handler already has code to handle the END command on the pipe.

Per your suggestion, I added in MathJax.pl a new sub End { print IPIPE "END\n"; } and at the end of main.pl I added MathJax::End();. I guess it's working (at least, no errors reported). Does that code sound about right? I won't be firing up MathJax unless Equation code is encountered, and then when PDF::Builder is done it will call End() if MathJax is running. That should avoid repeated startups and shutdowns while a document is being processed.

By the way, my last message to you is "marked as abuse" when I'm not signed in to Google Groups, but I didn't see that message when I logged in. Did you accidentally hit a "report as abuse" button when looking at it?

I have removed the mark.  Not sure whey that happened.

Hmm. I still see it on the web page as a guest, but not when signed in. Quite odd.

So that should pretty much take care of items 1-4 on my list. I need to think about mapping the SVG coordinate system to PDF's (and coordinating with text size, especially for inline), and how to handle the resizable display-with-equation-number. Does MathJax have any knowledge of the column width to format in, or is it the responsibility of the author to make sure that equations fit? It looks like the SVG processing is going to be a bit more complicated than I had hoped, but it's still feasible. Then I can play with headless vs browser and see what the effects are and whether it's worth adding support for browser use (the default, in your sample code, is headless?). I should probably add support to return an SVG file in case the pipes don't work, but I may put that off til later. It may be pointless to support HTML/CSS from MathJax, unless it gives significant additional capabilities.

Again, thanks much for all your work. I'll try to remember to drop a note here when I've got everything working later this year.

[Davide Cervone]

The use of MathJax will be an optional package, so I'll probably have them install NodeJS and then MathJax manually,

OK, that makes sense.

as well as your piped version of tex2svg.

OK, in that case, you may want to make an npm package for it so that it is easy to install, once you have node in place.  Then you just need to do

npm install <your-package-name>

in the appropriate place and that should be it.

I'm assuming that the full MathJax still needs to be installed, even with your package -- is that correct?

There are two MathJax npm packages -- mathjax and mathjax-full.  The latter includes the sources and other things, but you don't really need that for your use.  The plain mathjax package should be sufficient.  That is the one listed in the package.json file.  To make your own package, you will need to expand the package.json file a bit to include the package name and version, plus some other information about it (most of which is optional).  See

https://docs.npmjs.com/cli/v7/configuring-npm/package-json

for documentation on that.  With the mathjax package as a dependency of your package, it will be install automatically when yours is installed.

They may have to manually set a path to where MathJax things are, in Perl code (or a config file) -- I don't know yet where things will sift out to. To Be Seen.

OK, that makes sense.

2. Does anything need to be sent to shut down MathJax when I'm done with it? The main.pl sends two examples, so I'm guessing that you did implement  keeping the pipe open for multiple uses, but does it shut down at the end (such as when Perl goes away)?

On Unix, the mathjax sub-process should end when the perl program ends.  But if you want to be explicit about it, you can add an End() command to MathJax.pl that sends the END command to the MathJax pipe.  The pipe-handler already has code to handle the END command on the pipe.

Per your suggestion, I added in MathJax.pl a new sub End { print IPIPE "END\n"; } and at the end of main.pl I added MathJax::End();. I guess it's working (at least, no errors reported). Does that code sound about right?

Yes, that looks right.

I won't be firing up MathJax unless Equation code is encountered, and then when PDF::Builder is done it will call End() if MathJax is running. That should avoid repeated startups and shutdowns while a document is being processed.

Correct.

By the way, my last message to you is "marked as abuse" when I'm not signed in to Google Groups, but I didn't see that message when I logged in. Did you accidentally hit a "report as abuse" button when looking at it?

I have removed the mark.  Not sure whey that happened.

Hmm. I still see it on the web page as a guest, but not when signed in. Quite odd.

Well maybe you had a cached version in the browser, and it was showing that?  I have checked and there is nothing marked that I can see.

So that should pretty much take care of items 1-4 on my list. I need to think about mapping the SVG coordinate system to PDF's (and coordinating with text size, especially for inline), and how to handle the resizable display-with-equation-number.

The SVG element will have width and height (and vertical-align to get the baseline right) given in ex units, so if you know the ex-height of the surrounding font, you can use that to scale the image (as a browser would do).  The equations with numbers are handled with nested SVG's (the outer one with width="100%"), so you would need to stretch that to the width of the container.  They have a minwidth set so that number will not overlap the rest of the expression.  

Does MathJax have any knowledge of the column width to format in, or is it the responsibility of the author to make sure that equations fit?

Version 2 includes automatic line breaking, but that hasn't yet been ported to v3 (I am in the process of doing that as we speak, and it will be in the next release at the end of the summer).  One of the parameters you can pass to the typeset() command is the container width, and that is what will be used for the line breaking.  It is up to you to send the correct value for your situation.   (In the headless chrome setting, if you send a complete page, MathJax can figure out the container size, and you can extract the SVG elements from the processed page if you want them separately.)

It looks like the SVG processing is going to be a bit more complicated than I had hoped, but it's still feasible. Then I can play with headless vs browser and see what the effects are and whether it's worth adding support for browser use (the default, in your sample code, is headless?).

Well, I think we are using the terminology differently.  The gist I provided you does not use a browser (headless or otherwise), but instead uses a limited DOM implementation in javascript.  That DOM does not include computations of the sizes of things, or handle CSS or other fancy bits; it is just for the structure of the DOM.  The "puppeteer" directory in the node demos repository shows how to hook MathJax to a "headless chrome", which is a full Chrome application, but set up not to open any windows (that is the meaning of "headless" in this context).  Doing that makes it possible to handle all the CSS and size measurements that are possible in a normal browser, but without seeing the page in a window (so it can work on a server that doesn't even have a monitor to show windows on).  It is a more complicated arrangement (both in terms of installation and programming), and does take longer to spin up since it launches Chrome to do it, but once it is running, it should be reasonably efficient.  With the pipe-handler approach, that would certainly be a viable mechanism, if a bit more complicated.

Good luck with the project.

Davide

[Phil M Perry]

On 7/1/2022 5:05 PM, Davide Cervone wrote:

 plus some other information about it (most of which is optional).  See

https://docs.npmjs.com/cli/v7/configuring-npm/package-json

for documentation on that.  With the mathjax package as a dependency of your package, it will be install automatically when yours is installed.

OK, I presume I can figure out how to have my install package pull in the necessary MathJax package. The PDF::Builder user who wants to use MathJax equation formatting may end up having to do as little as installing NodeJS and cd'ing to the right place in PDF::Builder and running npm install. That shouldn't be an overwhelming burden to anyone.

Does MathJax have any knowledge of the column width to format in, or is it the responsibility of the author to make sure that equations fit?

Version 2 includes automatic line breaking, but that hasn't yet been ported to v3

One other thought regarding some sort of line breaking -- is it considered good typography to potentially split an equation across a page break? I would think that could cause problems for readers. Splitting up an equation to fit in a narrower column isn't too bad, but how hard should one strive to keep it all on the same page? As far as the mechanics of line breaking go, would the result be a series of SVG files (one per resulting line)? Anyway, that's something that might be wanted in the future -- I don't need it in the initial release. For now, it's up to the author to be aware of column width and fit things accordingly.

Well, I think we are using the terminology differently.  The gist I provided you does not use a browser (headless or otherwise), but instead uses a limited DOM implementation in javascript.  That DOM does not include computations of the sizes of things, or handle CSS or other fancy bits; it is just for the structure of the DOM.  The "puppeteer" directory in the node demos repository shows how to hook MathJax to a "headless chrome", which is a full Chrome application, but set up not to open any windows (that is the meaning of "headless" in this context).  Doing that makes it possible to handle all the CSS and size measurements that are possible in a normal browser, but without seeing the page in a window (so it can work on a server that doesn't even have a monitor to show windows on).  It is a more complicated arrangement (both in terms of installation and programming), and does take longer to spin up since it launches Chrome to do it, but once it is running, it should be reasonably efficient.  With the pipe-handler approach, that would certainly be a viable mechanism, if a bit more complicated.

OK, I stand corrected. It looks like the initial release will be based on your gist, and if anyone reports issues with character sizing and whatnot, I'll be back in contact to ask for help in using the headless Chrome browser.

Good luck with the project.

Davide

Thanks much for all your help!

Phil

[Davide Cervone]

OK, I presume I can figure out how to have my install package pull in the necessary MathJax package.

the package.json file in the gist already does that, so you would simply need to add the other properties to it that are needed for your package (like its name, version, maintainer, etc.)

One other thought regarding some sort of line breaking -- is it considered good typography to potentially split an equation across a page break?

I would avoid it, when possible.

Splitting up an equation to fit in a narrower column isn't too bad, but how hard should one strive to keep it all on the same page?

Since browsers don't really have page breaks on screen, I haven't thought about page breaking in a long while.  I would have to go back to the TeXBook to see how TeX handles that, but if I recall correctly, expressions created by the align environment and other multi-line environments are not breakable across pages without special effort on the author's part.  

As far as the mechanics of line breaking go, would the result be a series of SVG files (one per resulting line)?

No, MathJax produces a single SVG for the multi-line expression.  The lines will be in separate <g> tags, so one could theoretically pull it apart into individual lines, if need be.

Davide

[Phil M Perry]

OK, for the time-being, I'll just advise users to pay attention to their column width (that an equation should fit in) and to manually adjust the equation to fit (including breaking into multiple pieces). When creating a PDF, a page's column width should be pretty much fixed -- no need to worry about dynamically varying widths. Perhaps some day some sort of automatic or optional line breaking will be handled by MathJax, and my code will be updated to accommodate that.

Regarding breaking across pages, it's understandable that it's not desirable, so for the time-being I will keep equations (including multi-line ones) on one page, much like an image would be handled. Perhaps in the future, something can be done to allow splitting at a page break, while honoring typographic and mathematical standards.

While display equations are pretty straightforward, I can see issues with inline equations. Some could be potentially long enough, if not wrapped (split in the middle), to cause a huge hole in a paragraph if the whole thing has to be shoved off to the next line. Even worse, if it's longer than an available line width, it can't fit at all without breaking it. For now, I'll just have to warn users to be careful about long inline equations, and give an error message if it won't fit at all. A user could possibly manually split an inline equation by turning it into two or more adjacent inline equations, assuming the fragments will fit nicely next to each other. Support for breaking an inline equation at reasonable places (given the remaining space available) would be much appreciated, if it's not already there, as would be guidance on handling such things. Maybe something like a \b given at reasonable break points (such as after a + or -) could help. For multiplication, perhaps a \cdot or something could be added at a break (like a hyphen when breaking a word)?

On 7/11/2022 6:03 AM, Davide Cervone wrote:

[Phil M Perry]

On 7/14/2022 9:44 AM, Phil M Perry wrote:

A user could possibly manually split an inline equation by turning it into two or more adjacent inline equations, assuming the fragments will fit nicely next to each other.

I just tried this out with HTML in a browser. Say, you have a longish inline equation \(a long equation\) that might cause a large gap at the end of a line if it doesn't fit. Splitting it into \(a\)\(long\)\(equation\) breaks nicely at the end of the line, although the fragments (on the same line) are too close together. Putting a single space between each fragment seems to work pretty well: \(a\) \(long\) \(equation\). The spaces disappear at the line break, which is better than using ~. Any recommendations on this, or experience to report?

Support for breaking an inline equation at reasonable places (given the remaining space available) would be much appreciated, if it's not already there, as would be guidance on handling such things. Maybe something like a \b given at reasonable break points (such as after a + or -) could help. For multiplication, perhaps a \cdot or something could be added at a break (like a hyphen when breaking a word)?

That's still manually breaking up an equation, and a separate \b and \bmult might be needed (to add a \cdot for the latter), but it's better than leaving long gaps at the ends of lines. Something that might be thought about. It's a lot smaller change to MathJax than automated line breaking would be.

[Phil Perry]

Further thoughts on line-breaking a long inline equation: I see there are now configuration entries for linebreak(s), but I need something that doesn't rely on the "browser" (or in my case, the PDF creation code) to figure out where the best place is to break an equation across two lines. I'm looking for a conditional break (to fit) at a given location in the equation, rather than an unconditional one. What I need is the final size of a chunk of the equation (as SVG) so I can determine whether there is room for it on the current line. In addition, in some cases, changes to the equation are needed at a break, such as adding a \cdot. What I was thinking along the lines of would be a new operator \br{b}{nb}. The b code could be empty, or some equation code such as \cdot, to be added if the break will be there. If it will not be broken there, nb could be empty, or code to use (such as a space) if no break there. Note that this permits alternate code for break/no-break, rather than just adding in additional code.

The idea would be that my code would scan and remove any \br{b}{nb} entries before feeding it to MathJax. Initially, it would try the equation with all the nb codes. If too long, break at the last (or perhaps at the \br nearest the middle) and use the b code there. If still too long, back up one \br (or the one nearest to the middle) and try again. Eventually, we will find the longest fragment (if any) that fits on this line, and the remainder will be done on the following line(s). As with hyphenation, you may want to have a minimum length of equation at either end (perhaps best done by the user not placing a \br too close to either end).

Is there already something in MathJax which does essentially this, that I could tell how much horizontal space remains in the line (in points), and MathJax does the split for me? If not, could MathJax at least handle any \br it sees in a benign manner, always using the second (nb) code, while my code tests the length and decides whether to use the b code? If MathJax can handle control of breaking inline equations in such a manner, perhaps (La)TeX and other applications might too? The advantage over a simple "dumb" break at the end of a line is that equation code can be added (or dropped), unless MathJax is smart enough to handle all this already. Feel free to start a new thread (and "at" me so I see it) if you think \br would be a good general purpose addition to MathJax (and perhaps other applications).

[Davide Cervone]

Further thoughts on line-breaking a long inline equation: I see there are now configuration entries for linebreak(s),

Yes, v4.0 includes automatically line breaks for both display and in-line equations.

but I need something that doesn't rely on the "browser" (or in my case, the PDF creation code) to figure out where the best place is to break an equation across two lines.

Your code is taking the place of the browser, in this case, so you will need to do the determination of which potential breakpoints to use.  For SVG output, MathJax will already have broken the output into separate SVG nodes with intervening mjx-break elements that provide the potential breakpoints for the browser.  The SVG nodes will have width attributes that give their widths.  These will be in ex units, so you will need to use the font metrics for the font in use to convert that to whatever units you need.  The mjx-break elements will have a "size" attribute that is a number from 0 to 5 indicating the size of the space:

0 = 0em

1 = 0.111em

2 = 0.167em

3 = 0.222em

4 = 0.278em

5 = 0.333em

(basically it is (size + 1) / 18 ems, except for size = 0).  If there is no size attribute, then there should be an explicit letter-spacing CSS property that gives the size of the space; the size is the letter-spacing + 1 em.  Again this is in font-relative units (ams this time), so you will need to use the font metrics to convert to whatever units you are using.

When you decide to make a break, remove the mjx-break element at that breakpoint, as that is what the browser would do (it contains a space that would be removed by the browser).

That should get you what you need in terms of finding in-line breaks.  

in some cases, changes to the equation are needed at a break, such as adding a \cdot.

For in-line breaks, MathJax doesn't do that (because the browser is making the break, and MathJax can't tell if one will be made and so can't insert the needed extra operator).  MathJax does handle the MathML linebreakmultchar attribute when it makes display-math breaks, but can't do that for inline breaks.

What I was thinking along the lines of would be a new operator \br{b}{nb}. The b code could be empty, or some equation code such as \cdot, to be added if the break will be there. If it will not be broken there, nb could be empty, or code to use (such as a space) if no break there. Note that this permits alternate code for break/no-break, rather than just adding in additional code.

The idea would be that my code would scan and remove any \br{b}{nb} entries before feeding it to MathJax. Initially, it would try the equation with all the nb codes. If too long, break at the last (or perhaps at the \br nearest the middle) and use the b code there. If still too long, back up one \br (or the one nearest to the middle) and try again. Eventually, we will find the longest fragment (if any) that fits on this line, and the remainder will be done on the following line(s). As with hyphenation, you may want to have a minimum length of equation at either end (perhaps best done by the user not placing a \br too close to either end).

Since MathJax's SVG output is already broken into pieces at the potential breakpoints, I don't think you need to work that hard to handle this.  You could define your \br macro like

\newcommand{\br}[2]{\data{break={#1}}{#2}}

which will put the "b" LaTeX code into a data-break attribute of the internal MathML for the "nb" code.  Then if you decide to use a particular mjx-break for a breakpoint, you could then find the "b" code from following SVG element (call it node) by using 

node.querySelector('[data-mml-node="math"]').firstChild.getAttribute('data-break')

You could ask MathJax to typeset that code and insert it before "node" (after the break that you insert and before the following SVG element).

Note that MathJax only breaks in-line expressions where TeX would allow that (basically only at top-level BIN and REL nodes).  If you want to allow a break somewhere else, you will need to include something like \allowbreak or \goodbreak in your "nb" code.  E.g.

x\br{\cdot}{\allowbreak}y

would allow a break between the "x" and "y" that would not usually be allowed, and would use \cdot when that break is made.

This approach doesn't allow your alternative code idea, as it only adds more TeX at the breakpoint rather than removing other code.  I'm not sure that is a really practical thing to achieve, as the "nb" code my be what is allowing the break at that location and replacing it by "b" may no longer be a valid breakpoint.  Also, the order of the \br calls in the original TeX may not be the order they are used in the equation.  For example

\newcommand{\switch}[2]{#2x#1}

a\switch{\br{}{-}\br{}{+}}b

would produce "a + x - b" with the first \br being after the "x" and the second before it.  Of course, this is a crazy edge case, but it is possible to produce this kind of thing without too much trouble.  Tying the output node to the original input LaTeX is not easy, in general.

Davide

On Thursday, July 14, 2022 at 11:28:35 PM UTC-4 Phil M Perry wrote:

On 7/14/2022 9:44 AM, Phil M Perry wrote:

A user could possibly manually split an inline equation by turning it into two or more adjacent inline equations, assuming the fragments will fit nicely next to each other.

I just tried this out with HTML in a browser. Say, you have a longish inline equation \(a long equation\) that might cause a large gap at the end of a line if it doesn't fit. Splitting it into \(a\)\(long\)\(equation\) breaks nicely at the end of the line, although the fragments (on the same line) are too close together. Putting a single space between each fragment seems to work pretty well: \(a\) \(long\) \(equation\). The spaces disappear at the line break, which is better than using ~. Any recommendations on this, or experience to report? 

Support for breaking an inline equation at reasonable places (given the remaining space available) would be much appreciated, if it's not already there, as would be guidance on handling such things. Maybe something like a \b given at reasonable break points (such as after a + or -) could help. For multiplication, perhaps a \cdot or something could be added at a break (like a hyphen when breaking a word)?

That's still manually breaking up an equation, and a separate \b and \bmult might be needed (to add a \cdot for the latter), but it's better than leaving long gaps at the ends of lines. Something that might be thought about. It's a lot smaller change to MathJax than automated line breaking would be.

-- 
You received this message because you are subscribed to the Google Groups "MathJax Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mathjax-user...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/mathjax-users/16d2ea1e-9045-4731-aef0-92777fadbadbn%40googlegroups.com.

[Phil Perry]

Hi Davide,

Thank you for the quick and detailed response. Now that I have a working SVG-to-PDF converter, I can start to get to work on integrating MathJax into PDF::Builder within a few months. You say that MathJax is automatically generating line breaks now? I will have to try some long equations and see what it produces, and whether the SVG-to-PDF converter will need an upgrade to handle a new mjx-break tag (element), or if I should plan to remove them (and break up the SVG output) myself before feeding them to the converter. Keep in mind that I have none of the TeX facilities that you suggest using to define new commands, manipulate JS, etc. All I have for input is something like

    The equation <_eqn>a x^2 + b x^1 + c x^0 = 0</_eqn> is a simple quadratic.

(a single SVG equation fed to MathJax wrapped in " "), and all I need to do is wrap it at the end of a line if a single SVG image is too long (wide). I was thinking along the lines of

    The equation <_eqn>a x^2 +\br{}{ }b x^1 +\br{}{ }c x^0 = 0</_eqn> is a simple quadratic.

where it would be fed to MathJax as "a x^2 +", "b x^2 +", and "c x^0 = 0" (if the whole thing doesn't fit in the remaining line), but maybe you've done most of the work in figuring where to split it, and any \br would just be for special cases, such as adding a \cdot for an interrupted multiplication. In any case, I would have to remove any \br before sending it to MathJax, and possibly have to remove any mjx-break elements before feeding the returned SVG to the SVG-to-PDF converter (or else try to have it upgraded). 

The SVG returned from MathJax just goes to a simple-minded black-box SVG-to-PDF converter which I don't think handles mjx-break (yet). I can't manipulate or examine the object model or anything like that. I can add boilerplate settings and new definitions (e.g.., \newcommand\br) to the front of each equation sent to MathJax, if it can handle them.

Lots of experimenting ahead of me!

Phil

[Davide Cervone]

You say that MathJax is automatically generating line breaks now? 

It can, if you configure it to do so.  It is not clear how you are calling MathJax from your perl code.  I assume you are running an external node command, but is it one of your own making, or one from the MathJax-dems-node repository, or what?  I hope you have written what amounts to a MathJax service where you start up the MathJax code once and pass it the expressions to process and it sends back the results using pipes, rather than startup up a command-line tool for each expression, which would incur the MathJax startup costs for every expression processed.  Such a tool is not all that hard to make, and would improve the performance of your converter, but it would probably need to be written in JavaScript, and you may not be proficient in that.

I will have to try some long equations and see what it produces, and whether the SVG-to-PDF converter will need an upgrade to handle a new mjx-break tag (element), or if I should plan to remove them (and break up the SVG output) myself before feeding them to the converter.

To handle breaking in-line expressions, MathJax breaks up the result into multiple SVG elements separated by (HTML) mjx-break elements, so you don't get a single SVG in that case, but a collection of SVG and HTML elements.  Again, I don't know how you are calling MathJax, so can't tell whether you will be able to obtain this list or not.

The mjx-break elements are not within the SVG node itself, but inside the mjx-container node in between separate SVG elements. So The SVG is already broken up.  You will not need to upgrade the SVG-to-PDF for that.

Keep in mind that I have none of the TeX facilities that you suggest using to define new commands,

If you are able to call MathJax, then you have the TeX facilities to define new commands.  You point this out yourself by indicating that you could insert the \newcommand in front of the expression you are processing.  That is certainly one way to do it, but there are others.  For example, if you use the pipe-to-service approach I mentioned above, then you could send any definitions you need to the service when you start it up, and those definitions would be available to the expression it processes later.  This also means that if the math in the page you are processing includes its own definitions, they will be kept properly by the service (they would be lost if you are processing each expression in a new command0line instance of MathJax).  Finally, if the page uses automatic equation numbering or includes \ref or \eqref calls, the service approach would be able to support that (at least for backward references, but not forward references), as the data for that would be maintained in the service, while calling a new command line MathJax on each expression would not be able to do that.

An alternative would be to create your own TeX input extension that defines the needed commands and have your MathJax configuration load that.

manipulate JS, etc.

You are calling MathJax somehow, so there is some mechanism that you are using to run javascript code.  That gives you the ability to manipulate javascript.  Whatever technique you are using to run MathJax could run a modified version of the javascript that calls MathJax.  You are likely going to need to use that in order to get the inline expressions broken into pieces by MathJax in any case.

All I have for input is something like

    The equation <_eqn>a x^2 + b x^1 + c x^0 = 0</_eqn> is a simple quadratic.

(a single SVG equation fed to MathJax wrapped in " "), and all I need to do is wrap it at the end of a line if a single SVG image is too long (wide). I was thinking along the lines of

    The equation <_eqn>a x^2 +\br{}{ }b x^1 +\br{}{ }c x^0 = 0</_eqn> is a simple quadratic.

where it would be fed to MathJax as "a x^2 +", "b x^2 +", and "c x^0 = 0" (if the whole thing doesn't fit in the remaining line), 

Actually, that would not produce the proper results, as something like "a x^2 +" will treat the "+" as a postfix operator (no space before it) rather than an infix one, so the spacing will be wrong.  You also will not get the spacing that goes between the expressions this way.  The mjx-break elements that MathJax creates between the SVG nodes include an attribute that indicates the amount of space that they represent.  That is what I was trying to indicate in my previous message.

You must have a means of determining the size of the pieces you are using to form a line.  While I indicated that the SVG elements produces by MathJax have width attributes that indicate the size of (in ex) units, if you have a means of measuring the size of the svg returned by your converted, you won't need that.  I assume you can get that out the pdf that it has produced (via its bounding box comment or something).  The only thing you would need to worry about, then, is the size of the mjx-break element, and for that you need to know the em-size of the font currently in play.  If you don't have that explicitly, then you can measure it implicitly by measuring the size of a sequence of em-dashes, for example,  That is, make a string of 100 em-dashes (U+2014) or em-spaces (U+2003) and measure the width of that, then divide by 100 to get an empirical measure of the size of an em.  The use that to determine the size of the mjx-break element using the details from my previous message.

The SVG returned from MathJax just goes to a simple-minded black-box SVG-to-PDF converter which I don't think handles mjx-break (yet). I can't manipulate or examine the object model or anything like that.

You must have the SVG element before sending it to the converted.  Even if this is just a serialized SVG string, you can look at the attributes in that string to get the width attribute (if you aren't getting the size from the resulting PDF instead).  It is not clear how you are obtaining the result from MathJax, but you will probably need your own javascript code to properly handle the output for inline breaks.  That javascript could do the interpreting of the mjx-break code and send back the separate SVG node (serialized) together with the sizes of the mjx-break nodes.  Or that could be passed back as a JSON object.  That could be part of how the pipe-to-service process works, for example.

I can add boilerplate settings and new definitions (e.g.., \newcommand\br) to the front of each equation sent to MathJax, if it can handle them.

Yes, MathJax can handle \newcommand and \def commands.  You could also add them to the MathJax configuration (if you are using a mechanism that uses MathJax Components, as I hope you are), or you could create a TeX input extension that defines them.  Lots of ways to get the definitions to MathJax.

In terms of displayed math line breaking, that will produce a single SVG result, so you don't have to worry about mjx-break elements in that case.  But you will have to tell MathJax how wide the container is so that it know where to make the breaks.  That size is given in pixels, and you should tell it the em-size (in pixels), which is usually the font size (e.g., a font size in an HTML document of 12px would have em-size of 12px).  Again, if you don't know the em size, you can measure it implicitly as described above.  How you pass these values to MathJax depends on how you are calling it, so I can't tell you that without knowing more about how you do that.

I hope this clarifies the situation for you.

Davide