ImageMagick and the windows convert command

75 views
Skip to first unread message

Oscar Levin

unread,
Sep 27, 2023, 8:17:57 AM9/27/23
to PreTeXt development
I finally tried converting latex-image to png, and discovered I can't do this on Windows. This is because on my current laptop I have not installed ImageMagick.  I didn't get a warning though because Windows does have a `convert` command, used for converting disks to other formats.  Thankfully the options passed to `convert` by the pretext script just give an error and didn't reformat my hard drive.

I dug a little deeper, and it seems like the most recent ImageMagick version 7 uses a different command for converting: it's now `magick`.  

But it also seems like python might have a reliable module to do the conversion: pyMuPDF.  See https://pymupdf.readthedocs.io/en/latest/tutorial.html.

Has anyone tried this?  Is there interest?

Andrew Scholer

unread,
Sep 27, 2023, 10:56:30 AM9/27/23
to prete...@googlegroups.com
I have no experience with that module. But assuming it does what it is supposed to, relying on it instead of exec'ing ImageMagick seems like it would be easier in a lot of ways.

Andrew Scholer (he/him/his)
Computer Science Instructor/Program Chair
Chemeketa Community College


--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/915ddaf0-d4cc-4f54-8da7-ed2eecbd845en%40googlegroups.com.

Rob Beezer

unread,
Sep 27, 2023, 11:20:37 AM9/27/23
to prete...@googlegroups.com
Agreed - anything we can do via *reliable* Python modules is a win. Is there a
PyPi package for LaTeX? ;-)

Rob

On 9/27/23 07:55, Andrew Scholer wrote:
> I have no experience with that module. But assuming it does what it is supposed
> to, relying on it instead of exec'ing ImageMagick seems like it would be easier
> in a lot of ways.
>
> Andrew Scholer (he/him/his)
> Computer Science Instructor/Program Chair
> Chemeketa Community College
> 503.589.7649
> computerscience.chemeketa.edu/people/andrew-scholer/
> <http://computerscience.chemeketa.edu/people/andrew-scholer/>
>
>
> On Wed, Sep 27, 2023 at 5:17 AM Oscar Levin <oscar...@gmail.com
> <mailto:oscar...@gmail.com>> wrote:
>
> I finally tried converting latex-image to png, and discovered I can't do
> this on Windows. This is because on my current laptop I have not installed
> ImageMagick.  I didn't get a warning though because Windows does have a
> `convert` command, used for converting disks to other formats.  Thankfully
> the options passed to `convert` by the pretext script just give an error and
> didn't reformat my hard drive.
>
> I dug a little deeper, and it seems like the most recent ImageMagick version
> 7 uses a different command for converting: it's now `magick`.
>
> But it also seems like python might have a reliable module to do the
> conversion: pyMuPDF.  See
> https://pymupdf.readthedocs.io/en/latest/tutorial.html
> <https://pymupdf.readthedocs.io/en/latest/tutorial.html>.
>
> Has anyone tried this?  Is there interest?
>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pretext-dev...@googlegroups.com
> <mailto:pretext-dev...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pretext-dev/915ddaf0-d4cc-4f54-8da7-ed2eecbd845en%40googlegroups.com <https://groups.google.com/d/msgid/pretext-dev/915ddaf0-d4cc-4f54-8da7-ed2eecbd845en%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-dev...@googlegroups.com
> <mailto:pretext-dev...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pretext-dev/CACm44N-NP6z_Z394ymEXHZ8uaq%2B%3DnvecVNUffKm-Vhky1QYbaw%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-dev/CACm44N-NP6z_Z394ymEXHZ8uaq%2B%3DnvecVNUffKm-Vhky1QYbaw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Steven Clontz

unread,
Sep 28, 2023, 2:30:01 PM9/28/23
to prete...@googlegroups.com
https://github.com/JelteF/PyLaTeX

But that's actually wild: you create an abstraction of your LaTeX document in Python, which then outputs LaTeX code (and runs e.g. pdflatex on it if you want).

There's a few other Python/LaTeX libraries, but none seem to do what we actually want: a simple abstraction that takes a string of LaTeX markup (or a path to a file with LaTeX markup) and runs a LaTeX engine on it.


To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/MTAwMDAxOC5iZWV6ZXI.1695828034%40quikprotect.

Oscar Levin

unread,
Sep 28, 2023, 10:00:13 PM9/28/23
to PreTeXt development
So it is working even better than I hoped.  Not only does pyMuPDF read pdfs, convert to raster images and save as png (or lots of other formats), it also can read PDFs as SVGs, and those can be saved.  As such, this would remove the dependency on pdf2svg, which has long been a stumbling block on windows and mac for new users.  But also on ImageMagick/ghostscript.

Now for the tricky bit: how would you (Rob) like to test this?  I've produced a variety of png and svg images from latex-image, and everything looks fine.  A pull request would be easy, but would it make sense to initially have a (throwaway) PR that produces images using both methods, so we can compare and contrast?

Rob Beezer

unread,
Sep 28, 2023, 10:28:28 PM9/28/23
to prete...@googlegroups.com
Fantastic! Do a reasonably careful implementation, and hide it behind a debug.* stringparam that creates a boolean variable that defaults to old school processing. Anybody in the know can test.

Once happy, we back out the flag and go with the more modern approach.

See debug.knowl-production for a model. Might disappear Sunday night but you know how to temporarily checkout a prior commit.

Thanks,
Rob
>>> > 503.589.7649 <(503)%20589-7649>

Oscar Levin

unread,
Sep 28, 2023, 11:05:27 PM9/28/23
to PreTeXt development
I did something similar, but not a stringparam as this is entirely in the pretext/pretext script.  A new parameter for `latex_image_generation`, defaults to `False`, and the CLI will allow authors to test.

Rob Beezer

unread,
Sep 28, 2023, 11:13:33 PM9/28/23
to prete...@googlegroups.com
Right! Forgot where the action is. ;-)
>> >>> > 503.589.7649 <(503)%20589-7649> <(503)%20589-7649>

Rob Beezer

unread,
Sep 28, 2023, 11:14:50 PM9/28/23
to prete...@googlegroups.com
That is wild. But not much in the PyPi world surprises me anymore

Alex Jordan

unread,
Sep 29, 2023, 11:56:19 AM9/29/23
to prete...@googlegroups.com
pdf2svg is also a problem to get on an SE Linux system.

I applied pymupdf to a PDF from one of my latex images, and saved the resulting svg. It looks good.

Bonus points for Oscar if you can manipulate the resulting svg file, and insert a "desc" element whose contents is the image's "shortdescription".


Rob Beezer

unread,
Sep 29, 2023, 1:10:17 PM9/29/23
to prete...@googlegroups.com
I took the pull request for a drive. Very promising, but maybe needs some work
or more evaluation.

https://github.com/PreTeXtBook/pretext/pull/2081

* Requires Python 3.8. PreTeXt is at 3.6, so this would be a change.

* PNG seem to double in file size? (Sample size = 2!) Tried using "exiftool"
to read compressions but no luck. Unless somebody has a good reason otherwise,
I'd use the highest compression possible (level 9?).

* SVG are smaller. Maybe too small? "tikz-venn-diagram" from the sample
article has three colored, overlapping disks. In the new SVG they are
translucent, I can see the background (B/W checkerboard in my "eog" viewer). So
the image really is different. Did not read the tikz to see which is correct.
The SVG is *much* simpler, and in particular is missing a huge encoded raster
image (I think) - not sure how that happens?

So enough evidence for me that certainly more careful evaluation is called for.

I'll merge the PR as long as we are very clear this is present for testing and
evaluation only and warn that it might go away. Developers using the
pretext/pretext script will need to "pip install" the module and edit (on a
branch) the flag being sent to "latex_image_conversion".

Thanks,
Rob

On 9/29/23 08:56, Alex Jordan wrote:
> pdf2svg is also a problem to get on an SE Linux system.
>
> I applied pymupdf to a PDF from one of my latex images, and saved the resulting
> svg. It looks good.
>
> Bonus points for Oscar if you can manipulate the resulting svg file, and insert
> a "desc" element whose contents is the image's "shortdescription".
>
>
> On Thu, Sep 28, 2023 at 8:14 PM Rob Beezer <bee...@privacyport.com
> <mailto:bee...@privacyport.com>> wrote:
>
> That is wild.  But not much in the PyPi world surprises me anymore
>
> On September 28, 2023 11:29:41 AM PDT, Steven Clontz
> <steven...@gmail.com <mailto:steven...@gmail.com>> wrote:
> >https://github.com/JelteF/PyLaTeX <https://github.com/JelteF/PyLaTeX>
> >
> >But that's actually wild: you create an abstraction of your LaTeX document
> >in Python, which then outputs LaTeX code (and runs e.g. pdflatex on it if
> >you want).
> >
> >There's a few other Python/LaTeX libraries, but none seem to do what we
> >actually want: a simple abstraction that takes a string of LaTeX markup (or
> >a path to a file with LaTeX markup) and runs a LaTeX engine on it.
> >
> >
> >Steven Clontz
> >https://clontz.org <https://clontz.org> - steven...@gmail.com
> <mailto:steven...@gmail.com>
> >
> >
> >On Wed, Sep 27, 2023 at 10:20 AM Rob Beezer <bee...@privacyport.com
> <mailto:bee...@privacyport.com>> wrote:
> >
> >> Agreed - anything we can do via *reliable* Python modules is a win.  Is
> >> there a
> >> PyPi package for LaTeX?  ;-)
> >>
> >> Rob
> >>
> >> On 9/27/23 07:55, Andrew Scholer wrote:
> >> > I have no experience with that module. But assuming it does what it is
> >> supposed
> >> > to, relying on it instead of exec'ing ImageMagick seems like it would be
> >> easier
> >> > in a lot of ways.
> >> >
> >> > Andrew Scholer (he/him/his)
> >> > Computer Science Instructor/Program Chair
> >> > Chemeketa Community College
> >> > 503.589.7649
> >> > computerscience.chemeketa.edu/people/andrew-scholer/
> <http://computerscience.chemeketa.edu/people/andrew-scholer/>
> >> > <http://computerscience.chemeketa.edu/people/andrew-scholer/
> <http://computerscience.chemeketa.edu/people/andrew-scholer/>>
> >> >
> >> >
> >> > On Wed, Sep 27, 2023 at 5:17 AM Oscar Levin <oscar...@gmail.com
> <mailto:oscar...@gmail.com>
> <mailto:pretext-dev%2Bunsu...@googlegroups.com>
> >> >     <mailto:pretext-dev...@googlegroups.com
> <mailto:pretext-dev%2Bunsu...@googlegroups.com>>.
> https://groups.google.com/d/msgid/pretext-dev/915ddaf0-d4cc-4f54-8da7-ed2eecbd845en%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pretext-dev/915ddaf0-d4cc-4f54-8da7-ed2eecbd845en%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> >.
> >> >
> >> > --
> >> > You received this message because you are subscribed to the Google
> >> Groups
> >> > "PreTeXt development" group.
> >> > To unsubscribe from this group and stop receiving emails from it, send
> >> an email
> >> > to pretext-dev...@googlegroups.com
> <mailto:pretext-dev%2Bunsu...@googlegroups.com>
> >> > <mailto:pretext-dev...@googlegroups.com
> <mailto:pretext-dev%2Bunsu...@googlegroups.com>>.
> https://groups.google.com/d/msgid/pretext-dev/CACm44N-NP6z_Z394ymEXHZ8uaq%2B%3DnvecVNUffKm-Vhky1QYbaw%40mail.gmail.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pretext-dev/CACm44N-NP6z_Z394ymEXHZ8uaq%2B%3DnvecVNUffKm-Vhky1QYbaw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >> >.
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "PreTeXt development" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an
> >> email to pretext-dev...@googlegroups.com
> <mailto:pretext-dev%2Bunsu...@googlegroups.com>.
> >> To view this discussion on the web visit
> >>
> https://groups.google.com/d/msgid/pretext-dev/MTAwMDAxOC5iZWV6ZXI.1695828034%40quikprotect <https://groups.google.com/d/msgid/pretext-dev/MTAwMDAxOC5iZWV6ZXI.1695828034%40quikprotect>
> >> .
> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pretext-dev...@googlegroups.com
> <mailto:pretext-dev%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pretext-dev/MTAwMDAwMC5iZWV6ZXI.1695957288%40quikprotect <https://groups.google.com/d/msgid/pretext-dev/MTAwMDAwMC5iZWV6ZXI.1695957288%40quikprotect>.
>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-dev...@googlegroups.com
> <mailto:pretext-dev...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pretext-dev/CA%2BR-jrf2SAGbpbEitc0zP1U9mKhxRgRiq_Ae4L9WL2N9%2BgfQ%3Dw%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-dev/CA%2BR-jrf2SAGbpbEitc0zP1U9mKhxRgRiq_Ae4L9WL2N9%2BgfQ%3Dw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Oscar Levin

unread,
Sep 29, 2023, 1:10:36 PM9/29/23
to PreTeXt development
Oooh... the first time I read that I thought, don't be ridiculous, I would need so much extra credit.  But actually, the svg that is produced is just a string inside python, so it should be trivial to insert the shortdescription text.  I don't know svg formatting well enough to know where it needs to go, but I bet I can figure it out. 

Steven Clontz

unread,
Sep 29, 2023, 1:12:44 PM9/29/23
to prete...@googlegroups.com
> But actually, the svg that is produced is just a string inside python, so it should be trivial to insert the shortdescription text.

Suggestion: use lxml to parse the SVG string as an lxml object, then use lxml to insert the appropriate element with appropriate text.

Alex Jordan

unread,
Sep 29, 2023, 1:15:15 PM9/29/23
to prete...@googlegroups.com
Maybe forget I mentioned "desc". Yes, it would be a nice touch to insert that. But actually it should be something we do regardless of the SVG production mechanism. And it shouldn't be harder if it's one mechanism or another. We should just open the file, and insert it. So it's an idea for a separate PR.

Rob Beezer

unread,
Sep 29, 2023, 1:42:44 PM9/29/23
to prete...@googlegroups.com
On 9/29/23 10:10, Oscar Levin wrote:
>  I
> don't know svg formatting well enough to know where it needs to go, but I bet I
> can figure it out.

Or ask J. David Eisenberg, who wrote *the* O'Reilly book on SVGs.
Reply all
Reply to author
Forward
0 new messages