Help debugging WeBWorK-TikZ image generation

[Sean Fitzpatrick]

I noticed today that I am not getting back images from my WW server for WW problems in source with TikZ in them.

I haven't generated WeBWorK for APEX in awhile, so this isn't necessarily a recent issue.

Whie running WW generation, all of the *-image-1.tgz files are coming back empty from the server.

I regenerated the APEX.pl file in case it was out of date.

I think I need to go into my server to figure out what isn't working, but I forget where I need to check to access logs and temporary image files.

My server is using WW 2.19, so it's possible that the issue is due to the fact that changes to support 2.19 aren't all merged yet.

I've checked that the sample graphics problem included in the WeBWorK problem is loading correctly. (That is, the TikZ in that question is being built by the server.)

[Alex Jordan]

In your server, the image files are produced at:

/opt/webwork/webwork2/htdocs/tmp/course_name/images/

If image production is successful, you will find them there. 

In pg/lib/LaTeXImage.pm, there is a line that uses `remove_tree` which cleans up temporary files. Comment that out and restart webwork2, and then try to make images if you want the temporary files to persist so you can examine them. I am not 100% sure where the temporary folder for the image building happens though.

/opt/webwork/webwork2/logs/webwork2.log

is where I think you would find error log messages from LaTeXImage.pm

Since the sample chapter image is working for you, I wonder if it is something about bad syntax in APEX.pl. Some characters in your latex-image-preamble may need special attention to work for PG in addition to just working for tex output directly. For example, always use \(...\), not $...$, in latex-image-preamble. There may be other things like that that I am blanking on for now.

--
You received this message because you are subscribed to the Google Groups "PreTeXt support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-suppo...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pretext-support/0339ec15-602e-4bb7-889a-4ebd211e7a72n%40googlegroups.com.

[Sean Fitzpatrick]

Thanks. To clarify, I meant that if I log into WeBWorK and open a .pg file with TikZ in it, it renders correctly.

I just tried building the sample chapter against my server. I'm not seeing any warnings about missing images, but a few problems aren't building successfully on my server: 62, 63, 64, 65, 67, and 68. I'm not sure which ones these are.
I'll try to find some time to dig around later.

To view this discussion visit https://groups.google.com/d/msgid/pretext-support/CA%2BR-jrduHM0L-eXZmjgWMQ9eCZoyUfPtNhmSk%3Dsc17m9gZxBKg%40mail.gmail.com.

[Alex Jordan]

It may help to put labels on exercises, and then I think you will be more easily able to locate the corresponding source.

To view this discussion visit https://groups.google.com/d/msgid/pretext-support/CAH%2BNcPYYLFfdarWXi%3DpnsLF9Rnx-swj7_ctnfjEFqCfv0-7Q%3DA%40mail.gmail.com.

[Sean Fitzpatrick]

That's what I do, but this was just a test run against the WeBWorK sample chapter in the pretext repo to see what it would do. 

The sample chapter doesn't have labels on its exercises.

To view this discussion visit https://groups.google.com/d/msgid/pretext-support/CA%2BR-jrd9My-TMpmo4WJayGEzsX8ySvVX1BrRvep1sZd4Y_BsVw%40mail.gmail.com.

[Sean Fitzpatrick]

I checked the /opt/webwork/webwork2/htdocs/tmp/apex/images folder, and what is strange is that there is precisely one .tgz file there.

There should be many. I don't see anything helpful in the log file.

I also copied some of the generated .pg source files in the webwork-representations file, and pasted them into the problem editor on WeBWorK.

Images are rendering just fine.

I am wondering if images are going to the wrong tmp directory, and pretext can't find them.

I commented out some lines in localOverrides.conf that set temporary directories and restarted webwork2.

Good: now I am getting lots of .tgz files in the tmp/apex/images folder. And they are *not* empty.

Bad: despite the fact that these are being created, and contain what they're supposed to contain, I'm still getting the following sort of thing every time there's a TikZ image in a WW problem:

WARNING : C:\Users\Sean Fitzpatrick\Documents\github\APEXCalculusPTX\generated-assets\webwork\images\webwork-ex-limit-intro-numerical-graphical-3-image-1.tgz did not contain a .tex file
WARNING : C:\Users\Sean Fitzpatrick\Documents\github\APEXCalculusPTX\generated-assets\webwork\images\webwork-ex-limit-intro-numerical-graphical-3-image-1.tgz did not contain a .pdf file
WARNING : C:\Users\Sean Fitzpatrick\Documents\github\APEXCalculusPTX\generated-assets\webwork\images\webwork-ex-limit-intro-numerical-graphical-3-image-1.tgz did not contain a .svg file
WARNING : C:\Users\Sean Fitzpatrick\Documents\github\APEXCalculusPTX\generated-assets\webwork\images\webwork-ex-limit-intro-numerical-graphical-3-image-1.tgz did not contain a .png file

[Sean Fitzpatrick]

AHA!! The plot thickens.

This is not a problem with my server. It has something to do with how this part of the script is running on Windows machines.

If I run 'pretext generate webwork -t html', using CLI 2.14.0 on a Linux computer, everything works just fine.

But doing the same on Windows (with the same CLI version) leads to missing images.

[Oscar Levin]

That is very strange that there is a different behavior for windows than linux.  Can you send the log file from that pretext run?

To view this discussion visit https://groups.google.com/d/msgid/pretext-support/e26014e2-d3fc-48c7-9d15-d39b13955d08n%40googlegroups.com.

[Sean Fitzpatrick]

I've just left my office! (And the computers contained therein.) But I'll send you the log files tomorrow.

I can download the .tgz files from the server onto my Windows computer, and open them, and confirm that they have contents. So I'm not sure why pretext thinks they're empty.

To view this discussion visit https://groups.google.com/d/msgid/pretext-support/CAOU9BaX1b2Br8T0PcKJnawXfKd_zeuijABVhTAcTgf6cKApYiQ%40mail.gmail.com.

[Sean Fitzpatrick]

Here is a recent log file. I checked logs from around this time last year, and I was getting clean runs on everything.

Going back, I am seeing this problem with CLI 2.12 in January, and perhaps I didn't notice it, but 2.6.2 ran into the same issue last September, and 2.4.0 last May.

I think something changed in Windows, because I have a clean run using 2.3.9 on April 12, 2024, and a failed run on April 18, 2024, also using 2.3.9.

(I can't think of any reason why I would have made changes to the WeBWorK server around that time.)

That would have been right at the end of term, at which point I would have abandoned my Windows computer for Linux for the summer.

[Oscar Levin]

Thanks.  I'ts frustrating that the log didn't show anything helpful.  Could you try again with -v debug set?  (so pretext -v debug generate....)

The other thing I'd like to try is to check whether you have developer mode enabled.  See https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode.  If you can, try enabling it and maybe the problem goes away.  This doesn't solve the main issue, but might help me figure out how to make it work better.

[Sean Fitzpatrick]

Here is a log file with debugging turned on.

I don't have developer mode enabled (yet).

That's the next thing to try.

[Sean Fitzpatrick]

Trying to open developer settings crashes the Windows settings app... Le sigh.

[Sean Fitzpatrick]

OK, a reboot and developer settings are on. But I'm still not getting images from the .tgz files.

[Sean Fitzpatrick]

It's been awhile, but I'm giving this a nudge because it's still an issue. 

Since I'm on vacation I decided to only travel with one work computer, and I needed my Windows computer for reasons that wouldn't apply if I could actually vacation. 

Anyway... I was trying to do some testing with a PDF build, and got hung up because I can't get any images out of WeBWorK problems on Windows. 

Since it's been awhile: on Windows, when generating WeBWorK, it claims that the .tgz file it gets back is empty for each exercise with a Tikz image. This problem is unique to Windows: the same source builds without issue on Linux. 

I can just wait until I'm reunited with my Linux laptop in two weeks, but if someone only has Windows, they would not be able to build a PreTeXt book that has Tikz image in WeBWorK.

[D. Brian Walton]

Sean,

I asked Rob's assistant what to make of the report and got the following reply. It might help whoever knows how to resolve the issue.

- Brian

Claude's reply:

I dug through pretext/lib/webwork.py, the code path that pulls WeBWorK-rendered images (~line 630–729 in the image-download loop). The likely culprit is here:

python

if image_extension == ".tgz":
    tgzfile = tarfile.open(destination_image_file)
    tgzfile.extractall(os.path.join(ww_images_dir))
    tgzfile.close()
    msg = "{} did not contain a .{} file"
    for ext in ["tex", "pdf", "svg", "png"]:
        try:
            os.rename(
                os.path.join(ww_images_dir, "image.{}".format(ext)),
                os.path.join(ww_images_dir, ptx_image_name + ".{}".format(ext)),
            )
        except:
            log.warning(msg.format(destination_image_file, ext))
    os.remove(os.path.join(ww_images_dir, ptx_image_filename))

The tgz is extracted (producing image.tex, image.pdf, etc. in ww_images_dir), then each is renamed to the problem-specific filename with os.rename.

That os.rename call is the OS-sensitive spot. os.rename behaves differently on Windows vs. POSIX/Linux when the destination file already exists: on Linux it silently overwrites; on Windows it raises FileExistsError instead. This code swallows that error with a bare except, so on Windows every rename after the first successful build (or any rerun where a target file with that name already exists in ww_images_dir) fails, and you get exactly the four warnings ("did not contain a .tex/.pdf/.svg/.png file") reported — even though the tgz genuinely contained those files and extraction succeeded. This matches the reporter's experience: same source, same WeBWorK server, works cleanly on Linux, fails consistently on Windows, and the "empty tgz" framing was a red herring from an earlier, separate server-side issue (localOverrides.conf temp dirs) that they'd already fixed.

The fix (not applied, since you asked me not to debug) would be replacing os.rename with os.replace, which has consistent overwrite-if-exists semantics on both platforms — or explicitly removing any stale target file before renaming.

To view this discussion visit https://groups.google.com/d/msgid/pretext-support/3b134f4c-24c4-4085-831b-35a911d9c1c1n%40googlegroups.com.

[Rob Beezer]

Thanks, Brian, that sounds plausible.

With this, I can have my assistant make a PR, test on Linux, and then let Sean
test on Windows.

Rob

On 7/27/26 06:27, D. Brian Walton wrote:
> Sean,
>
> I asked Rob's assistant what to make of the report and got the following reply.
> It might help whoever knows how to resolve the issue.
>
> - Brian
>
> Claude's reply:
>

> I dug through |pretext/lib/webwork.py|, the code path that pulls WeBWorK-

> rendered images (~line 630–729 in the image-download loop). The likely culprit
> is here:
>
> python
>
> |if image_extension == ".tgz": tgzfile = tarfile.open(destination_image_file)
> tgzfile.extractall(os.path.join(ww_images_dir)) tgzfile.close() msg = "{} did
> not contain a .{} file" for ext in ["tex", "pdf", "svg", "png"]: try:
> os.rename( os.path.join(ww_images_dir, "image.{}".format(ext)),
> os.path.join(ww_images_dir, ptx_image_name + ".{}".format(ext)), ) except:
> log.warning(msg.format(destination_image_file, ext))
> os.remove(os.path.join(ww_images_dir, ptx_image_filename))|
>
> The tgz is extracted (producing |image.tex|, |image.pdf|, etc. in |

> ww_images_dir|), then each is renamed to the problem-specific filename with |

> os.rename|.
>
> That |os.rename| call is the OS-sensitive spot. |os.rename| behaves differently
> on Windows vs. POSIX/Linux when the destination file already exists: on Linux it
> silently overwrites; on Windows it raises |FileExistsError| instead. This code

> swallows that error with a bare |except|, so on Windows every rename after the

> first successful build (or any rerun where a target file with that name already

> exists in |ww_images_dir|) fails, and you get exactly the four warnings ("did

> not contain a .tex/.pdf/.svg/.png file") reported — even though the tgz
> genuinely contained those files and extraction succeeded. This matches the
> reporter's experience: same source, same WeBWorK server, works cleanly on Linux,
> fails consistently on Windows, and the "empty tgz" framing was a red herring
> from an earlier, separate server-side issue (localOverrides.conf temp dirs) that
> they'd already fixed.
>
> The fix (not applied, since you asked me not to debug) would be replacing |

> os.rename| with |os.replace|, which has consistent overwrite-if-exists semantics

> development#activate-developer-mode <https://
> learn.microsoft.com/en-us/windows/apps/get-started/enable-
> your-device-for-development#activate-developer-mode>.  If

> support/0339ec15-602e-4bb7-889a-4ebd211e7a72n%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/0339ec15-602e-4bb7-889a-4ebd211e7a72n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message
> because you are subscribed
> to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this
> group and stop receiving
> emails from it, send an
> email to pretext-
> suppo...@googlegroups.com.
> To view this discussion
> visit https://
> groups.google.com/d/msgid/

> pretext-support/CA%2BR-
> jrduHM0L-
> eXZmjgWMQ9eCZoyUfPtNhmSk%3Dsc17m9gZxBKg%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-support/CA%2BR-jrduHM0L-eXZmjgWMQ9eCZoyUfPtNhmSk%3Dsc17m9gZxBKg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message
> because you are subscribed to
> the Google Groups "PreTeXt
> support" group.
> To unsubscribe from this group
> and stop receiving emails from
> it, send an email to pretext-
> suppo...@googlegroups.com.
> To view this discussion visit
> https://groups.google.com/d/
> msgid/pretext-support/

> CAH%2BNcPYYLFfdarWXi%3DpnsLF9Rnx-swj7_ctnfjEFqCfv0-7Q%3DA%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-support/CAH%2BNcPYYLFfdarWXi%3DpnsLF9Rnx-swj7_ctnfjEFqCfv0-7Q%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because
> you are subscribed to the Google
> Groups "PreTeXt support" group.
> To unsubscribe from this group and
> stop receiving emails from it, send
> an email to pretext-
> suppo...@googlegroups.com.
>
> To view this discussion visit
> https://groups.google.com/d/msgid/

> pretext-support/CA%2BR-jrd9My-
> TMpmo4WJayGEzsX8ySvVX1BrRvep1sZd4Y_BsVw%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-support/CA%2BR-jrd9My-TMpmo4WJayGEzsX8ySvVX1BrRvep1sZd4Y_BsVw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are
> subscribed to the Google Groups "PreTeXt
> support" group.
> To unsubscribe from this group and stop
> receiving emails from it, send an email to
> pretext-suppo...@googlegroups.com.
>
> To view this discussion visit https://
> groups.google.com/d/msgid/pretext-support/

> e26014e2-d3fc-48c7-9d15-
> d39b13955d08n%40googlegroups.com <https://
> groups.google.com/d/msgid/pretext-support/
> e26014e2-d3fc-48c7-9d15-
> d39b13955d08n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to pretext-suppo...@googlegroups.com <mailto:pretext-
> support+u...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-

> support/3b134f4c-24c4-4085-831b-35a911d9c1c1n%40googlegroups.com <https://
> groups.google.com/d/msgid/pretext-
> support/3b134f4c-24c4-4085-831b-35a911d9c1c1n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to pretext-suppo...@googlegroups.com <mailto:pretext-
> support+u...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/pretext-support/

> CAH7VRoLaf_jG%3DAMVmUCa0RNEoGacE6s2RpHbG3b83GH%3DCjr1XQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/pretext-support/
> CAH7VRoLaf_jG%3DAMVmUCa0RNEoGacE6s2RpHbG3b83GH%3DCjr1XQ%40mail.gmail.com?
> utm_medium=email&utm_source=footer>.

[Rob Beezer]

Sean - can you test on Windows in your current state? You could even read the
PR, and make just teh rename() -> replace() change on whatever you have in
place right now. The movement of the error messages was just a bit of housekeeping.

If not, perhaps someone else who develops on Windows can take this out for a spin.

Thanks, Brian, for the nudge.

https://github.com/PreTeXtBook/pretext/pull/3078

Rob

On 7/27/26 06:27, D. Brian Walton wrote:

> Sean,
>
> I asked Rob's assistant what to make of the report and got the following reply.
> It might help whoever knows how to resolve the issue.
>
> - Brian
>
> Claude's reply:
>

> I dug through |pretext/lib/webwork.py|, the code path that pulls WeBWorK-

> rendered images (~line 630–729 in the image-download loop). The likely culprit
> is here:
>
> python
>
> |if image_extension == ".tgz": tgzfile = tarfile.open(destination_image_file)
> tgzfile.extractall(os.path.join(ww_images_dir)) tgzfile.close() msg = "{} did
> not contain a .{} file" for ext in ["tex", "pdf", "svg", "png"]: try:
> os.rename( os.path.join(ww_images_dir, "image.{}".format(ext)),
> os.path.join(ww_images_dir, ptx_image_name + ".{}".format(ext)), ) except:
> log.warning(msg.format(destination_image_file, ext))
> os.remove(os.path.join(ww_images_dir, ptx_image_filename))|
>

> The tgz is extracted (producing |image.tex|, |image.pdf|, etc. in |

> ww_images_dir|), then each is renamed to the problem-specific filename with |

> os.rename|.
>
> That |os.rename| call is the OS-sensitive spot. |os.rename| behaves differently
> on Windows vs. POSIX/Linux when the destination file already exists: on Linux it
> silently overwrites; on Windows it raises |FileExistsError| instead. This code

> swallows that error with a bare |except|, so on Windows every rename after the

> first successful build (or any rerun where a target file with that name already

> exists in |ww_images_dir|) fails, and you get exactly the four warnings ("did

> not contain a .tex/.pdf/.svg/.png file") reported — even though the tgz
> genuinely contained those files and extraction succeeded. This matches the
> reporter's experience: same source, same WeBWorK server, works cleanly on Linux,
> fails consistently on Windows, and the "empty tgz" framing was a red herring
> from an earlier, separate server-side issue (localOverrides.conf temp dirs) that
> they'd already fixed.
>
> The fix (not applied, since you asked me not to debug) would be replacing |

> os.rename| with |os.replace|, which has consistent overwrite-if-exists semantics

> developer mode enabled.  See https://learn.microsoft.com/en-
> us/windows/apps/get-started/enable-your-device-for-
> development#activate-developer-mode <https://
> learn.microsoft.com/en-us/windows/apps/get-started/enable-

> your-device-for-development#activate-developer-mode>.  If

> visit https://
> groups.google.com/d/
> msgid/pretext-
> support/0339ec15-602e-4bb7-889a-4ebd211e7a72n%40googlegroups.com <https://groups.google.com/d/msgid/pretext-support/0339ec15-602e-4bb7-889a-4ebd211e7a72n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>

> --
> You received this message
> because you are subscribed
> to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this
> group and stop receiving
> emails from it, send an
> email to pretext-
> suppo...@googlegroups.com.
> To view this discussion

> visit https://
> groups.google.com/d/msgid/
> pretext-support/CA%2BR-
> jrduHM0L-
> eXZmjgWMQ9eCZoyUfPtNhmSk%3Dsc17m9gZxBKg%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-support/CA%2BR-jrduHM0L-eXZmjgWMQ9eCZoyUfPtNhmSk%3Dsc17m9gZxBKg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>

> --
> You received this message
> because you are subscribed to
> the Google Groups "PreTeXt
> support" group.
> To unsubscribe from this group
> and stop receiving emails from
> it, send an email to pretext-
> suppo...@googlegroups.com.
> To view this discussion visit

> https://groups.google.com/d/
> msgid/pretext-support/
> CAH%2BNcPYYLFfdarWXi%3DpnsLF9Rnx-swj7_ctnfjEFqCfv0-7Q%3DA%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-support/CAH%2BNcPYYLFfdarWXi%3DpnsLF9Rnx-swj7_ctnfjEFqCfv0-7Q%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>

> --
> You received this message because
> you are subscribed to the Google
> Groups "PreTeXt support" group.
> To unsubscribe from this group and
> stop receiving emails from it, send
> an email to pretext-
> suppo...@googlegroups.com.
>
> To view this discussion visit

> https://groups.google.com/d/msgid/
> pretext-support/CA%2BR-jrd9My-
> TMpmo4WJayGEzsX8ySvVX1BrRvep1sZd4Y_BsVw%40mail.gmail.com <https://groups.google.com/d/msgid/pretext-support/CA%2BR-jrd9My-TMpmo4WJayGEzsX8ySvVX1BrRvep1sZd4Y_BsVw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>

> --
> You received this message because you are
> subscribed to the Google Groups "PreTeXt
> support" group.
> To unsubscribe from this group and stop
> receiving emails from it, send an email to
> pretext-suppo...@googlegroups.com.
>

> To view this discussion visit https://
> groups.google.com/d/msgid/pretext-support/
> e26014e2-d3fc-48c7-9d15-
> d39b13955d08n%40googlegroups.com <https://
> groups.google.com/d/msgid/pretext-support/
> e26014e2-d3fc-48c7-9d15-
> d39b13955d08n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.
>

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to pretext-suppo...@googlegroups.com <mailto:pretext-
> support+u...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/pretext-
> support/3b134f4c-24c4-4085-831b-35a911d9c1c1n%40googlegroups.com <https://
> groups.google.com/d/msgid/pretext-
> support/3b134f4c-24c4-4085-831b-35a911d9c1c1n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.
>

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to pretext-suppo...@googlegroups.com <mailto:pretext-
> support+u...@googlegroups.com>.

[Sean Fitzpatrick]

Interesting! 

So this will be down to the fact that the 'mv' command in Linux corresponds to two different commands in Windows! 

Rob, if you at me on GitHub I'll sneak some time to test this. Or I could just edit that line in my local copy.

To view this discussion visit https://groups.google.com/d/msgid/pretext-support/MTAwMDAyMXdqZnlGZ3c.1785164499%40pnsh.