failure compiling tex files (on Debian Linux)

[Sanjoy Mahajan]

I've been meaning for a while to switch the build system for my
textbooks from make to tup. I'm now biting that bullet for my next
textbook. But I cannot get tex files to compile when tex is called by
tup.

Here is a minimal example using plain TeX. I'm using Debian Linux
'unstable', which has a version of texlive 2025
and has tup 0.8-1+b1 [debian].

The Tupfile is one line:

: test.tex |> pdftex %f |> %B.pdf %B.log

The test.tex file is these four lines:

\def\x{\times}
\message{HELLO THERE}
$2\x2$
\end

When I run tup, the pdftex command fails as follows:

[ tup ] [0.000s] Scanning filesystem...
[ tup ] [0.000s] Reading in new environment variables...
[ tup ] [0.001s] No Tupfiles to parse.
[ tup ] [0.001s] No files to delete.
[ tup ] [0.001s] Executing Commands...
* 100% 0) pdftex test.tex
This is pdfTeX, Version 3.141592653-2.6-1.40.26 (TeX Live 2025/dev/Debian) (preloaded format=pdftex)
restricted \write18 enabled.
entering extended mode
(./test.tex)
*
! Emergency stop.
<*>

! ==> Fatal error occurred, no output PDF file produced!
Transcript written on test.log.
*** tup messages ***
*** Command ID=12 failed with return value 1
*** tup: 1 job failed.

However, test.tex compiles fine run from the command line:

$ pdftex test.tex
This is pdfTeX, Version 3.141592653-2.6-1.40.26 (TeX Live 2025/dev/Debian) (preloaded format=pdftex)
restricted \write18 enabled.
entering extended mode
(./test.tex HELLO THERE [1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
)</usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></usr/
share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmsy10.pfb>
Output written on test.pdf (1 page, 16123 bytes).
Transcript written on test.log.

The "HELLO THERE" text is from the \message macro, which I included so
that I could see whether test.tex is being loaded. In the working run
(from the command line directly), the message shows up. However, in the
failing run (when pdftex is run by tup), the message does not -- which
must somehow be related to the failure to compile, but I don't know
which problem is a symptom and what the root cause is.

Am I doing something silly? Or is pdftex doing something that doesn't
play well with the filesystem trickery that tup does?

I tried explicitly including all my environment variables (except PWD,
OLDPWD, HOME, and PATH) using 'export' statements, but no luck.
I've also tried stracing but couldn't see why it has gone awry.

--
-Sanjoy

"An error can never become true however many times you repeat it.
The truth can never be wrong, even if no one ever hears about it."
--Mahatma Gandhi

"Not to know is bad. Not to wish to know is worse." -African proverb

[Sanjoy Mahajan]

On 2025-07-15 17:25, Sanjoy Mahajan <san...@mit.edu> wrote:

> I've also tried stracing but couldn't see why it has gone awry.

On second thought, the strace did reveal a strange behavior. Here is
the first time that test.tex is read from (by pdftex, which is being run
under tup). It is fine. The newfstatat and fstat calls correctly state
that the file contains 49 bytes. Then the read syscall shows these 49
bytes being read in.

825721 access("./test.tex", R_OK) = 0
825721 newfstatat(AT_FDCWD</home/sanjoy/calculus-textbook/Tuptextest>, "./test.tex", {st_dev=makedev(0, 0x53), st_ino=14, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=8, st_size=49, st_atime=1752591356 /* 2025-07-15T16:55:56.190571542+0200 */, st_atime_nsec=190571542, st_mtime=1752591347 /* 2025-07-15T16:55:47.266565209+0200 */, st_mtime_nsec=266565209, st_ctime=1752591347 /* 2025-07-15T16:55:47.266565209+0200 */, st_ctime_nsec=266565209}, 0) = 0
825721 openat(AT_FDCWD</home/sanjoy/calculus-textbook/Tuptextest>, "./test.tex", O_RDONLY) = 3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>
825721 fstat(3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>, {st_dev=makedev(0, 0x53), st_ino=14, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=8, st_size=49, st_atime=1752591356 /* 2025-07-15T16:55:56.190571542+0200 */, st_atime_nsec=190571542, st_mtime=1752591347 /* 2025-07-15T16:55:47.266565209+0200 */, st_mtime_nsec=266565209, st_ctime=1752591347 /* 2025-07-15T16:55:47.266565209+0200 */, st_ctime_nsec=266565209}) = 0
825721 read(3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>, "\\def\\x{\\times}\n\\message{HELLO THERE}\n$2\\x2$\n\\end\n", 4096) = 49
825721 close(3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>) = 0

But the second read of test.tex goes awry. It is allegedly from the
same file, at least by name, but this time the file has zero length
(st_size=0) and is owned by user nobody (65534:65534). When this
zero-length file is read in (when the read syscall returns 0) and
nothing else is given to pdftex, then pdftex correctly complains in its
log file about 'End of file on the terminal!'

825721 access("./test.tex", R_OK) = 0
825721 newfstatat(AT_FDCWD</home/sanjoy/calculus-textbook/Tuptextest>, "./test.tex", {st_dev=makedev(0, 0x53), st_ino=14, st_mode=S_IFREG|000, st_nlink=0, st_uid=65534, st_gid=65534, st_blksize=4096, st_blocks=0, st_size=0, st_atime=0, st_atime_nsec=0, st_mtime=0, st_mtime_nsec=0, st_ctime=0, st_ctime_nsec=0}, 0) = 0
825721 openat(AT_FDCWD</home/sanjoy/calculus-textbook/Tuptextest>, "test.tex", O_RDONLY) = 3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>
825721 fstat(3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>, {st_dev=makedev(0, 0x53), st_ino=14, st_mode=S_IFREG|000, st_nlink=0, st_uid=65534, st_gid=65534, st_blksize=4096, st_blocks=0, st_size=0, st_atime=0, st_atime_nsec=0, st_mtime=0, st_mtime_nsec=0, st_ctime=0, st_ctime_nsec=0}) = 0
825721 read(3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>, "", 4096) = 0
825721 close(3</home/sanjoy/calculus-textbook/Tuptextest/test.tex>) = 0

[Pat Pannuto]

It's been a few years since I've used tup with latex (sadly; got too hard to support other os x users/collaborators), but circa 2019 it worked fine.

I'm surprised you are seeing multiple reads to test.tex from one invocation of pdftex, at least with pdflatex as a driver, you have to explicitly call the command multiple times. Tup isn't okay with multiple build rules modifying the same output file, so you have to have one single tup command do all of the (la)tex driver invocations, i.e., here's the relevant snippet from a paper from 2016:

: paper.tex | ../figs/<figsgroup> ../images/<imagesgroup> *.cls *.bib |> \
^ pdflatex ^ \
pdflatex -jobname=@(PAPER_NAME) %f; bibtex @(PAPER_NAME); pdflatex -jobname=@(PAPER_NAME) %f; pdflatex -jobname=@(PAPER_NAME) %f  |> \
@(PAPER_NAME).pdf | @(PAPER_NAME).aux @(PAPER_NAME).bbl @(PAPER_NAME).blg @(PAPER_NAME).log @(PAPER_NAME).out

Later on, we moved to a wrapper-driver which more intelligently knew how to run the latex driver "enough" times (but, critically, was still one command from tup's perspective:

: paper.tex | ../figs/<figsgroup> ../images/<imagesgroup> *.cls *.bib |> \
^ pdflatex ^ ../support/latexrun.py --latex-args="-shell-escape" -Wall -o @(PAPER_NAME).pdf -O ./ %f |> \
@(PAPER_NAME).pdf | paper.aux paper.bbl paper.blg paper.dep paper.log paper.out paper.fls paper.pdf latexrun.db latexrun.db.lock paper.dft

HTH,

-Pat

--
--
tup-users mailing list
email: tup-...@googlegroups.com
unsubscribe: tup-users+...@googlegroups.com
options: http://groups.google.com/group/tup-users?hl=en
---
You received this message because you are subscribed to the Google Groups "tup-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tup-users+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/tup-users/87v7ns339h.fsf%40mit.edu.

[Sanjoy Mahajan]

On 2025-07-16 11:25, Pat Pannuto <pat.p...@gmail.com> wrote:

> I'm surprised you are seeing multiple reads to test.tex from one invocation
> of pdftex, at least with pdflatex as a driver, you have to explicitly call
> the command multiple times.

I also tested it running pdftex under strace, and there were multiple
reads (even for the one pdftex pass). But I don't think that the
multiple reads are causing the problem with tup. For I also use context
(another tex cousin), which does the multiple passes on its own reading
the .tex file (at least) once each time, and it works fine with tup.

So it may be a subtle interaction between pdftex and fuse? (I also had
the same problem with pdflatex.)

> Tup isn't okay with multiple build rules modifying the same output
> file, so you have to have one single tup command do all of the (la)tex
> driver invocations, i.e., here's the relevant snippet from a paper
> from 2016:

Ah, that's good to know. For latex, you could solve the problem by
using latexmk. It does all the rerunning with interleaved bibtexing.
It's very useful for Makefiles (and, once this issue is solved,
Tupfiles).

I'm going to try "tup --debug-fuse" to see if I get any clues there.

Best,
-Sanjoy

[Sanjoy Mahajan]

On 2025-07-16 11:25, Pat Pannuto <pat.p...@gmail.com> wrote:

> It's been a few years since I've used tup with latex (sadly; got too hard
> to support other os x users/collaborators), but circa 2019 it worked
> fine.

Your comment gave me a debugging idea. I downgraded to tup 0.7.11-6
[debian], using the package in the Debian snapshot archive
<https://snapshot.debian.org/package/tup/0.7.11-6/#tup_0.7.11-6>

Now tup runs pdftex without any problem. So it seems to be a regression
in tup v 0.8.

$ tup
[ tup ] [0.026s] Scanning filesystem...
[ tup ] [0.049s] Reading in new environment variables...
[ tup ] [0.070s] Parsing Tupfiles...
1) [0.003s] .
0) [0.000s] auto
[ ] 100%
[ tup ] [0.088s] No files to delete.
[ tup ] [0.088s] Generating .gitignore files...
[ tup ] [0.120s] Executing Commands...
0) [0.353s] pdftex test.tex

This is pdfTeX, Version 3.141592653-2.6-1.40.26 (TeX Live 2025/dev/Debian) (preloaded format=pdftex)
restricted \write18 enabled.
entering extended mode
(./test.tex HELLO THERE [1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
)</usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></usr/
share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmsy10.pfb>
Output written on test.pdf (1 page, 16123 bytes).
Transcript written on test.log.

[ ] 100%
[ tup ] [0.515s] Updated.

[Sanjoy Mahajan]

Further testing makes me think that it's a problem only with recent
versions of fuse3 (on Debian/amd64, at least). I tested all the Debian
package versions from 0.7.11 to now:

tup_0.7.11-1_amd64.deb
tup_0.7.11-2_amd64.deb
tup_0.7.11-3_amd64.deb
tup_0.7.11-4_amd64.deb
tup_0.7.11-5_amd64.deb
tup_0.7.11-6_amd64.deb
tup_0.8-1_amd64.deb
tup_0.8-1+b1_amd64.deb [FAILED!]

Only the last package failed. The only difference between it and the
previous one (0.8-1) is that 0.8-1+b1 was compiled against fuse3
v3.17.2, whereas 0.8-1 was compiled against fuse3 v3.14.0.

I've filed a Debian bug report:

<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1109719>

In short, I'm now not sure whether the problem is in tup or in fuse3 (or
in a recent change in the fuse3 interface that tup hasn't accounted
for).

-Sanjoy

[Marcus]

I'm likewise having issues on the latest version of tup.

It's a strange problem. I have a simple bash script to cat the results of a file. If I run the script manually, it prints the expected data. When running through tup, `cat` echos nothing. It's somehow empty.