very slow pdflatex
Tim Arnold
I've upgraded to texlive2009 and now my pdflatex runs are extremely
slow. I'm not saying this has anything to do with texlive except that
I must have messed up my configuration somehow during the upgrade.
I've got an 8,000 page document that's taking a little over an hour to
do a single compile using pdflatex. At first it speeds along nicely;
stdout streaming like a firehose, but it gets slower over time...by
the time it's on page 4000, the stdout actually stops and starts,
sometimes taking seconds to move again.
The places it stops aren't at included images. The same doc used to
compile much faster before I upgraded.
Any ideas on what I should look for in the configuration?
BTW, the machine is quad-core, 8cpu, amd64 (FreeBSD 6.3)
thanks,
--Tim Arnold
Lars Madsen
I think it denpends on on the complexity of your document. (the use of
hyperref etc.)
I have a similar experience in my LaTeX book where the index takes a
very long time to compile when hyperref is enabled.
/daleif
Dan Luecking
wrote:
>Tim Arnold wrote:
>> Hi,
>> I've upgraded to texlive2009 and now my pdflatex runs are extremely
>> slow. I'm not saying this has anything to do with texlive except that
>> I must have messed up my configuration somehow during the upgrade.
>>
>> I've got an 8,000 page document that's taking a little over an hour to
>> do a single compile using pdflatex. At first it speeds along nicely;
>> stdout streaming like a firehose, but it gets slower over time...by
>> the time it's on page 4000, the stdout actually stops and starts,
>> sometimes taking seconds to move again.
>>
>> The places it stops aren't at included images. The same doc used to
>> compile much faster before I upgraded.
>>
>> Any ideas on what I should look for in the configuration?
>>
>> BTW, the machine is quad-core, 8cpu, amd64 (FreeBSD 6.3)
>>
>> thanks,
>> --Tim Arnold
>
>I think it denpends on on the complexity of your document. (the use of
>hyperref etc.)
>
This doesn't explain that the same document now
takes much longer. I expect one of the following
1. pdftex has changed and takes more memory than before.
2. a package has changed (perhaps hyperref) that now
takes more memory than before.
3. A bug in the management of memory in pdftex.
Certainly pdftex has changed in between tl2008 and 2009,
but whether that explains the slowdown I don't know. The
reason I suspect memory is that the slowdown happens
after so many pages. Usually main memory is pretty well
cleared after each page, but in the pdf format, some
data is written only at the end of the file.
Another bottleneck is disk access. Some packages write
many temporary files and re-read them. This would cause
the stated behavior only if these disk accesses began
around page 4000.
I dare say this is an extreme case that the tl2009
developers never encountered. A bug report to the
list might be in order (run "texdoc texlive" and read
section "Getting Help" to find the bug report address).
Dan
To reply by email, change LookInSig to luecking
Tim Arnold
I will make a few more test runs to get more info and then report to
the developers. I just tried setting draft mode on hyperref with the
same results as before. This is what I'm seeing from the 'top' command
during the compile:
CPU states: 12.3% user, 0.0% nice, 0.2% system, 0.1% interrupt,
87.3% idle
Mem: 210M Active, 6463M Inact, 269M Wired, 11M Cache, 214M Buf, 8606M
Free
Swap: 30G Total, 30G Free
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU
COMMAND
28771 tiarno 1 101 0 80492K 56384K CPU5 7 43:52 97.71%
pdftex
pdftex takes nearly 100% of any particular cpu (there are 8 cpus on
this machine).
From that data, it appears that memory is not a problem--the numbers
don't change very much throughout the compilation. When I check the IO
stats, it almost always says the same thing:
PID USERNAME VCSW IVCSW READ WRITE FAULT TOTAL PERCENT
COMMAND
28771 tiarno 0 24 0 0 0 0 0.00%
pdftex
with very occasional 100% usage on the WRITE.
I'll keep testing for a while and report here if I find out what my
own problem is, or to the developers if I get nowhere.
thanks,
--Tim
Michael D. Sofka
> Hi,
> I've upgraded to texlive2009 and now my pdflatex runs are extremely
> slow. I'm not saying this has anything to do with texlive except that
> I must have messed up my configuration somehow during the upgrade.
Did you remake the format files?
--
Michael D. Sofka sof...@rpi.edu
C&MT Sr. Systems Programmer, Email, HPC, TeX, Epistemology
Rensselaer Polytechnic Institute, Troy, NY. http://www.rpi.edu/~sofkam/
Jonathan Fine
> I will make a few more test runs to get more info and then report to
> the developers.
My guess is that there's a macro somewhere that has been changed in a
way that greatly increases its running time. It might be something to
do with strings, or searching in arrays.
Here's something you might try. Interupt the run perhaps 10 times while
it is slow to see what is executing. If the interrupts mostly give
similar traceback stack then perhaps there is your culprit.
Jonathan
Tim Arnold
Thanks for that suggestion Jonathan. The first 2000 pages go by in
about 4 minutes and things get progressively slower. When I interrupt
the compile at points where it seems to dawdle for a second or two, I
get two types of traceback. Well, sometimes it's writing the
intermediate chapter toc files for minitoc which can take 2-5 seconds
each time. But except for that, these are the two trackbacks that
appear:
(1)
MT@setupfont ...lias fi MT@tracking MT@check@font
ifMT@inlist@ else
MT@vinfo...
(2)
MT@in@clist ...xpandafter MT@res@a expandafter ,#2
,#1,@empty @nnil
The second one is the likeliest, but I have no idea what it means.
Just to add a bit more data, (pdf version 1.5, compress-level=3,
although it makes little difference in timing)
draft mode:
no verbatim input files, no graphics: 6.35 minutes, 14.5mb
verbatim input files, no graphics: 12.17 minutes, 16.5 mb
verbatim and graphic input files: 15.45 minutes, 16.9mb
final mode (with verbatim and graphic files):
use draft for hypersetup only: 59 minutes, 122.7mb
no draft on hypersetup: 1.12 hours 124.6mb
minute pages
1 800
2 1300
3 1750
4 2100
5 2450
6 2670
7 2790
8 3000
9 3220
10 3360
11 3385
.
13 3700
15 3920
etc.
Still tinkering and thinking.
thanks,
--Tim
Robert
> (1)
> MT@setupfont ...lias fi MT@tracking MT@check@font
> ifMT@inlist@ else
> MT@vinfo...
>
> (2)
> MT@in@clist ...xpandafter MT@res@a expandafter ,#2
> ,#1,@empty @nnil
>
> The second one is the likeliest, but I have no idea what it means.
These commands are from microtype (it also accepts the draft option, so
you can verify whether it's really the culprit). \MT@check@font calls
\MT@in@clist to check whether the font was already set up by matching it
against a comma-separated list of fonts. This list can get quite long,
as it will contain all fonts that have been used so far (where every
combination of NFSS axes/sizes counts as a different font). For example,
at the end of microtype.dtx, it contains 110 fonts, but without any
noticeable slowdown.
So either your font setup is enormously complex, with thousands of
fonts, or there's a bug in microtype so that it adds fonts repeatedly to
the list (which I cannot reproduce). Maybe you could add \tracingmacros1
to page 2000 to see what's in the font list. (Sorry for asking this, but
this would be another possibility: you did not put \tracingmacros1
anywhere in your document?) Which version of microtype are you using?
Regards,
--
Robert
Christian Justen
Could it possibly be microtype's font expansion? What happenes if you
try \usepackage[expansion=false]{microtype}.
The font expansion feature quite enormously blows the pdf documents up,
which is the reason why I normally do not use it at all.
Regards,
Christian.
Tim Arnold
Thanks Christian, Robert, I tried expansion=false with no difference
in compile time. But when I omit microtype completely, I get
dramatically faster results:
5 minutes for 8000 pages, and a pdf file size of 123,986,036.
So you guys really helped. Do you know who I should report my result
to in case this is a bug in microtype?
pdflatex 1.40.10 texlive 2009
microtype 2010/01/10 v2.4
I'll be glad to serve as a test runner if needed.
thanks,
--Tim
Tim Arnold
one more thing--pdffonts shows the final doc to have 43 fonts.
--Tim
davi...@openbet.com
by omit do you mean not include package?
I'm having a problem latex compilation speed, that if I'm logged in as the super user, pdflatex compiles slowly. But If I'm logged in as other users, if compiles really fast.
Dr Eberhard Lisse
Maybe upgrade to TeXLive 2013 first?
el
on 2013-10-17 08:39 davi...@openbet.com said the following:
[...]
jon
>
> by omit do you mean not include package?
> I'm having a problem latex compilation speed, that if I'm logged in as the super user, pdflatex compiles slowly. But If I'm logged in as other users, if compiles really fast.
>
effect when processing documents with fewer than 8000 pages..?
can a so-called minimum working example (see ctan faq) be constructed?
cheers,
jon.
Tim
> Hi Tim,
> by omit do you mean not include package?
>
> I'm having a problem latex compilation speed, that if I'm logged in as the super user, pdflatex compiles slowly. But If I'm logged in as other users, if compiles really fast.
yes, by 'omit', I meant that I didn't include the package.
The pdflatex distro may be ancient now, but the original question was posed in 2010.
In any case, upgrading pdflatex and microtype did fix the problem. Well, the 8000 page document is slower using microtype than not using it, but it compiles in a reasonable time now.
Your problem of pdflatex being slower when you're the superuser is a different problem. Often the PATH differs when you switch to superuser. A guess is that TEXINPUTS variable may be different, or you could be using a completely different TeX distribution. You might check the difference between your environment variables as yourself and as superuser.