PDF doc generation broken with doxygen 1.8.16 and later?

[Albrecht Schlosser]

All,

I just noticed that I can't generate our PDF docs with my upgraded Linux
system:

- Linux Mint 20 (old: 19.2/19.3)

- based on Ubuntu 20.04 (old: 18.04)

- doxygen version 1.8.17 (old: 1.8.13)

The html docs generation works well, the PDF generation is terminated
with a more or less useless error message:

cp: cannot stat 'latex/refman.pdf': No such file or directory
ninja: build stopped: subcommand failed.

Note that I'm using ninja, but

I built other versions of doxygen from Git but couldn't find a newer
version that worked:

1.8.13: OK
1.8.14: OK
1.8.15: OK
1.8.16: broken
1.8.17: broken
1.8.18: broken
1.8.19: untested
1.8.20: broken

Note: I found that I can use the built doxygen version easily from its
build directory without installing it.


Can anybody confirm this (or parts of it)?

What doxygen versions are you using?

Maybe it works for you and my `latex` installation is "broken" (but why
does it work with older doxygen versions?).

I'd appreciate all feedback before I try to debug this further and also
any help you can give in debugging this. I know from previous experience
that it's hard to do because "I don't speak latex" ;-) Maybe somebody
with specific (latex) experience could debug the PDF generation...


[ additional test notes following ]

Build and test instructions, just in case you can build and are willing
to test different doxygen versions:

Get doxygen (Git):

git clone https://github.com/doxygen/doxygen.git
cd doxygen

Check out release 1.8.XX:

git tag # list all (release) tags

git checkout Release_1_8_XX

Build this release:

mkdir -p build/1.8.XX
cd build/1.8.XX
cmake ../..
make [-j4]

With this done it's only a little more work to use this doxygen version
to generate our FLTK docs (html+pdf):

cd fltk-build-dir
cmake -D
DOXYGEN_EXECUTABLE:FILEPATH=/path-to/doxygen/build/1.8.XX/bin/doxygen
<fltk-source> [other CMake options]

make pdf

If you're using autotools/make you need to edit this line in
`makeinclude` to point at the correct doxygen executable:

DOXYDOC = /usr/bin/doxygen

[Albrecht Schlosser]

On 9/16/20 3:02 PM Albrecht Schlosser wrote:
>
> The html docs generation works well, the PDF generation is terminated
> with a more or less useless error message:
>
> cp: cannot stat 'latex/refman.pdf': No such file or directory
> ninja: build stopped: subcommand failed.
>
> Note that I'm using ninja, but

[continuing]

... the same happens with autotools/make:

$ cd documentation
$ make pdf
Generating Doxybook ...
Generating PDF documentation ...
warning: Tag 'PERL_PATH' at line 1315 of file 'Doxybook' has become
obsolete.
To avoid this warning please remove this line from your
configuration file or upgrade it using "doxygen -u"
warning: Tag 'MSCGEN_PATH' at line 1337 of file 'Doxybook' has become
obsolete.
To avoid this warning please remove this line from your
configuration file or upgrade it using "doxygen -u"

cp: cannot stat 'latex/refman.pdf': No such file or directory

make: *** [Makefile:174: pdf] Error 1

Note that the warnings are benign. I removed them by upgrading
Doxyfile.in (see warning above) but this didn't change anything.

[duncan]

I'm afraid that I don't have a Linux box at home under my own control, and

only a centrally managed CentOS 7.7 box at work which is quite old, where:

latex -v shows it's from the TeX Live 2013 distribution (but I found it's broken)

doxygen -v shows 1.8.5

cmake --version shows 2.8.12.2

However, I tried to download newer versions into a scratch area and then

set up the various PATHs appropriately but not 100% successfully

After installingTeX Live 2020 and cmake 3.18.2 I was able to build the

current fltk-1.4.x documentation, both html and pdf with doxygen 1.8.5

So partial success at this point

I cloned the doxygen git repository and checked out 1.8.15, which should

have worked, if your experience is anything to go by, but I got a warning

and/or error message when trying to build it, which might be significant:

-- Performing Test ICONV_COMPILES - Success
CMake Warning (dev) at /.../fltk-doxygen/cmake/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
  The package name passed to `find_package_handle_standard_args` (ICONV) does
  not match the name of the calling package (Iconv).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/FindIconv.cmake:87 (find_package_handle_standard_args)
  CMakeLists.txt:95 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

But when I then tried to build fltk-1.4.x using the DOXYGEN_EXECUTABLE

parameter to cmake, the build fails with the following, but this might be

because I also need to update quite a few other support packages too :-(

-- POSIX compatible scandir - found
CMake Warning at /.../fltk-doxygen/cmake/share/cmake-3.18/Modules/FindDoxygen.cmake:447 (message):
  Unable to determine doxygen version: Permission denied
Call Stack (most recent call first):
  /.../fltk-doxygen/cmake/share/cmake-3.18/Modules/FindDoxygen.cmake:610 (_Doxygen_find_doxygen)
  CMake/resources.cmake:242 (find_package)
  CMakeLists.txt:60 (include)

-- Found Doxygen: /.../fltk-doxygen/doxygen/doxygen/build-1.8.15 (found version "") found components: doxygen dot
CMake Error at /.../fltk-doxygen/cmake/share/cmake-3.18/Modules/FindDoxygen.cmake:689 (message):
  Unable to generate Doxyfile template: Permission denied
Call Stack (most recent call first):
  CMake/resources.cmake:242 (find_package)
  CMakeLists.txt:60 (include)

-- Configuring incomplete, errors occurred!
See also "/.../fltk-doxygen/fltk/build-1.8.15/CMakeFiles/CMakeOutput.log".
See also "/.../fltk-doxygen/fltk/build-1.8.15/CMakeFiles/CMakeError.log".

I'm afraid I don't have time to track this down while I'm at work, sorry.

I think it would make more sense to use a full distro where all packages

should already be compatible and test against each other, than to try

to continue on a piecemeal basis.

D.

[Albrecht Schlosser]

NOTE: Problem solved, see below!
--------------------------------

On 9/17/20 1:38 PM duncan wrote:
> I'm afraid that I don't have a Linux box at home under my own control, and
> only a centrally managed CentOS 7.7 box at work which is quite old, where:
>
> latex -v shows it's from the TeX Live 2013 distribution (but I found
> it's broken)

Mine is:

$ latex -v
pdfTeX 3.14159265-2.6-1.40.20 (TeX Live 2019/Debian)
...

> doxygen -v shows 1.8.5
> cmake --version shows 2.8.12.2
>

> After installingTeX Live 2020 and cmake 3.18.2 I was able to build the
> current fltk-1.4.x documentation, both html and pdf with doxygen 1.8.5
> So partial success at this point

Thanks for your help, I appreciate that you took the time to do all
this. BTW, sorry for the late reply, I've been busy with other stuff today.

> I cloned the doxygen git repository and checked out 1.8.15, which should
> have worked, if your experience is anything to go by,

Yes, I could indeed build doxygen 1.8.15 using cmake 3.16.3 and it
worked (with our old header file, see below for more info).

> but I got a warning
> and/or error message when trying to build it, which might be significant:

> ...

> But when I then tried to build fltk-1.4.x using the DOXYGEN_EXECUTABLE
> parameter to cmake, the build fails with the following, but this might be
> because I also need to update quite a few other support packages too :-(

Again, thanks a lot... (more error messages elided)

> I'm afraid I don't have time to track this down while I'm at work, sorry.
>
> I think it would make more sense to use a full distro where all packages
> should already be compatible and test against each other, than to try
> to continue on a piecemeal basis.

I agree. My idea (question) was more about whether someone had a newer
doxygen version (newer than 1.8.15) and *could* build the PDF docs, or -
if not - the second question would be if an older doxygen version (like
1.8.15) worked as it did for me. This should not only be an issue on
Linux but also on macOS (or Windows if anybody installed doxygen and
latex on Windows).

Anyway, thanks for all your work. I *just* found the cause (but not yet
a complete solution):

I had a faint recollection that a previous issue with similar negative
results was caused by our doxygen pdf/latex header file
'fltk-book.tex.in' which can be (re)created partially by doxygen. This
header file contains (obviously necessary) LaTeX code which *is*
{doxygen + latex} version dependent, i.e. the header file appears to
define some latex code (macros, commands?) used in the latex code
generated by doxygen to build the pdf docs.

Consequently I regenerated the latex header file [1], merged it with our
existing 'fltk-book.tex.in', et voilà: it worked. I can now generate
FLTK pdf docs with my upgraded Linux box and its default doxygen version
(1.8.17).

To do: the next problem will be to find out how to integrate this with
our sources so devs with older doxygen/latex versions can still generate
the pdf docs. But that's something I can do myself later.

-----

[1] see doxygen docs (man doxygen):
$ doxygen -w latex headerFile footerFile styleSheetFile
Then merge 'headerFile' with 'fltk-book.tex.in' keeping our own title
and FLTK specific header parts. Attached file fltk-book.diff is a
working patch for now.

[duncan]

For more info...

On my MacOSX 10.15.6 Catalina, I have just installed Tex Live 2020,

and updated to cmake-3.18.2 and doxygen-1.8.20 by using

  brew cask install mactex ; brew install cmake ; brew install doxygen

I have commented out the PERL_PATH and MCSGEN_PATH lines in

the Doxygen.in file, as well as the lines:

  INPUT += @FLTK_SOURCE_DIR@/FL

  INPUT += @FLTK_SOURCE_DIR@/cairo

  INPUT += @FLTK_SOURCE_DIR@/src

and I can successfully make a reduced fltk.pdf file, so it's not that there

is something inherently wrong with doxygen > 1.8.15 as such, BUT...

If I include the three INPUT lines above, I can see from the make output

that there are warnings/errors relating to \deprecated in FL/Fl_Menu_Item.H

and then the build/documentation/latex/refman.log concludes with an error

message about "! TeX capacity exceeded, sorry [input stack size = 5000]"

but searching the web suggests this is the result of an incomplete macro

call somewhere in the input which leads to runaway recursion internally.

It's been at least 10 years since I last made an attempt at writing LaTeX,

and I remember that many error messages are obscure, and it's often the

case that I needed to remove input to home in on the real error site...

Hope this helps, but it's time for bed now, might continue at the weekend

D

[Albrecht Schlosser]

On 9/18/20 1:35 AM duncan wrote:
> For more info...

Our work and posts crossed in time, see my previous post with the cause
of the error.

> On my MacOSX 10.15.6 Catalina, I have just installed Tex Live 2020,
> and updated to cmake-3.18.2 and doxygen-1.8.20 by using
>
>   brew cask install mactex ; brew install cmake ; brew install doxygen
>
> I have commented out the PERL_PATH and MCSGEN_PATH lines in
> the Doxygen.in file, as well as the lines:
>
>   INPUT += @FLTK_SOURCE_DIR@/FL
>   INPUT += @FLTK_SOURCE_DIR@/cairo
>   INPUT += @FLTK_SOURCE_DIR@/src
>
> and I can successfully make a reduced fltk.pdf file, so it's not that there
> is something inherently wrong with doxygen > 1.8.15 as such, BUT...

Yep, tried this as well, but this was a dead end...

> If I include the three INPUT lines above, I can see from the make output
> that there are warnings/errors relating to \deprecated in FL/Fl_Menu_Item.H
>
> and then the build/documentation/latex/refman.log concludes with an error
> message about "! TeX capacity exceeded, sorry [input stack size = 5000]"

Not a very helpful error message :-(

> but searching the web suggests this is the result of an incomplete macro
> call somewhere in the input which leads to runaway recursion internally.

This seems to be true

> It's been at least 10 years since I last made an attempt at writing LaTeX,
> and I remember that many error messages are obscure, and it's often the
> case that I needed to remove input to home in on the real error site...
>
> Hope this helps, but it's time for bed now, might continue at the weekend

Don't bother, but if I hadn't found the cause already your comments
would likely have pointed me in the right direction -> fltk-book.tex[.in].

Thanks again.

[Albrecht Schlosser]

On 9/18/20 1:35 AM duncan wrote:

> For more info...
>
> On my MacOSX 10.15.6 Catalina, I have just installed Tex Live 2020,
> and updated to cmake-3.18.2 and doxygen-1.8.20 by using
>
>   brew cask install mactex ; brew install cmake ; brew install doxygen

...

> It's been at least 10 years since I last made an attempt at writing LaTeX,
> and I remember that many error messages are obscure, and it's often the
> case that I needed to remove input to home in on the real error site...
>
> Hope this helps, but it's time for bed now, might continue at the weekend

Can you please try the patch I posted ~25 minutes ago with your macOS
setup and report if it works? TIA.

[duncan]

> On my MacOSX 10.15.6 Catalina, I have just installed Tex Live 2020,
> and updated to cmake-3.18.2 and doxygen-1.8.20 by using
>   brew cask install mactex ; brew install cmake ; brew install doxygen

 

Can you please try the patch I posted ~25 minutes ago with your macOS

setup and report if it works? TIA.

For info: I didn't notice the patch attachment first time because it doesn't

show up if reading via the FLTK website forums....

I've re-opened your article via google groops, downloaded and applied the

patch, and re-run cmake, ccmake, make pdf successfully. W00t!

D.

PS. The same three errors / warnings about Fl_Menu_Item.H appear, with

"blank" entries for them in the Deprecated List chapter in the fltk.pdf.

[Albrecht Schlosser]

On 9/18/20 8:54 AM duncan wrote:
>
> I've re-opened your article via google groops, downloaded and applied the
> patch, and re-run cmake, ccmake, make pdf successfully. W00t!

Great, thanks for the feedback.

> PS. The same three errors / warnings about Fl_Menu_Item.H appear, with
> "blank" entries for them in the Deprecated List chapter in the fltk.pdf.

Working on this one, will commit a fix soon.

[Note: I missed to send this post hours ago (was distracted by other
stuff). Meanwhile Manolo committed a fix, but I'll see if I can merge my
(larger) documentation updates later. Anyway, this issue should be fixed
now.]

[Albrecht Schlosser]

On 9/18/20 5:01 PM Albrecht Schlosser wrote:
> On 9/18/20 8:54 AM duncan wrote:
>>
>> I've re-opened your article via google groops, downloaded and applied the
>> patch, and re-run cmake, ccmake, make pdf successfully. W00t!
>
> Great, thanks for the feedback.

With the following ...

commit b713b919756fd420063e7defc30bfc21996de636
Author: Albrecht Schlosser
[...]
CommitDate: Mon Sep 21 14:32:46 2020 +0200

PDF docs: generate LaTeX header file from source
[...]
This makes the PDF documentation independent of the doxygen and
latex versions of the build system.

... the PDF documentation *should* be working with all recent and
hopefully future doxygen and latex versions.

I built some old doxygen versions but could only test on my new Linux
system with latex 2.6.4 with almost full success:


1.7.6.1-20111226: works

1.8.11 up to 1.8.20: works *except* 1.8.14


I don't know why 1.8.14 doesn't work but I didn't bother to test this
since all other tested 1.8.x versions worked.

I'd appreciate if you all could test with the versions available to you
and report if pdf generation works.

Note that older versions may have some layout issues, but for this
software update it's only important if 'fltk.pdf' can successfully be
built. Thank you!

[duncan]

With the following ...

commit b713b919756fd420063e7defc30bfc21996de636
Author: Albrecht Schlosser
[...]
CommitDate: Mon Sep 21 14:32:46 2020 +0200

PDF docs: generate LaTeX header file from source
[...]
This makes the PDF documentation independent of the doxygen and
latex versions of the build system.

... the PDF documentation *should* be working with all recent and
hopefully future doxygen and latex versions.

 

For info:

on my outdated CentOS 7.7 box at work, which has:

 * cmake2.8.12, doxygen-1.8.5, TexLive 2013

the CMake build falls at the first hurdle because of the minimum_cmake_required

With a private version of cmake-3.18.2, the html docs build, but the pdf fails:

sh: epstopdf: command not found
error: Problems running epstopdf. Check your TeX installation!

I don't this this is a problem with the new FLTK cmake / doxygen config as such,

but it might be worth updating the README files to indicate minimum versions.

D.

PS. Hardly anyone uses LaTeX so it's not even worth reporting this

to the IT team until after we've been migrated to CentOS 8 :-(

PPS. I'll test the MacOSX configuration this evening...

[duncan]

With the following ...

commit b713b919756fd420063e7defc30bfc21996de636
Author: Albrecht Schlosser
[...]
CommitDate: Mon Sep 21 14:32:46 2020 +0200

PDF docs: generate LaTeX header file from source
[...]
This makes the PDF documentation independent of the doxygen and
latex versions of the build system.

... the PDF documentation *should* be working with all recent and
hopefully future doxygen and latex versions.

 

PPS. I'll test the MacOSX configuration this evening...

I can build the html and pdf documentation on MacOs 10.15.6 Catalina

with brew-installed cmake-3.18.2, doxygen-1.8.20 and TexLive 2020

D.

[Albrecht Schlosser]

On 9/22/20 11:49 AM duncan wrote:
>
> ... the PDF documentation *should* be working with all recent and
> hopefully future doxygen and latex versions.
>
> For info:
>
> on my outdated CentOS 7.7 box at work, which has:
> * cmake2.8.12, doxygen-1.8.5, TexLive 2013
> the CMake build falls at the first hurdle because of the
> minimum_cmake_required
>
> With a private version of cmake-3.18.2, the html docs build, but the pdf
> fails:
> sh: epstopdf: command not found
> error: Problems running epstopdf. Check your TeX installation!
>
> I don't this this is a problem with the new FLTK cmake / doxygen config
> as such,
> but it might be worth updating the README files to indicate minimum
> versions.

Thanks for trying anyway. The minimum CMake version is still work in
progress (there's much new stuff we *may* likely need or want to have)
but I'll make sure it's documented before we release 1.4.0. I'd like to
keep it as low as possible, but I still need to find out what we really
need.

If you have a running very old doxygen version it *might* work with it,
and in your case I assume that you would need to install some more
(La)TeX packages, but this is something I can only guess. I would expect
a list of LaTeX packages required to build PDF docs with Doxygen in the
Doxygen docs but I didn't find any requirements in the doxygen docs,
hence it's always a wild guess what to install to make it work. HTML
seems to work always, but PDF is tricky...

> PS. Hardly anyone uses LaTeX so it's not even worth reporting this
> to the IT team until after we've been migrated to CentOS 8 :-(

I agree, it's not worth it for such old systems.

> PPS. I'll test the MacOSX configuration this evening...

Thanks.

[Albrecht Schlosser]

On 9/22/20 7:44 PM duncan wrote:
>
> PPS. I'll test the MacOSX configuration this evening...
>
> I can build the html and pdf documentation on MacOs 10.15.6 Catalina
> with brew-installed cmake-3.18.2, doxygen-1.8.20 and TexLive 2020

Thanks, I had hoped so. I could also build the PDF docs with a
self-built doxygen 1.8.20 and my latex installation in Linux Mint 20
(based on Ubuntu 20.04), so these are pretty new installations that are
working. Good to know.