Bug#1019000: pandoc: LaTeX Error: Environment Shaded undefined.

[Johannes Schauer Marin Rodrigues]

Package: pandoc
Version: 2.17.1.1-1
Severity: normal
X-Debbugs-Cc: jo...@debian.org

Hi,

since pandoc 2.17.1.1-1, the latex generated from rst by [this script]
fails to build with:

[14] [15] [16]
Chapter 5.
(/usr/share/texlive/texmf-dist/tex/latex/microtype/mt-cmr.cfg)
Overfull \hbox (15.37852pt too wide) in paragraph at lines 251--258
\TU/Inter(0)/m/n/10 for your computer to execute, but also to write programs (s
cripts)
[17]

! LaTeX Error: Environment Shaded undefined.

See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...

l.278 \begin{Shaded}

?
! Emergency stop.
...

l.278 \begin{Shaded}

Output written on reform-handbook.pdf (17 pages).
Transcript written on reform-handbook.log.

Responsible for this is pandoc as evident from a debbisect run with this
reproducer script test.sh:

#!/bin/sh
set -e
chroot "$1" useradd --home-dir /home/user --create-home user
chroot "$1" runuser -u user -- env --chdir=/home/user sh -e << EOF
pip3 install sphinx
wget https://github.com/rsms/inter/releases/download/v3.15/Inter-3.15.zip
mkdir -p /home/user/.local/share/fonts
unzip Inter-3.15.zip
mv 'Inter Desktop' /home/user/.local/share/fonts/
fc-cache /home/user/.local/share/fonts
git clone https://source.mnt.re/reform/reform-handbook.git
mkdir -p reform-handbook/src/build
env --chdir=reform-handbook/src ./build-pdf-book.sh
EOF

And this command line:

$ debbisect --cache=./cache --depends=python3-pip,texlive-xetex,texlive-latex-extra,pandoc,wget,unzip,fontconfig,texlive-font-utils,git,ca-certificates,texlive-fonts-recommended,ghostscript 20220813T160100Z 20220813T214310Z ./test.sh
snapshot timestamp difference: 0.237616 days
approximately 2 steps left to test
#1: using cached results from debbisect.20220813T160100Z.log.good
computation time left: 0:00:00.000067
approximately 1 steps left to test
#2: using cached results from debbisect.20220813T214310Z.log.bad
snapshot timestamp difference: 0.237616 days
computation time left: 0:00:00
approximately 0 steps left to test
#3: using cached result (was good) from debbisect.20220813T160100Z.log.good
bisection finished successfully
last good timestamp: 2022-08-13 16:01:00+00:00
first bad timestamp: 2022-08-13 21:43:10+02:00
the following packages differ between the last good and first bad timestamp:
liblua5.3-0:amd64 (n.a.) -> 5.3.6-1
libyaml-0-2:amd64 (n.a.) -> 0.2.5-1
pandoc 2.9.2.1-3+b2 -> 2.17.1.1-1
pandoc-data 2.9.2.1-3 -> 2.17.1.1-1
test installing liblua5.3-0:amd64 5.3.6-1...
installing liblua5.3-0:amd64 does not cause the problem
test installing libyaml-0-2:amd64 0.2.5-1...
installing libyaml-0-2:amd64 does not cause the problem
test upgrading pandoc 2.9.2.1-3+b2 -> 2.17.1.1-1...
upgrading pandoc triggered the problem
additional packages that got upgraded/installed at the same time:
liblua5.3-0:amd64 (n.a.) -> 5.3.6-1
libyaml-0-2:amd64 (n.a.) -> 0.2.5-1
pandoc-data 2.9.2.1-3 -> 2.17.1.1-1
test upgrading pandoc-data 2.9.2.1-3 -> 2.17.1.1-1...
upgrading pandoc-data triggered the problem
additional packages that got upgraded/installed at the same time:
liblua5.3-0:amd64 (n.a.) -> 5.3.6-1
libyaml-0-2:amd64 (n.a.) -> 0.2.5-1
pandoc 2.9.2.1-3+b2 -> 2.17.1.1-1

Thanks!

cheers, josch

[this script] https://source.mnt.re/reform/reform-handbook/-/blob/master/src/build-pdf-book.sh

[Jonas Smedegaard]

Hi Josch,

Quoting Johannes Schauer Marin Rodrigues (2022-09-03 06:47:48)

> since pandoc 2.17.1.1-1, the latex generated from rst by [this script]
> fails to build with:
>
> [14] [15] [16]
> Chapter 5.
> (/usr/share/texlive/texmf-dist/tex/latex/microtype/mt-cmr.cfg)
> Overfull \hbox (15.37852pt too wide) in paragraph at lines 251--258
> \TU/Inter(0)/m/n/10 for your computer to execute, but also to write programs (s
> cripts)
> [17]
>
> ! LaTeX Error: Environment Shaded undefined.

Please tell which texlive-* packages you have installed.

Pandoc can translate between many different formats, and some of them
require additional packages that are only suggested (to not pull in
gigantic amounts not needed by all Pandoc users).

Quick online search seems to indicate that "Environment Shaded" is part
of LaTeX file "framed.dty", and "apt-file search /framed.sty" indicates
that that file is provided by Debian package texlive-latex-extra - so
please check that you have that package installed, and if not try
install it and report back here if that helped.


Kind regards,

- Jonas

--
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/

[x] quote me freely [ ] ask before reusing [ ] keep private

[Johannes Schauer Marin Rodrigues]

Hi Jonas,

Quoting Jonas Smedegaard (2022-09-03 10:59:04)

> Quoting Johannes Schauer Marin Rodrigues (2022-09-03 06:47:48)
> > since pandoc 2.17.1.1-1, the latex generated from rst by [this script]
> > fails to build with:
> >
> > [14] [15] [16]
> > Chapter 5.
> > (/usr/share/texlive/texmf-dist/tex/latex/microtype/mt-cmr.cfg)
> > Overfull \hbox (15.37852pt too wide) in paragraph at lines 251--258
> > \TU/Inter(0)/m/n/10 for your computer to execute, but also to write programs (s
> > cripts)
> > [17]
> >
> > ! LaTeX Error: Environment Shaded undefined.
>
> Please tell which texlive-* packages you have installed.
>
> Pandoc can translate between many different formats, and some of them
> require additional packages that are only suggested (to not pull in
> gigantic amounts not needed by all Pandoc users).
>
> Quick online search seems to indicate that "Environment Shaded" is part
> of LaTeX file "framed.dty", and "apt-file search /framed.sty" indicates
> that that file is provided by Debian package texlive-latex-extra - so
> please check that you have that package installed, and if not try install it
> and report back here if that helped.

thanks for looking into this!

I sent you a script that you can use to reproduce the problem in my initial
mail. If you grep in that mail for texlive-latex-extra you will see that yes,
it is installed.

cheers, josch

[Jonas Smedegaard]

Quoting Johannes Schauer Marin Rodrigues (2022-09-03 11:13:47)

True, I didn't inspect your script.

It would be helpful if you could isolate the problem, so that it is
possible to replicate with as minimal as possible code besides Pandoc.

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Jonas Smedegaard (2022-09-03 11:41:22)

> It would be helpful if you could isolate the problem, so that it is possible
> to replicate with as minimal as possible code besides Pandoc.

so if you run this:

echo ".. code-block:: none" | pandoc -o out.tex -frst

then with the old pandoc 2.9.2.1 you get:

\begin{verbatim}
\end{verbatim}

and with pandoc 2.17.1.1 you get:

\begin{Shaded}
\begin{Highlighting}[]
\end{Highlighting}
\end{Shaded}

so for this to work with pandoc 2.17.1.1 we need some usepackage for the Shaded
environment.

So lets see how to fix this from the latex side. You pointed out that the
Shaded environment comes from framed.sty, so we try:

\documentclass{article}
\usepackage{framed}
\begin{document}
\begin{Shaded}
\begin{Highlighting}[]
foobar
\end{Highlighting}
\end{Shaded}
\end{document}

But we still get:

! LaTeX Error: Environment Shaded undefined.

So how do I prepare my latex that was generated by recent pandoc so that it
works?

Thanks!

cheers, josch

[Jonas Smedegaard]

Quoting Johannes Schauer Marin Rodrigues (2022-09-03 15:35:36)

> Quoting Jonas Smedegaard (2022-09-03 11:41:22)
> > It would be helpful if you could isolate the problem, so that it is
> > possible to replicate with as minimal as possible code besides
> > Pandoc.
>
> so if you run this:
>
> echo ".. code-block:: none" | pandoc -o out.tex -frst
>
> then with the old pandoc 2.9.2.1 you get:
>
> \begin{verbatim}
> \end{verbatim}
>
> and with pandoc 2.17.1.1 you get:
>
> \begin{Shaded}
> \begin{Highlighting}[]
> \end{Highlighting}
> \end{Shaded}

Thanks a lot - this makes it much easier to examine the issue further!

> so for this to work with pandoc 2.17.1.1 we need some usepackage for
> the Shaded environment.
>
> So lets see how to fix this from the latex side. You pointed out that
> the Shaded environment comes from framed.sty, so we try:
>
> \documentclass{article}
> \usepackage{framed}
> \begin{document}
> \begin{Shaded}
> \begin{Highlighting}[]
> foobar
> \end{Highlighting}
> \end{Shaded}
> \end{document}
>
> But we still get:
>
> ! LaTeX Error: Environment Shaded undefined.
>
> So how do I prepare my latex that was generated by recent pandoc so
> that it works?

One extra clue I just found is that apparently the tex chunk involving
"Shading" is not part of Pandoc templates but injected by Haskell
library skylighting:
https://github.com/Wandmalfarbe/pandoc-latex-template/issues/209#issuecomment-744068500

[Jonas Smedegaard]

Control: tags -1 +unreproducible

Quoting Jonas Smedegaard (2022-09-03 16:21:21)

> Quoting Johannes Schauer Marin Rodrigues (2022-09-03 15:35:36)

> > so if you run this:
> >
> > echo ".. code-block:: none" | pandoc -o out.tex -frst
> >
> > then with the old pandoc 2.9.2.1 you get:
> >
> > \begin{verbatim}
> > \end{verbatim}
> >
> > and with pandoc 2.17.1.1 you get:
> >
> > \begin{Shaded}
> > \begin{Highlighting}[]
> > \end{Highlighting}
> > \end{Shaded}
>
> Thanks a lot - this makes it much easier to examine the issue further!

...but while the above minimal command demonstrates that "Shaded" gets
introduced in intermediary LaTeX code, this slightly different command
to generate a PDF from that same intermediary LaTeX code does not fail:

echo ".. code-block:: none" | pandoc -o out.pdf -frst

I tested in a somewhat clean sid environment with this prepation:

apt --no-install-recommends install pandoc texlive-latex-recommended lmodern

> > so for this to work with pandoc 2.17.1.1 we need some usepackage for
> > the Shaded environment.

I guess (but again I have *not* looked at your original external code,
so really I am only guessing here!) that the real issue might be that a
custom template is used and that custom template needs to be updated to
match newer pandoc processing.

To demonstrate that this is a bug in pandoc I will need to be able to
trigger it with a minimal failing code example.

If you agree that the bug may be in some custom template needing update,
then you can probably find help in comparing the output from older and
newer pandoc of this command:

pandoc --print-default-template latex

If you want to explore stepwise what is happening between your

echo ".. code-block:: none" | pandoc -o out.tex -frst

and my

echo ".. code-block:: none" | pandoc -o out.pdf -frst

then you are probably helped by addding option "--standalone":

echo ".. code-block:: none" | pandoc -o out.tex -frst --standalone


Kind regards,

[Jonas Smedegaard]

Quoting Johannes Schauer Marin Rodrigues (2022-09-03 17:37:29)
> Quoting Jonas Smedegaard (2022-09-03 16:54:53)

> > If you want to explore stepwise what is happening between your
> >
> > echo ".. code-block:: none" | pandoc -o out.tex -frst
> >
> > and my
> >
> > echo ".. code-block:: none" | pandoc -o out.pdf -frst
> >
> > then you are probably helped by addding option "--standalone":
> >
> > echo ".. code-block:: none" | pandoc -o out.tex -frst --standalone
>

> Bingo! Using the --standalone option to pandoc indeed made it generate the
> definition for the Highlighting and Verbatim environments.
>
> Is there a way to tell pandoc to just give me the preamble
> that --standalone produces so that nothing breaks the next
> time pandoc is updated?

There is no command-line option for pandoc to get more finegrained
templating components. If you want to programmatically query in more
detail, then you need to interact with the underlying Haskell library
"skylighting" as mentioned in my previous post (which it seems I
accidentally didn't cc you - sorry about that!).


> Anyways, not pandoc's fault, so closing this bug.

Happy that it got solved. Have a nice weekend!