Bug#413045: texlive-latex-recommended: UTF8 listings error
Benoît Dejean
Version: 2005.dfsg.3-1
Severity: minor
Hello,
i have switched to texlive and i cannot compile an old report anymore
\documentclass[a4paper, 12pt]{report}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[francais]{babel}
\usepackage{listings}
\begin{document}
\begin{lstlisting}[language=sh, basicstyle=\ttfamily\footnotesize]
ip prefix-list FILTRE_ANNONCES_ENTRANTES seq 5 deny <réseau> le 32
\end{lstlisting}
\begin{lstlisting}[language=cpp, basicstyle=\ttfamily\footnotesize]
void foo() {
// réseau
}
\end{lstlisting}
\end{document}
latex Rapport.tex
This is pdfeTeX, Version 3.141592-1.30.5-2.2 (Web2C 7.5.5)
entering extended mode
(./Rapport.tex
LaTeX2e <2003/12/01>
Babel <v3.8g> and hyphenation patterns for english, usenglishmax, dumylang, noh
yphenation, loaded.
(/usr/share/texmf-texlive/tex/latex/base/report.cls
Document Class: report 2004/02/16 v1.4f Standard LaTeX document class
(/usr/share/texmf-texlive/tex/latex/base/size12.clo))
(/usr/share/texmf-texlive/tex/latex/base/inputenc.sty
(/usr/share/texmf-texlive/tex/latex/base/utf8.def
(/usr/share/texmf-texlive/tex/latex/base/t1enc.dfu)
(/usr/share/texmf-texlive/tex/latex/base/ot1enc.dfu)
(/usr/share/texmf-texlive/tex/latex/base/omsenc.dfu)))
(/usr/share/texmf-texlive/tex/latex/base/fontenc.sty
(/usr/share/texmf-texlive/tex/latex/base/t1enc.def))
(/usr/share/texmf-texlive/tex/generic/babel/babel.sty
(/usr/share/texmf-texlive/tex/generic/babel/frenchb.ldf
(/usr/share/texmf-texlive/tex/generic/babel/babel.def)
Package babel Warning: No hyphenation patterns were loaded for
(babel) the language `French'
(babel) I will use the patterns loaded for \language=0 instead.
*************************************
* Local config file frenchb.cfg used
*
(/usr/share/texmf-texlive/tex/generic/babel/frenchb.cfg)))
(/usr/share/texmf-texlive/tex/latex/listings/listings.sty
(/usr/share/texmf-texlive/tex/latex/graphics/keyval.sty)
(/usr/share/texmf-texlive/tex/latex/listings/lstpatch.sty)
(/usr/share/texmf-texlive/tex/latex/listings/lstmisc.sty)
(/usr/share/texmf-texlive/tex/latex/listings/listings.cfg)) (./Rapport.aux)
(/usr/share/texmf/tex/latex/lm/t1lmr.fd)
(/usr/share/texmf-texlive/tex/latex/listings/lstlang1.sty)
(/usr/share/texmf-texlive/tex/latex/base/t1cmtt.fd)
! Package inputenc Error: Unicode char \u8:�\expandafter not set up for use wit
h LaTeX.
See the inputenc package documentation for explanation.
Type H <return> for immediate help.
...
l.13 ...E_ANNONCES_ENTRANTES seq 5 deny <réseau>
le 32
?
It looks like listings doesn't like UTF-8 chars inside brackets.
Is my source wrong ?
Thanks.
######################################
List of ls-R files
-rw-r--r-- 1 root root 983 2007-03-01 20:35 /var/lib/texmf/ls-R
lrwxrwxrwx 1 root root 29 2007-02-27 23:42 /usr/share/texmf/ls-R -> /var/lib/texmf/ls-R-TEXMFMAIN
lrwxrwxrwx 1 root root 27 2007-03-01 20:31 /usr/share/texmf-texlive/ls-R -> /var/lib/texmf/ls-R-TEXLIVE
lrwxrwxrwx 1 root root 27 2007-03-01 20:31 /usr/share/texmf-texlive/ls-R -> /var/lib/texmf/ls-R-TEXLIVE
######################################
Config files
lrwxrwxrwx 1 root root 20 2007-02-27 23:42 /usr/share/texmf/web2c/texmf.cnf -> /etc/texmf/texmf.cnf
-rw-r--r-- 1 root root 5931 2007-03-01 20:35 /var/lib/texmf/web2c/fmtutil.cnf
-rw-r--r-- 1 root root 7710 2007-03-01 20:35 /var/lib/texmf/web2c/updmap.cfg
-rw-r--r-- 1 root root 4481 2007-03-01 20:35 /var/lib/texmf/tex/generic/config/language.dat
-- System Information:
Debian Release: sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: powerpc (ppc)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.20.1-ben
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to fr_FR.UTF-8)
Versions of packages texlive-latex-recommended depends on:
ii texlive-common 2005.dfsg.3-1 TeX Live: Base component
ii texlive-latex-base 2005.dfsg.3-1 TeX Live: Basic LaTeX packages
Versions of packages texlive-latex-recommended recommends:
ii latex-beamer 3.06.dfsg.1-0.1 LaTeX class to produce presentatio
ii latex-xcolor 2.09-1 Easy driver-independent TeX class
ii prosper 1.00.4+cvs.2006.10.22-1.1 LaTeX class for writing transparen
Versions of packages tex-common depends on:
ii debconf [debconf-2.0] 1.5.11 Debian configuration management sy
ii ucf 2.0018.1 Update Configuration File: preserv
Versions of packages texlive-latex-recommended is related to:
pn tetex-base <none> (no description available)
pn tetex-bin <none> (no description available)
pn tetex-extra <none> (no description available)
-- debconf information:
tex-common/singleuser: false
tex-common/check_texmf_wrong:
tex-common/check_texmf_missing:
Frank Küster
> Package: texlive-latex-recommended
> Version: 2005.dfsg.3-1
> Severity: minor
>
> Hello,
>
> i have switched to texlive and i cannot compile an old report anymore
[...]
> It looks like listings doesn't like UTF-8 chars inside brackets.
> Is my source wrong ?
Hm, this is strange, since the versions of listings in texlive 2005 and
teTeX 3.0 (the one in etch and sid) are the same. However, I can
reproduce the problem here. If I change utf8 to latin1 and save the
file in iso-8859-1 encoding, then it works.
> \begin{lstlisting}[language=cpp, basicstyle=\ttfamily\footnotesize]
> void foo() {
> // réseau
> }
> \end{lstlisting}
>
> \end{document}
Except that cpp doesn't exist (it's 'C++'), have you defined it locally?
> Babel <v3.8g> and hyphenation patterns for english, usenglishmax, dumylang, noh
> yphenation, loaded.
[...]
> Package babel Warning: No hyphenation patterns were loaded for
> (babel) the language `French'
> (babel) I will use the patterns loaded for \language=0 instead.
You should probably install texlive-lang-french...
Regards, Frank
--
Dr. Frank Küster
Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich
Debian Developer (teTeX/TeXLive)
Frank Küster
> Benoît Dejean <ben...@placenet.org> wrote:
>
>> Package: texlive-latex-recommended
>> Version: 2005.dfsg.3-1
>> Severity: minor
>>
>> Hello,
>>
>> i have switched to texlive and i cannot compile an old report anymore
> [...]
>> It looks like listings doesn't like UTF-8 chars inside brackets.
>> Is my source wrong ?
>
> Hm, this is strange, since the versions of listings in texlive 2005 and
> teTeX 3.0 (the one in etch and sid) are the same. However, I can
> reproduce the problem here. If I change utf8 to latin1 and save the
> file in iso-8859-1 encoding, then it works.
I found that it also works with utf8 when you add
\lstset{extendedchars=false}
I've not found the string "utf" in the documentation of listings-1.3,
but in the PDF for 1.4 on CTAN. Even that one doesn't talk about
general utf encoding, only about CJK and similar things.
I'll ask the upstream author.
Frank Küster
a user of listings in Debian noticed that there's a small glitch when
the file is saved in UTF-8 encoding. It is already mentioned in the
documentation, but only takes into account CJK-latex or the ucs
package. Actually, the inputenc package also supports utf8.
I think it would be a good idea to document the "extendedchars=false"
workaround also for that, much more common, case, or make it unnecessary
(cannot this be detected automatically?).
A test document is below (Benoît, I don't think the <..> are special, it
also fails in the second listing after s/cpp/C++/).
Regards, Frank
Benoît Dejean <ben...@placenet.org> wrote:
> \documentclass[a4paper, 12pt]{report}
>
> \usepackage[utf8]{inputenc}
> \usepackage[T1]{fontenc}
> \usepackage[francais]{babel}
> \usepackage{listings}
>
> \begin{document}
>
> \begin{lstlisting}[language=sh, basicstyle=\ttfamily\footnotesize]
> ip prefix-list FILTRE_ANNONCES_ENTRANTES seq 5 deny <réseau> le 32
> \end{lstlisting}
>
> \begin{lstlisting}[language=cpp, basicstyle=\ttfamily\footnotesize]
> void foo() {
> // réseau
> }
> \end{lstlisting}
>
> \end{document}
Benoît Dejean
> Frank Küster <fr...@debian.org> wrote:
>
> > Benoît Dejean <ben...@placenet.org> wrote:
> >
> >> Package: texlive-latex-recommended
> >> Version: 2005.dfsg.3-1
> >> Severity: minor
> >>
> >> Hello,
> >>
> >> i have switched to texlive and i cannot compile an old report anymore
> > [...]
> >> It looks like listings doesn't like UTF-8 chars inside brackets.
> >> Is my source wrong ?
> >
> > Hm, this is strange, since the versions of listings in texlive 2005 and
> > teTeX 3.0 (the one in etch and sid) are the same. However, I can
> > reproduce the problem here. If I change utf8 to latin1 and save the
> > file in iso-8859-1 encoding, then it works.
>
> I found that it also works with utf8 when you add
>
> \lstset{extendedchars=false}
>
> I've not found the string "utf" in the documentation of listings-1.3,
> but in the PDF for 1.4 on CTAN. Even that one doesn't talk about
> general utf encoding, only about CJK and similar things.
(cpp was a mistake)
<> fail too :
! Package inputenc Error: Unicode char \u8:�\expandafter not set up for
use wit
h LaTeX.
See the inputenc package documentation for explanation.
Type H <return> for immediate help.
...
l.13 ...E_ANNONCES_ENTRANTES seq 5 deny <réseau>
le 32
?
! Emergency stop.
or with ucs/utf8x
See the utf8x package documentation for explanation.
Type H <return> for immediate help.
...
l.214 ..._ANNONCES_ENTRANTES seq 5 deny <réseau>
le 32
?
> I'll ask the upstream author.
Thanks.
Using \lstset{extendedchars=false makes it compile, but the output dvi
is wrong. Instead of "// réseau" it renders "// érseau".
--
Benoît Dejean
GNOME http://www.gnomefr.org/
LibGTop http://directory.fsf.org/libgtop.html
Frank Küster
> Dear Brooks,
>
> a user of listings in Debian noticed that there's a small glitch when
> the file is saved in UTF-8 encoding.
Hm, Benoît has pointed out that it's more than "a small glitch"
> I think it would be a good idea to document the "extendedchars=false"
> workaround also for that, much more common, case, or make it unnecessary
> (cannot this be detected automatically?).
The problem is that the dvi file produced with extendedchares=false is
wrong. Instead of 'réseau', it prints 'érseau':
\documentclass[a4paper, 12pt]{report}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[francais]{babel}
\usepackage{listings}
\lstset{extendedchars=false}
\begin{document}
\begin{lstlisting}[language=sh, basicstyle=\ttfamily\footnotesize]
ip prefix-list FILTRE_ANNONCES_ENTRANTES seq 5 deny <réseau> le 32
\end{lstlisting}
\begin{lstlisting}[language=C++, basicstyle=\ttfamily\footnotesize]
void foo() {
// réseau
}
\end{lstlisting}
\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% coding: utf-8-unix
%%% End:
The resulting dvi file is at
http://people.debian.org/~frank/413045-listings-encoding.dvi
Any idea how to fix this?
TIA, Frank
Brooks Moses
>Hm, Benoît has pointed out that it's more than "a small glitch"
>
> > I think it would be a good idea to document the "extendedchars=false"
> > workaround also for that, much more common, case, or make it unnecessary
> > (cannot this be detected automatically?).
>
>The problem is that the dvi file produced with extendedchares=false is
>wrong. Instead of 'réseau', it prints 'érseau':
>Any idea how to fix this?
Unfortunately, at this point the only suggestion I have is that extended
characters from packages that support multibyte characters need to go
within TeX escapes within the listing. I believe that works. (To do that
within the C++ example you included, just include "texcl" to the lstset
options list. It's a bit more complicated in the sh example, and there may
not be a completely satisfactory solution there.)
I had thought that extendedchars=false was sufficient, because the problem
was originally reported with Chinese characters, where the misordering (and
incorrect font, in some cases) is not visible, since they're not mixed with
non-extended characters. It seems that I was wrong.
Unfortunately, making listings compatible with multibyte characters is
going to be a long-term project. It is, however, the top long-term project
on my priority queue, which means that there is some possibility that it
will happen by the 1.5 release.
Sorry I don't have better news.
- Brooks
Benoît Dejean
> Brooks Moses <bmo...@stanford.edu> wrote:
> \inputencoding{latin1}
> \lstinputlisting{latin1file.java}
> \inputencoding{utf8}
> \end{document}
Hello,
This workaround has worked fine so far whenever i've been able to
convert input text to latin1. But now i'm stuck with non-translatable
chars. Regression is still there.