An odd bug when using polyglossia (german), listings, and textcase

[Kevin Godby]

I'm not sure quite where the bug is here; I'm hoping someone can help
me pinpoint it.

% --- Minimal document to compile with XeLaTeX:
\documentclass{book}
\usepackage{xltxtra}
\usepackage[german]{polyglossia}
\usepackage{listings}
\usepackage{textcase}
\begin{document}
\MakeTextLowercase{A~Z}
\end{document}
\endinput

When I compile this with XeLaTeX, it gives me an error at the
\MakeTextLowercase line:

! Undefined control sequence.
<to be read again> az
z
l.7 \MakeTextLowercase{A~Z}

If I change the language from 'german' to 'american' for polyglossia,
it compiles successfully.

Alternatively, if I comment out the listings package, it compiles
successfully.

Any ideas as to where the problem lies or how I can troubleshoot it
further?

Thanks!

--Kevin

[Ulrike Fischer]

The problem is not confined to polyglossia. I get it with babel too,
and there more languages are affected. The source of the problem
seems to be that listings sets the lccode of ~ to 122. But why this
gives errors only for some languages with polyglossia I don't know.

\documentclass{book}
%\usepackage[german]{polyglossia}
\usepackage[english]{babel}
%\usepackage{listings}
\usepackage{textcase}
\begin{document}
\lccode`\~=122
%\lccode`\~=0
\MakeTextLowercase{A~Z}

\end{document}

--
Ulrike Fischer

[Enrico Gregorio]

Ulrike Fischer <ne...@nililand.de> wrote:

Since ~ is active, \lowercase{~} is equivalent to using an
active character whose character code is the lccode of ~;
since ASCII 122 is a "z" and there's no definition for an
"active z", the error is explained.

Indeed, after adding the usual

{\catcode`z=\active \gdefz{ZETA}}

there is no error any more, and the output is "aZETAz".

Ciao
Enrico

[Ulrike Fischer]

OK. But why does it happen with polyglossia only for german and not
for english? ~ has catcode 13 in both cases.

\documentclass{book}
%\usepackage[german]{polyglossia}

\usepackage[english]{polyglossia}
\usepackage{textcase}

{\catcode`z=\active \gdefz{ZETA}}

\begin{document}

\lccode`\~=122
%\lccode`\~=0
\MakeTextLowercase{A~Z}

\end{document}

--
Ulrike Fischer

[Enrico Gregorio]

Ulrike Fischer <ne...@nililand.de> wrote:

Adding \tracingmacros=1 I get:

1) with german polyglossia (loads babelsh.def)
\reserved@a ->\endgroup \@skipmath \lowercase A~Z$\valign $

2) with english polyglossia (doesn't load babelsh.def)
\reserved@a ->\endgroup \@skipmath
\lowercase A\protect \nobreakspace {}Z$\valign $

3) with german babel (loads babel.def)
\reserved@a ->\endgroup \@skipmath \lowercase A~Z$\valign $

4) with english babel (loads babel.def)
\reserved@a ->\endgroup \@skipmath \lowercase A~Z$\valign $

In cases 1, 3 and 4, the output is indeed "aZETAz", in case
2 it is "a z". When babel.def or babelsh.def is read in, the
expansion of ~ becomes

\active@prefix ~\active@char~

which then causes the ~ to go into textcase's processing "as is",
while in the case without babel(sh).def, textcase will expand ~
before passing anything to \lowercase.

Ciao
Enrico

[Dan Luecking]

On Wed, 28 Apr 2010 10:29:10 +0200, Ulrike Fischer <ne...@nililand.de>
wrote:

>
>The problem is not confined to polyglossia. I get it with babel too,
>and there more languages are affected. The source of the problem
>seems to be that listings sets the lccode of ~ to 122. But why this
>gives errors only for some languages with polyglossia I don't know.
>
>\documentclass{book}
>%\usepackage[german]{polyglossia}
>\usepackage[english]{babel}
>%\usepackage{listings}
>\usepackage{textcase}
>\begin{document}
>\lccode`\~=122
>%\lccode`\~=0
>\MakeTextLowercase{A~Z}
>
>\end{document}

This is almost certainly a bug in listings.sty. It uses
\lowercase tricks to define certain active active
characters. For example:

\def\lst@Def#1{\lccode`\~=#1\lowercase{\def~}}

After this one could define an active z to produce a
nonactive z with added space via

\lst@Def{`\z}{\,z\,}

Without some sort of grouping, this would leave ~ with
the \lccode equal to `\z (=122).

There are 7 places (like the above) where this trick is
not guarded with \begingroup \endgroup. It might be better
if it were defined
\def\lst@Def#1{%
\begingroup\lccode`\~=#1\lowercase{\endgroup\def~}}

I went all though listings.sty, adding these groups in
those 7 places and the problem disappeared. Presumably
at least one of those grouping guards is necessary
(perhaps all of them). I did not test if that broke
anything in the listings package (I have never used it).

In addition to "~", "/" also comes out of listings.sty with
the wrong \lccode. This is because of
\lst@RestoreCatcodes
which starts with (essentially)
\lccode`\/=`\/
and is executed \AtEndofPackage. The normal \lccode of "/"
is 0, but this sets it to 47 (the character code of "/").
This make "/" look like a lowercase letter to TeX.

The \lccode table affects not only lowercase conversion.
TeX's hypnenation mechanism distinguishes a "word" from its
attached punctuation in part by examining \lccodes (non-zero
means it is taken to be part of the word). See page 454 of
the TeXbook, 2nd and 3rd paragraphs.

Also, the inclusion of metapost graphics in pdf
(accomplished at one time via supp-pdf.tex) used to rely on
the \lccode of "-" being 0. It is not out of the question
that the wrong \lccode of "/" could break some package.

Dan
To reply by email, change LookInSig to luecking