Bug#703234: libzbar0 recodes 8-bit QR-encoded data into UTF-8

[Ivan Shmakov]

Package: libzbar0
Version: 0.10+doc-7+b2

As it was pointed out [1] by Ben Morrow in a recent
news:comp.lang.perl.misc thread, libzbar0 appears to assume
ISO-8859-1 encoding for the input QR-encoded 8-bit data, and
attempts to unconditionally recode it into UTF-8, thus giving
(possibly) incorrect results.

The issue arises with both the zbarimg(1) command-line tool
(which disregards the locale currently in effect), and the
Perl's Barcode::ZBar interface (suggesting that the resulting
UTF-8-coded string is not being properly passed back to the
calling Perl code.)

As was suggested, I've used the “level L” Imager::QRCode option
and a test vector of 17 ‘\xFF’ octets. As expected, the
resulting QR code was 21 by 21 dots, thus ruling out the
possibility of ISO-8859-1 to UTF-8 expansion happening within
the generating Imager::QRCode module.

Consider, e. g. (the code is MIME'd):

$ LC_ALL=C perl -- z97wbjf3xptrg36ymsrczz5ggy.perl
Blob: pack("H*","ffffffffffffffffffffffffffffffffff")
Image: 42 by 42
[["QR-Code", ("\xC3\xBF" x 17)]]
$

Where \xC3\xBF's are ISO-8859-1 \xFF's recoded into UTF-8:

$ printf \\xc3\\xbf \
| iconv -f UTF-8 -t ISO-8859-1 \
| od -t x1
0000000 ff
0000001
$

[1] news:u0m61a-...@anubis.morrow.me.uk

--
FSF associate member #7257

[Mauro Carvalho Chehab]

ZBar upstream considers that the output should be done in utf-8. Yet, it
has a limited support for charset conversions, supporting "UTF-8", "ISO8859-1",
"SJIS" and "BIG-5".

Its decoding logic tries to detect the original charset and do the conversion.
Sometimes, it may guess things wrong, though.

Due to that, ZBar also supports a raw mode, where the charset conversion
is disabled:


$ gpg --export 5EA01078| ascii85| head -10 > orig && cat orig | qrencode -8 -o key.png && ./zbarimg/zbarimg --oneshot --raw key.png >parsed && diff -s orig parsed
scanned 1 barcode symbols from 1 images in 0.05 seconds

Files orig and parsed are identical

So, if the decoding is not converting things right, you can simply use
-raw mode and call "iconv" utility by hand, in order to manually do the
charset conversion manually.

Thanks,
Mauro

[Matheus Afonso Martins Moreira]

In addition to number and text encodings, QR codes have an 8 bit mode
and an Extended Channel Interpretation mode.
This ECI mode is like 8 bit mode but it also contains metadata
which tells the decoder how to convert the binary data to text.
For example, ECI 26 mode is decoded as UTF-8.

This problem happens because older versions of ZBar treat 8 bit mode
QR codes as unknown ECI mode QR codes. It tries to autodetect
the character set and unconditionally converts the binary data to it.
This destroys the data.

The new version of ZBar will have a ZBAR_CFG_BINARY decoder option
which will make the QR decoder return the data unmodified.
Users of the zbarimg and zbarcam tools will be able to pass
this option to the QR decoder through the -Sqr.binary option,
or simply -Sbinary.

Another source of corruption is the fact these tools output
metadata such as format descriptors and line separators.
These features were no doubt meant to make it easier to parse
the output of the tools but they can corrupt binary data.
This extra output can be suppressed with the --raw --oneshot options:
they will cause the ZBar tools to decode exactly one barcode
and output nothing but the data contained within it.
They should prove useful in scripts.

Finally, with these new features it should be possible to QR encode
secret keys in binary form and use ZBar to restore them:

$ gpg --export-secret-key $fingerprint \
| paperkey --output-type raw \
| qrencode --8bit --output secret-key.qr.png

$ zbarimg --raw --oneshot -Sbinary secret-key.qr.png \
| paperkey --pubring public-key.asc \
| gpg --import

I wrote an answer on StackOverflow about this problem:

https://stackoverflow.com/a/60518608

It has lots of references to source code and other barcode readers
in case anyone is interested.