Problem reading with bz2.BZ2File(). Bug?

[Clodoaldo Pinto Neto]

When comparing two files which should be equal the last line is
different:

The first file is a bzip2 compressed file and is read with
bz2.BZ2File()
The second file is the same file uncompressed and read with open()

The first file named file.txt.bz2 is uncompressed with:

$ bunzip2 -k file.txt.bz2

To compare I use this script:
###############################
import bz2

f1 = bz2.BZ2File(r'file.txt.bz2', 'r')
f2 = open(r'file.txt', 'r')
lines = 0
while True:
line1 = f1.readline()
line2 = f2.readline()
if line1 == '':
break
lines += 1
if line1 != line2:
print 'line number:', lines
print repr(line1)
print repr(line2)
f1.close()
f2.close()
##############################

The offending file is 5.5 MB. Sorry, i could not reproduce this problem
with a smaller file.
http://fahstats.com/img/file.txt.bz2

Regards, Clodoaldo Pinto Neto

[Fredrik Lundh]

Clodoaldo Pinto Neto wrote:

> The offending file is 5.5 MB. Sorry, i could not reproduce this problem
> with a smaller file.

but surely you can post the repr() of the last two lines?

</F>

[Clodoaldo Pinto Neto]

This is the output:

$ python bzp.py
line number: 588317
'\x07'
''

Clodoaldo

[Leo Kislov]

Confirmed on windows with 2.4 and 2.5:

C:\p>\Python24\python.exe bzp.py
line number: 588317
'\x1e'
''

C:\p>\Python25\python.exe bzp.py
line number: 588317
'\x1e'
''

Looks like one byte of garbage is appended at the end of file. Please
file a bug report. As a workaround "rU" mode seems to work fine for
this file.

-- Leo

[Clodoaldo Pinto Neto]

Leo Kislov wrote:
> Confirmed on windows with 2.4 and 2.5:
>
> C:\p>\Python24\python.exe bzp.py
> line number: 588317
> '\x1e'
> ''
>
> C:\p>\Python25\python.exe bzp.py
> line number: 588317
> '\x1e'
> ''
>
> Looks like one byte of garbage is appended at the end of file. Please
> file a bug report.

Bug number 1597011

Clodoaldo