Bug#1031701: python3-pandas: Pandas requires version '2.0.1' or newer of 'xlrd'

[Christopher Jones]

Package: python3-pandas
Version: 1.5.3+dfsg-1
Severity: normal

Dear Maintainer,

When attempting to use pandas read_excel function I get the following error message:
ImportError: Pandas requires version '2.0.1' or newer of 'xlrd' (version '1.2.0' currently installed).

xlrd 1.2 appears to be the latest currently available release packaged for Debian.

-- System Information:
Debian Release: bookworm/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

[Christopher Jones]

Hi,

Sadly they really are .xls files rather than the more slightly more sane .xlsx.

Thanks!

—
Chris

> On 20 Feb 2023, at 22:39, Rebecca N. Palmer <rebecca...@zoho.com> wrote:
>
> Is the file you're trying to open .xlsx or .xls? Do you have python3-openpyxl installed? If .xlsx and no, try installing it.
>
> (python3-)xlrd 2.0+ can only open .xls files, not .xlsx. Hence, pandas 1.5+ read_excel() always uses (python3-)openpyxl for .xlsx files. python3-pandas already Recommends python3-openpyxl. (This intentionally isn't a hard Depends because pandas can be used on other data formats without it, but the error message probably should point to openpyxl not xlrd.)

[Rebecca N. Palmer]

[Rebecca N. Palmer]

Control: reassign -1 python3-xlrd
Control: affects -1 python3-pandas
Control: retitle -1 pandas can't open xls (not xlsx) files, xlrd too old

Then yes, you do need xlrd. As Debian is currently in freeze, I suggest
installing it from PyPI (warning, this doesn't automatically install
security updates):

sudo apt-get install python3-pip
pip3 install --user xlrd

(I'm not actually sure whether Debian _should_ update to xlrd 2, as
there are some (possibly not-technically-valid) .xlsx files that xlrd 1
can open but openpyxl can't, and xlrd 2 is .xls-only.)

[Diane Trout]

Sorry my coworker got hit by this too, he worked around it by using
libreoffice to convert the .xls file to .xlsx.

I'd updated the xlrd package to 2.0.1 and pushed it to experimental to
see how much it might break,

Looks like there was some more discussion while I was fiddling with the
package.

On the plus side the update also enabled autopkgtests for xlrd.

Though I had to switch from pulling from pypi to github to keep being
able to build the -docs package, as upstream removed it from their pypi
release.

Diane

[Diane Trout]

I'm trying to talk the release team into letting me update, but I do
think the update would likely break a different, so it's likely the
answer will be no. I will try to get the updated xlrd into backports
when it opens though.

If you want to manually use debian packages instead of pip the updated
package from experimental can be found here.

https://packages.debian.org/source/experimental/python-xlrd

[Rebecca N. Palmer]

I don't consider the lack of .xls in pandas worth a freeze exception,
but consider it reasonable for others to disagree with that.

As noted in the bug, there are some (possibly not-technically-valid)
.xlsx files that xlrd 1 can open but openpyxl can't - _pandas_ won't be
able to open those either way, but allowing other applications to do so
is still worth something.

There also may be applications that could switch to openpyxl but simply
haven't. I don't know how much effort switching is / whether it would
be reasonably possible for us to do it.

However, I wasn't aware of the security issues in xlrd 1 when I wrote
that, and they may well be a reason to go to xlrd 2 and accept this
breakage. Are they the long-standing "denial of service via excessive
XML entity expansion" or is there now (also) a risk of something worse?