ANNOUNCE: xlrd 0.5.2 -- extract data from Excel spreadsheets

34 views
Skip to first unread message

John Machin

unread,
Mar 18, 2006, 2:10:43 PM3/18/06
to pytho...@python.org
I am pleased to announce a new general release (0.5.2) of xlrd, a Python
package for extracting data from Microsoft Excel spreadsheets.

CHANGES:

* Book and sheet objects can now be pickled and unpickled. Instead of
reading a large spreadsheet multiple times, consider pickling it once
and loading the saved pickle; can be much faster.

* Now works with Python 2.1. Backporting to Python 2.1 was partially
funded by Journyx - provider of timesheet and project accounting
solutions (http://journyx.com/)

* open_workbook() can be given the contents of a file instead of its name.

* Now more tolerant of files written in unexpected ways by 3rd party
software.

* Speed improvements. Minor bugfixes.

MAIN FEATURES OF xlrd:

* Library for developers; not a tool for end-users.

* Platform-independent pure Python – you don't need Windows, Excel,
COM, ...

* Handles all Excel file versions back to 3.0.

* Strong support for Excel dates.

AVAILABLE FROM:

http://cheeseshop.python.org/pypi/xlrd

http://www.lexicon.net/sjmachin/xlrd

ENQUIRIES:

E-mail to sjmachin at lexicon.net with [xlrd] in the subject.

Cheers,
John

Kent Johnson

unread,
Mar 18, 2006, 4:31:01 PM3/18/06
to
John Machin wrote:
> I am pleased to announce a new general release (0.5.2) of xlrd, a Python
> package for extracting data from Microsoft Excel spreadsheets.

How does xlrd compare with pyexcelerator? At a glance they look pretty
similar.

Thanks,
Kent

John Machin

unread,
Mar 18, 2006, 7:01:17 PM3/18/06
to Roman Kiseliov

I have an obvious bias, so I'll just leave you with a not-very-PC
analogy to think about:

Depending on the ambient light and the quantity of mead drunk in the
hall, Helen of Troy and Grendel's mum might at a glance look pretty
similar, too. Perhaps a more thorough investigation is needed. What
about your requirements: a partner for the graduation ball, or someone
to lift the truck off you when the jack collapses?

Cheers,
John

Waldemar Osuch

unread,
Mar 18, 2006, 8:55:45 PM3/18/06
to
- xlrd seems to be focused on extracting data.
- pyexcelerator can also generate Excel files.

Kent Johnson

unread,
Mar 18, 2006, 10:30:43 PM3/18/06
to

That didn't shed much light. I'm interested in your biased opinion,
certainly you must have had a reason to write a new package.

I don't have current requirements for this, I'm curious.

Kent

John Machin

unread,
Mar 19, 2006, 4:27:14 AM3/19/06
to Roman Kiseliov

* It's not new. First public release was on 2005-05-15. When I started
writing it, there was no pure-Python Excel reading facility available at
all. Roman Kiseliov's pyExcelerator was first announced on c.l.py.ann
on 2005-03-22 (write only) and later with an import facility on 2005-05-12.

* I wrote it because I needed to get data out of Excel spreadsheets in a
production environment. I had tried COM, ODBC, and manual save-as-CSV
and all those approaches were unsatisfactory in terms of reliability and
robustness.

* Creating Excel files: xlrd doesn't do this. If you have Python 2.4,
pyExcelerator is the best choice. The alternative is PyXLWriter which
works with Python 2.2 onwards but is no longer maintained and writes
only the older Excel 5.0 / Excel 95 file format (no Unicode support,
limited to 16K rows).

* Herewith the biased comparison:
[I've CCed Roman on this & the previous posting, by the way]

** Python versions: xlrd 2.1 to 2.4, pyExcelerator 2.4 only

** Excel file versions: xlrd 3.0 onwards, pyExcelerator 5.0 onwards [I
doubt there are many 3.0, 4S and 4W files hanging about but I had some
Excel 4.0 files so I did it as a jeu d'esprit]

** Date support: Excel stores dates as real numbers of days sort-of
since some variable date ... a long shocking story, read the xlrd docs.
xlrd puts considerable effort into examining the formats used by number
cells so that they can be classifed as date or not-date. It provides
functions for converting between Excel date numbers and datetime tuples.

** Speed: On a 3.2GHz Intel P4 with 1 GB of RAM, xlrd loads a 128 Mb
spreadsheet [yes sir, real live user data] in a little over a minute,
while pyExcelerator takes a little over 3 minutes. Taking out half of
that memory doesn't bother xlrd at all, but ...

** Memory footprint: pyExcelerator can use from 2.5 to 4 times as much
memory as xlrd.

** Docs: xlrd has docs.

Hoping that shed some light :-)
Cheers,
John

Kent Johnson

unread,
Mar 19, 2006, 6:53:07 AM3/19/06
to
John Machin wrote:
> On 19/03/2006 2:30 PM, Kent Johnson wrote:

>>That didn't shed much light. I'm interested in your biased opinion,
>>certainly you must have had a reason to write a new package.
>

> * It's not new. First public release was on 2005-05-15. When I started
> writing it, there was no pure-Python Excel reading facility available at
> all. Roman Kiseliov's pyExcelerator was first announced on c.l.py.ann
> on 2005-03-22 (write only) and later with an import facility on 2005-05-12.

Ah, my mistake, sorry. I didn't recall hearing about xlrd before and
jumped to the conclusion that it was new, though your OP is very clear
that this is an update release.

> * Herewith the biased comparison:

Thank you!

Kent

Scott David Daniels

unread,
Mar 19, 2006, 10:14:42 AM3/19/06
to
Kent Johnson wrote:
> John Machin wrote:

>> * Herewith the biased comparison: ....
>
> Thank you!

Thank you (John) as well. I realize you are a bit reluctant to toot
your own horn, but it is just this kind of biased comparison that
let's us know whether to investigate further. It also helps that
you mention what pyExcelerator is good at.

--
-Scott David Daniels
scott....@acm.org

jcmendez

unread,
Mar 20, 2006, 5:28:10 AM3/20/06
to
John

Thanks for walking us through the comparison. On the xlrd website I
saw that it does not import formulas from the Excel files, which is
what I'm looking for. Any suggestions?

Juan C.

John Machin

unread,
Mar 20, 2006, 5:39:25 AM3/20/06
to jcmendez
Juan, what do you want to do with the formulas after importing them?
Pretty-print them? Evaluate them?
Cheers,
John

jcmendez

unread,
Mar 20, 2006, 6:00:45 AM3/20/06
to
Hi John

I'd like to create a dependency graph and plot it with Graphviz. I've
played a bit with exporting the sheet in XML format, and parsing the
XML. That somehow works, but it would be much better if the users
wouldn't need to save as the sheets, just put them is a shared
directory where I can get them with the script.

Thanks!!

John Machin

unread,
Mar 20, 2006, 6:17:28 AM3/20/06
to jcmendez

I guess that you need, for each formula cell in a worksheet, a list of
the cells that are referred to by the formula ... is that correct?


jcmendez

unread,
Mar 20, 2006, 6:37:17 AM3/20/06
to
Exactly. Once I get the formulas, I can do a weak parsing of them and
find the references.

John Machin

unread,
Mar 20, 2006, 7:01:59 AM3/20/06
to jcmendez
On 20/03/2006 10:37 PM, jcmendez wrote:
> Exactly.

Yes, your requirement is exactly that, a list of references.


> Once I get the formulas, I can do a weak parsing of them and
> find the references.
>

A formula is not stored as input e.g. "(A1+A2)*3.0+$Z$29"; it's kept as
an RPN stream of variable-length tokens. You don't need to parse it in
the sense you were probably thinking of; *ALL* [ho ho chuckle chuckle]
you need to do is step through the tokens and do something with the ones
that contain references. Have a peek at this:
http://sc.openoffice.org/excelfileformat.pdf -- formulas are covered in
40+ pages starting at about page 25.

Cheers,
John

Felipe Almeida Lessa

unread,
Mar 20, 2006, 9:14:35 AM3/20/06
to John Machin, pytho...@python.org
Em Seg, 2006-03-20 às 23:01 +1100, John Machin escreveu:
> *ALL* [ho ho chuckle chuckle]
> you need to do is step through the tokens and do something with the ones
> that contain references.

And contribute back the code? =)

--
Felipe.

cu...@journyx.com

unread,
Mar 30, 2006, 11:22:26 AM3/30/06
to
the 2nd url seems to be a dead link?

Reply all
Reply to author
Forward
0 new messages