ANNOUNCE: xlrd 0.5.2 -- extract data from Excel spreadsheets

[John Machin]

I am pleased to announce a new general release (0.5.2) of xlrd, a Python
package for extracting data from Microsoft Excel spreadsheets.

CHANGES:

* Book and sheet objects can now be pickled and unpickled. Instead of
reading a large spreadsheet multiple times, consider pickling it once
and loading the saved pickle; can be much faster.

* Now works with Python 2.1. Backporting to Python 2.1 was partially
funded by Journyx - provider of timesheet and project accounting
solutions (http://journyx.com/)

* open_workbook() can be given the contents of a file instead of its name.

* Now more tolerant of files written in unexpected ways by 3rd party
software.

* Speed improvements. Minor bugfixes.

MAIN FEATURES OF xlrd:

* Library for developers; not a tool for end-users.

* Platform-independent pure Python – you don't need Windows, Excel,
COM, ...

* Handles all Excel file versions back to 3.0.

* Strong support for Excel dates.

AVAILABLE FROM:

http://cheeseshop.python.org/pypi/xlrd

http://www.lexicon.net/sjmachin/xlrd

ENQUIRIES:

E-mail to sjmachin at lexicon.net with [xlrd] in the subject.

Cheers,
John

[Kent Johnson]

John Machin wrote:
> I am pleased to announce a new general release (0.5.2) of xlrd, a Python
> package for extracting data from Microsoft Excel spreadsheets.

How does xlrd compare with pyexcelerator? At a glance they look pretty
similar.

Thanks,
Kent

[John Machin]

I have an obvious bias, so I'll just leave you with a not-very-PC
analogy to think about:

Depending on the ambient light and the quantity of mead drunk in the
hall, Helen of Troy and Grendel's mum might at a glance look pretty
similar, too. Perhaps a more thorough investigation is needed. What
about your requirements: a partner for the graduation ball, or someone
to lift the truck off you when the jack collapses?

Cheers,
John

[Waldemar Osuch]

- xlrd seems to be focused on extracting data.
- pyexcelerator can also generate Excel files.

[Kent Johnson]

That didn't shed much light. I'm interested in your biased opinion,
certainly you must have had a reason to write a new package.

I don't have current requirements for this, I'm curious.

Kent

[John Machin]

* It's not new. First public release was on 2005-05-15. When I started
writing it, there was no pure-Python Excel reading facility available at
all. Roman Kiseliov's pyExcelerator was first announced on c.l.py.ann
on 2005-03-22 (write only) and later with an import facility on 2005-05-12.

* I wrote it because I needed to get data out of Excel spreadsheets in a
production environment. I had tried COM, ODBC, and manual save-as-CSV
and all those approaches were unsatisfactory in terms of reliability and
robustness.

* Creating Excel files: xlrd doesn't do this. If you have Python 2.4,
pyExcelerator is the best choice. The alternative is PyXLWriter which
works with Python 2.2 onwards but is no longer maintained and writes
only the older Excel 5.0 / Excel 95 file format (no Unicode support,
limited to 16K rows).

* Herewith the biased comparison:
[I've CCed Roman on this & the previous posting, by the way]

** Python versions: xlrd 2.1 to 2.4, pyExcelerator 2.4 only

** Excel file versions: xlrd 3.0 onwards, pyExcelerator 5.0 onwards [I
doubt there are many 3.0, 4S and 4W files hanging about but I had some
Excel 4.0 files so I did it as a jeu d'esprit]

** Date support: Excel stores dates as real numbers of days sort-of
since some variable date ... a long shocking story, read the xlrd docs.
xlrd puts considerable effort into examining the formats used by number
cells so that they can be classifed as date or not-date. It provides
functions for converting between Excel date numbers and datetime tuples.

** Speed: On a 3.2GHz Intel P4 with 1 GB of RAM, xlrd loads a 128 Mb
spreadsheet [yes sir, real live user data] in a little over a minute,
while pyExcelerator takes a little over 3 minutes. Taking out half of
that memory doesn't bother xlrd at all, but ...

** Memory footprint: pyExcelerator can use from 2.5 to 4 times as much
memory as xlrd.

** Docs: xlrd has docs.

Hoping that shed some light :-)
Cheers,
John

[Kent Johnson]

John Machin wrote:
> On 19/03/2006 2:30 PM, Kent Johnson wrote:

>>That didn't shed much light. I'm interested in your biased opinion,
>>certainly you must have had a reason to write a new package.
>

> * It's not new. First public release was on 2005-05-15. When I started
> writing it, there was no pure-Python Excel reading facility available at
> all. Roman Kiseliov's pyExcelerator was first announced on c.l.py.ann
> on 2005-03-22 (write only) and later with an import facility on 2005-05-12.

Ah, my mistake, sorry. I didn't recall hearing about xlrd before and
jumped to the conclusion that it was new, though your OP is very clear
that this is an update release.

> * Herewith the biased comparison:

Thank you!

Kent

[Scott David Daniels]

Kent Johnson wrote:
> John Machin wrote:

>> * Herewith the biased comparison: ....
>
> Thank you!

Thank you (John) as well. I realize you are a bit reluctant to toot
your own horn, but it is just this kind of biased comparison that
let's us know whether to investigate further. It also helps that
you mention what pyExcelerator is good at.

--
-Scott David Daniels
scott....@acm.org

[jcmendez]

John

Thanks for walking us through the comparison. On the xlrd website I
saw that it does not import formulas from the Excel files, which is
what I'm looking for. Any suggestions?

Juan C.

[John Machin]

Juan, what do you want to do with the formulas after importing them?
Pretty-print them? Evaluate them?
Cheers,
John

[jcmendez]

Hi John

I'd like to create a dependency graph and plot it with Graphviz. I've
played a bit with exporting the sheet in XML format, and parsing the
XML. That somehow works, but it would be much better if the users
wouldn't need to save as the sheets, just put them is a shared
directory where I can get them with the script.

Thanks!!

[John Machin]

I guess that you need, for each formula cell in a worksheet, a list of
the cells that are referred to by the formula ... is that correct?

[jcmendez]

Exactly. Once I get the formulas, I can do a weak parsing of them and
find the references.

[John Machin]

On 20/03/2006 10:37 PM, jcmendez wrote:
> Exactly.

Yes, your requirement is exactly that, a list of references.

> Once I get the formulas, I can do a weak parsing of them and
> find the references.
>

A formula is not stored as input e.g. "(A1+A2)*3.0+$Z$29"; it's kept as
an RPN stream of variable-length tokens. You don't need to parse it in
the sense you were probably thinking of; *ALL* [ho ho chuckle chuckle]
you need to do is step through the tokens and do something with the ones
that contain references. Have a peek at this:
http://sc.openoffice.org/excelfileformat.pdf -- formulas are covered in
40+ pages starting at about page 25.

Cheers,
John

[Felipe Almeida Lessa]

Em Seg, 2006-03-20 às 23:01 +1100, John Machin escreveu:
> *ALL* [ho ho chuckle chuckle]
> you need to do is step through the tokens and do something with the ones
> that contain references.

And contribute back the code? =)

--
Felipe.

[cu...@journyx.com]

the 2nd url seems to be a dead link?