ElementTree - Why not part of the core?
doug.b...@gmail.com
I've recently been developing a script for accessing the Miva API only
to find all the core API's provided by Python for parsing XML is messy
and complicated. Many of the examples I see for parsing the data using
these API's uses a similar additional Class for collapsing the XML data
into a more manageable format.
This is clearly not following the Python-way of clean, simple code and
easy development.
ElementTree on the other hand provides incredibly easy access to XML
elements and works in a more Pythonic way. Why has the API not been
included in the Python core?
jelle
I agree with you, ElementTree is fast & pythonic.
Certainly does make sense to me.
Steven Bethard
> ElementTree on the other hand provides incredibly easy access to XML
> elements and works in a more Pythonic way. Why has the API not been
> included in the Python core?
While I fully agree that ElementTree is far more Pythonic than the
dom-based stuff in the core, this issue has been discussed on
python-dev[1]. Fredrik Lundh's response:
shipping stable versions of ElementTree/cElementTree (or PIL, or
python-doc, or exemaker, or what else you might find useful) with
official Python releases is perfectly okay.
moving the main trunk and main development over to the Python CVS is
another thing, entirely.
I think some people were hoping that instead of adding these things to
the standard library, we would come up with a better package manager
that would make adding these things to your local library much simpler.
STeVe
[1]http://www.python.org/dev/summary/2005-06-01_2005-06-15.html#reorganising-the-standard-library-again
Jarek Zgoda
>> ElementTree on the other hand provides incredibly easy access to XML
>> elements and works in a more Pythonic way. Why has the API not been
>> included in the Python core?
>
> While I fully agree that ElementTree is far more Pythonic than the
> dom-based stuff in the core, this issue has been discussed on
> python-dev[1]. Fredrik Lundh's response:
>
> shipping stable versions of ElementTree/cElementTree (or PIL, or
> python-doc, or exemaker, or what else you might find useful) with
> official Python releases is perfectly okay.
>
> moving the main trunk and main development over to the Python CVS is
> another thing, entirely.
>
> I think some people were hoping that instead of adding these things to
> the standard library, we would come up with a better package manager
> that would make adding these things to your local library much simpler.
What about more intelligent installer, that after installing core
packages, would present user a list of recommended packages in few
groups (like "markup processing", "popular RDBMS interaction", "web
frameworks", "multimedia", etc.) and would be able to fetch and install
packages using PyPI descriptions? Unattended install with this feature
would be great.
--
Jarek Zgoda
http://jpa.berlios.de/
Fredrik Lundh
> > ElementTree on the other hand provides incredibly easy access to XML
> > elements and works in a more Pythonic way. Why has the API not been
> > included in the Python core?
>
> While I fully agree that ElementTree is far more Pythonic than the
> dom-based stuff in the core, this issue has been discussed on
> python-dev[1]. Fredrik Lundh's response:
>
> shipping stable versions of ElementTree/cElementTree (or PIL, or
> python-doc, or exemaker, or what else you might find useful) with
> official Python releases is perfectly okay.
>
> moving the main trunk and main development over to the Python CVS is
> another thing, entirely.
>
> I think some people were hoping that instead of adding these things to
> the standard library, we would come up with a better package manager
> that would make adding these things to your local library much simpler.
I still hope that the standard distribution will, in a not too distant future,
bundle more external libraries. as things are today, "including something
in the core" means that you have to transfer code and rights to the PSF.
as I've said many times, if the Linux folks can build distributions that con-
sists of thousands of individually maintained pieces, the Python distributors
should be able to handle a few dozen components.
</F>
Simon Percivall
different versions of a package. You'll want to be able to use newer
versions of external packages without breakage in the standard library.
ru...@yahoo.com
Steven Bethard wrote:
--snip--
> I think some people were hoping that instead of adding these things to
> the standard library, we would come up with a better package manager
> that would make adding these things to your local library much simpler.
>
> STeVe
>
> [1]http://www.python.org/dev/summary/2005-06-01_2005-06-15.html#reorganising-the-standard-library-again
A better package manager would be great but does not
replace having things in the core. Distributed code that
relies on external packages puts a significantly greater
burden on the user of the code. I suppose there are
ways to get around it but often it means at a minimum
the user will need to read the install instructions (which
seems to be asking a lot for some people.) And at some
companies, one has to jump though beauracratic hoops
for each external package installed. And I personally
stear away from packages that have a long list of
prerequisites.
With things I've written that I thought might be used
by others, I've gone to some lengths to use only modules
in the standard library.
Mike Meyer
> as I've said many times, if the Linux folks can build distributions that con-
> sists of thousands of individually maintained pieces, the Python distributors
> should be able to handle a few dozen components.
Yes, but "distributers" is not necessarily "maintainers". How many
components are there in the Activestate Python distribution?
Enthoughts? Others?
The maintainers of Linux/gcc/etc. don't worry about creating a
distribution that includes lots of third party software. Why should
the maintainers of Python be any different?
<mike
--
Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Alex Martelli
...
> seems to be asking a lot for some people.) And at some
> companies, one has to jump though beauracratic hoops
> for each external package installed. And I personally
> stear away from packages that have a long list of
> prerequisites.
Funny -- me, I prefer to steer away from companies with lots of
bureaucracy, instead!-)
Alex
Magnus Lycka
> ElementTree on the other hand provides incredibly easy access to XML
> elements and works in a more Pythonic way. Why has the API not been
> included in the Python core?
I'd really like to see that too. Sure, it's fairly trivial to install
it, but each different package that needs to be installed from another
source, built and tested on multiple platforms etc, means more work.
For complex software systems, these kinds of configuration issues
take a lot of efforts.
We're deploying our software on a number of different platforms. We
certainly depend on Python, so a standard Python install will always
be included. Using standard library modules is for free. Using yet
another third party library has a cost, even if some Cheese Shop or
whatever might make the cost slightly lower.
We have very friendly configuration management guys here, so if I
insisted, I'm sure they'd let me have elementTree included, but I
hesitate to ask for it. It does add a burden on CM, increases the
risk that a deployment will fail, and from a code maintenance point
of view, it's a double edged sword: On one hand, it's a much better
API, but on the other hand, it's not documented in the standard
Python docs or in the Nutshell book, and a new developer would need
to figure out what this foreign beast was, and where to find docs.
(Alternatively, I'd have to (God forbid) document my actions. ;)
Fredrik Lundh
> We're deploying our software on a number of different platforms. We
> certainly depend on Python, so a standard Python install will always
> be included. Using standard library modules is for free. Using yet
> another third party library has a cost, even if some Cheese Shop or
> whatever might make the cost slightly lower.
>
> We have very friendly configuration management guys here, so if I
> insisted, I'm sure they'd let me have elementTree included, but I
> hesitate to ask for it. It does add a burden on CM, increases the
> risk that a deployment will fail, and from a code maintenance point
> of view, it's a double edged sword: On one hand, it's a much better
> API, but on the other hand, it's not documented in the standard
> Python docs or in the Nutshell book, and a new developer would need
> to figure out what this foreign beast was, and where to find docs.
> (Alternatively, I'd have to (God forbid) document my actions. ;)
on the other hand, you can bundle ElementTree with your applications simply
by including the ElementTree.py and (optionally) the ElementPath.py modules.
(you are allowed to add new Python modules to the project, I hope ;-)
(and yes, HTML documentation is included in the source distribution kit)
but seriously, given how easy it is to build things with distutils, I don't think your
configuration folks would have much trouble adding support for "anything that has
a setup file, and is reasonably self-contained" to their build scripts.
we usually use one large shell script with a bunch of "cd source; run setup install"
sections in it, e.g.
cd $PREFIX
cd src/griblib-2.1.1-20051125
$PYTHON setup.py install --prefix=$PREFIX
plus a Python script that does a sanity check at the end to make sure that the build
script actually managed to build and install everything we need. running the regression
test suite on the build is a good way to weed out build problems too; just make sure
you redirect the build output to a file, so you can check the logs afterwards.
all source kits are checked into the version management system, of course. no tarballs
are involved in the build process. just check things out and run the script, wait a couple
of minutes, and you're done.
</F>
Fuzzyman
I'd like to add my vote in favour of this.
There are a few popular extensions that most users would like easy
access to. PIL and ElementTree both fall into this category.
Thanks
Fuzzyman
http://www.voidspace.org.uk/python/index.shtml
> </F>
Giovanni Bajo
>> I think some people were hoping that instead of adding these things
>> to
>> the standard library, we would come up with a better package manager
>> that would make adding these things to your local library much
>> simpler.
>>
>> STeVe
>>
>>
[1]http://www.python.org/dev/summary/2005-06-01_2005-06-15.html#reorganising-th
e-standard-library-again
>
> A better package manager would be great but does not
> replace having things in the core. Distributed code that
> relies on external packages puts a significantly greater
> burden on the user of the code.
Seconded.
One thing I really fear about the otherwise great EasyInstall (and Python Eggs)
is that we could forget about
Let's not turn the Python standard library into the CPAN mess, where there are
5 different libraries for adding two numbers, so that it's then impossible to
grab a random perl program and read it, without going through 150 different man
pages you never saw before. I don't need 450 libraries to compute MD5, or to
zip a file, or 140 different implementations of random numbers. There will
always be external libraries for specific purposes, but I'd rather the standard
library to stay focused on provided a possibly restricted set of common
features with a decent interface/implementation.
This said, I'd also like to see ElementTree in the standard library. We already
have a SAX and a DOM, but we need a Pythonic XML library, and ElementTree is
just perfect.
--
Giovanni Bajo
Giovanni Bajo
> One thing I really fear about the otherwise great EasyInstall (and
> Python Eggs) is that we could forget about...
... how important is to have a standard library. The fact that it's easy to
install external modules shouldn't make us drop the standard library. A
standard library means a great uniformity across programs. Whenever I open a
Python program which uses ZipFile, or socket, or re, I can read it
*immediately*. If it uses an external library / framework, I have to go study
its manual and documentation. Proliferation of external libraries is good, but
people should normally use the standard library modules for uniformity.
In other words, I disagree with this message:
http://mail.python.org/pipermail/python-dev/2005-June/054102.html
My personal hope is that Python 3.0 will have a nice cleaned-up standard
library with even more features than Python 2.x. As I said the in other
message, let's not end up into the CPAN mess just because it's now technically
easier!
--
Giovanni Bajo
Magnus Lycka
> but seriously, given how easy it is to build things with distutils, I don't think your
> configuration folks would have much trouble adding support for "anything that has
> a setup file, and is reasonably self-contained" to their build scripts.
True. It's one more thing to keep track of though. It's a separate
configuration item. We need to keep track of which version to use,
and it needs to get built, installed and validated on all supported
platforms.
I already include Twisted (and thus ZopeInterface) like that. We
sometimes need high speed in XML parsing, so I'd like to use
cElementTree, but just like the two other packages, I assume that
it's unproblematic to build the included C code. So, you're right,
it's not much trouble, just two more pebbles (cElementTree and
ElementTree) on a fairly heavy load, where stuff like Oracle and MQ
Series are more like boulders.
Right now, I would like to improve the performace of a program that
does a lot of XML parsing. I haven't profiled that yet, so I don't
know if XML parsing is the bottle neck, but if I can improve perfor-
mance significantly with cElementTree, I'll certainly push for that.
/Magnus
sk...@pobox.com
>> ElementTree on the other hand provides incredibly easy access to XML
>> elements and works in a more Pythonic way. Why has the API not been
>> included in the Python core?
Magnus> I'd really like to see that too. Sure, it's fairly trivial to
Magnus> install it, but each different package that needs to be
Magnus> installed from another source, built and tested on multiple
Magnus> platforms etc, means more work.
More work than reimplementing the functionality or living with a miserable
API (I'm thinking normal DOM APIs)?
Skip
sk...@pobox.com
>> ElementTree on the other hand provides incredibly easy access to XML
>> elements and works in a more Pythonic way. Why has the API not been
>> included in the Python core?
I think the key here is ElementTree's Pythoninc API. While it's clearly
possible to install it as a third-party package, I think there's a clear
best-of-breed aspect here that suggests it belongs in the standard
distribution simply to discourage continued use of DOM-based APIs.
Skip
Jeremy Hylton
> bundle more external libraries. as things are today, "including something
> in the core" means that you have to transfer code and rights to the PSF.
Your description of how to include something in the core isn't entirely
accurate. If you visit http://www.python.org/psf/contrib.html, you'll
see that you don't need to transfer your rights to the PSF. Rather,
you need to sign an agreement giving the PSF the rights to relicense
the code. The PSF currently require that the original submission be
licensed under one of two open source licenses. Personally, I can
imagine accepting a wider range of initial licenses in the future.
As for transferring the code, there needs to be a copy in the Python
source control system, sure. There's some unavoidable complexity
involved in managing a software distribution composed of third party
software packages. At the very least, you've got the original sources
and the copy in the distribution package, which leads to a
synchronization problem.
Jeremy
Steven Bethard
> I think the key here is ElementTree's Pythoninc API. While it's clearly
> possible to install it as a third-party package, I think there's a clear
> best-of-breed aspect here that suggests it belongs in the standard
> distribution simply to discourage continued use of DOM-based APIs.
I second this. Guido has said many times that the stdlib is for
best-of-breed modules that have proven themselves in the wild.
ElementTree has proven itself in the wild and is clearly best-of-breed.
And dramatically better (IMHO) than the APIs currently included in the
stdlib[1].
I don't have a whole lot of free time, and I'm not sure exactly how I
could help, but if there's anything I could do that would help get
ElementTree into the stdlib, let me know.
STeVe
[1] If I had my way, we'd deprecate and then remove the current Python
xml modules. But of course then people would complain that Python
doesn't have a SAX or DOM API. Of course we could tell them that they
don't need it and that ElementTree is easier, but I'm not sure people
really want to fight that battle.
Magnus Lycka
More work for the CM people. Less work for me.
Fredrik Lundh
> ElementTree on the other hand provides incredibly easy access to XML
> elements and works in a more Pythonic way. Why has the API not been
> included in the Python core?
$ svn up
$ make
...
$ python
Python 2.5a0 (#1, Dec 12 2005, 19:26:49)
>>> import xml.etree.ElementTree as ET
>>> root = ET.XML("<message>hello, world!</message>")
>>> root
<Element text at 402f73ac>
>>> root.text
'hello, world!'
http://svn.python.org/view/python/trunk/Lib/xml/etree/
(and it's all your fault ;-)
</F>
Gregory Petrosyan
Python 2.5a0 (#1, Dec 14 2005, 14:11:55)
[GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.etree.ElementTree as ET
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.5/xml.py", line 20, in <module>
import xmlcore
ImportError: No module named xmlcore
>>>
Gregory Petrosyan
make install
and
me@home:~$ python
When launching ./python from ./trunk, everything is OK. Is it a problem
with me or with installation?
Istvan Albert
> Python 2.5a0 (#1, Dec 12 2005, 19:26:49)
>>> import xml.etree.ElementTree as ET
hip hip hurray!
> http://svn.python.org/view/python/trunk/Lib/xml/etree/
don't know how this works, the link now seems to be:
Steve Holden
Compatibility code will replace the current xml package, to ensure that
core modules not replaced by other components will still be available
under their old names. This will allow better handling of fallbacks
when, say, an optional C extension module is absent and it can be
replaced by an xmlcore pure Python module.
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/
Fredrik Lundh
> don't know how this works, the link now seems to be:
>
> http://svn.python.org/view/python/trunk/Lib/xmlcore/etree/
people are moving things around, but xml.etree.ElementTree should still
work when they're done.
("xml" dispatches to either xmlcore or _xmlplus, depending on what you
have installed. "xmlcore" always gives you the bundled stuff.)
</F>
Fredrik Lundh
things are being moved around, and installation seems to be broken at the
moment (it doesn't install the Lib/xmlcore tree)
</F>
Fredrik Lundh
> > with me or with installation?
>
> moment (it doesn't install the Lib/xmlcore tree)
if you have time, can you update and try again ?
(let me know if it's still broken on your machine)
</F>
Gregory Petrosyan
Python 2.5a0 (#4, Dec 14 2005, 22:08:59)
[GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> ET
<module 'xml.etree.ElementTree' from
'/usr/local/lib/python2.5/xmlcore/etree/ElementTree.pyc'>
>>>
Thanks for fix.