Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
xml.etree and namespaces -- why?
451 views
Skip to first unread message
Robert Latest
Oct 19, 2022, 9:26:16 AM10/19/22
to
Hi all,
For the impatient: Below the longish text is a fully self-contained Python
example that illustrates my problem.
I'm struggling to understand xml.etree's handling of namespaces. I'm trying to
parse an Inkscape document which uses several namespaces. From etree's
documentation:
If the XML input has namespaces, tags and attributes with prefixes in the
form prefix:sometag get expanded to {uri}sometag where the prefix is
replaced by the full URI.
Which means that given an Element e, I cannot directly access its attributes
using e.get() because in order to do that I need to know the URI of the
namespace. So rather than doing this (see example below):
label = e.get('inkscape:label')
I need to do this:
label = e.get('{' + uri_inkscape_namespace + '}label')
...which is the method mentioned in etree's docs:
One way to search and explore this XML example is to manually add the URI
to every tag or attribute in the xpath of a find() or findall().
[...]
A better way to search the namespaced XML example is to create a
dictionary with your own prefixes and use those in the search functions.
Good idea! Better yet, that dictionary or rather, its reverse, already exists,
because etree has used it to unnecessarily mangle the namespaces in the first
place. The documentation doesn't mention where it can be found, but we can
just use the 'xmlns:' attributes of the <svg> root element to rebuild it. Or
so I thought, until I found out that etree deletes exactly these attributes
before handing the <svg> element to the user.
I'm really stumped here. Apart from the fact that I think XML is bloated shit
anyway and has no place outside HTML, I just don't get the purpose of etree's
way of working:
1) Evaluate 'xmlns:' attributes of the <svg> element
2) Use that info to replace the existing prefixes by {uri}
3) Realizing that using {uri} prefixes is cumbersome, suggest to
the user to build their own prefix -> uri dictionary
to undo the effort of doing 1) and 2)
4) ...but witholding exactly the information that existed in the original
document by deleting the 'xmlns:' attributes from the <svg> tag
Why didn't they leave the whole damn thing alone? Keep <svg> intact and keep
the attribute 'prefix:key' literally as they are. For anyone wanting to use
the {uri} prefixes (why would they) they could have thrown in a helper
function for the prefix->URI translation.
I'm assuming that etree's designers knew what they were doing in order to make
my life easier when dealing with XML. Maybe I'm missing the forest for the
trees. Can anybody enlighten me? Thanks!
#### self-contained example
import xml.etree.ElementTree as ET
def test_svg(xml):
root = ET.fromstring(xml)
for e in root.iter():
print(e.tag) # tags are shown prefixed with {URI}
if e.tag.endswith('svg'):
# Since namespaces are defined inside the <svg> tag, let's use the info
# from the 'xmlns:' attributes to undo etree's URI prefixing
print('Element <svg>:')
for k, v in e.items():
print(' %s: %s' % (k, v))
# ...but alas: the 'xmlns:' attributes have been deleted by the parser
xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
width="210mm"
height="297mm"
viewBox="0 0 210 297"
version="1.1"
id="svg285"
inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
sodipodi:docname="test.svg"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<sodipodi:namedview
id="namedview287"
pagecolor="#ffffff"
bordercolor="#000000"
borderopacity="0.25"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
inkscape:document-units="mm"
showgrid="false"
inkscape:zoom="0.2102413"
inkscape:cx="394.78447"
inkscape:cy="561.25984"
inkscape:window-width="1827"
inkscape:window-height="1177"
inkscape:window-x="85"
inkscape:window-y="-8"
inkscape:window-maximized="1"
inkscape:current-layer="layer1" />
<defs
id="defs282" />
<g
inkscape:label="Ebene 1"
inkscape:groupmode="layer"
id="layer1">
<rect
style="fill:#aaccff;stroke-width:0.264583"
id="rect289"
width="61.665253"
height="54.114403"
x="33.978813"
y="94.38559" />
</g>
</svg>
'''
if __name__ == '__main__':
test_svg(xml)
For the impatient: Below the longish text is a fully self-contained Python
example that illustrates my problem.
I'm struggling to understand xml.etree's handling of namespaces. I'm trying to
parse an Inkscape document which uses several namespaces. From etree's
documentation:
If the XML input has namespaces, tags and attributes with prefixes in the
form prefix:sometag get expanded to {uri}sometag where the prefix is
replaced by the full URI.
Which means that given an Element e, I cannot directly access its attributes
using e.get() because in order to do that I need to know the URI of the
namespace. So rather than doing this (see example below):
label = e.get('inkscape:label')
I need to do this:
label = e.get('{' + uri_inkscape_namespace + '}label')
...which is the method mentioned in etree's docs:
One way to search and explore this XML example is to manually add the URI
to every tag or attribute in the xpath of a find() or findall().
[...]
A better way to search the namespaced XML example is to create a
dictionary with your own prefixes and use those in the search functions.
Good idea! Better yet, that dictionary or rather, its reverse, already exists,
because etree has used it to unnecessarily mangle the namespaces in the first
place. The documentation doesn't mention where it can be found, but we can
just use the 'xmlns:' attributes of the <svg> root element to rebuild it. Or
so I thought, until I found out that etree deletes exactly these attributes
before handing the <svg> element to the user.
I'm really stumped here. Apart from the fact that I think XML is bloated shit
anyway and has no place outside HTML, I just don't get the purpose of etree's
way of working:
1) Evaluate 'xmlns:' attributes of the <svg> element
2) Use that info to replace the existing prefixes by {uri}
3) Realizing that using {uri} prefixes is cumbersome, suggest to
the user to build their own prefix -> uri dictionary
to undo the effort of doing 1) and 2)
4) ...but witholding exactly the information that existed in the original
document by deleting the 'xmlns:' attributes from the <svg> tag
Why didn't they leave the whole damn thing alone? Keep <svg> intact and keep
the attribute 'prefix:key' literally as they are. For anyone wanting to use
the {uri} prefixes (why would they) they could have thrown in a helper
function for the prefix->URI translation.
I'm assuming that etree's designers knew what they were doing in order to make
my life easier when dealing with XML. Maybe I'm missing the forest for the
trees. Can anybody enlighten me? Thanks!
#### self-contained example
import xml.etree.ElementTree as ET
def test_svg(xml):
root = ET.fromstring(xml)
for e in root.iter():
print(e.tag) # tags are shown prefixed with {URI}
if e.tag.endswith('svg'):
# Since namespaces are defined inside the <svg> tag, let's use the info
# from the 'xmlns:' attributes to undo etree's URI prefixing
print('Element <svg>:')
for k, v in e.items():
print(' %s: %s' % (k, v))
# ...but alas: the 'xmlns:' attributes have been deleted by the parser
xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
width="210mm"
height="297mm"
viewBox="0 0 210 297"
version="1.1"
id="svg285"
inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
sodipodi:docname="test.svg"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<sodipodi:namedview
id="namedview287"
pagecolor="#ffffff"
bordercolor="#000000"
borderopacity="0.25"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
inkscape:document-units="mm"
showgrid="false"
inkscape:zoom="0.2102413"
inkscape:cx="394.78447"
inkscape:cy="561.25984"
inkscape:window-width="1827"
inkscape:window-height="1177"
inkscape:window-x="85"
inkscape:window-y="-8"
inkscape:window-maximized="1"
inkscape:current-layer="layer1" />
<defs
id="defs282" />
<g
inkscape:label="Ebene 1"
inkscape:groupmode="layer"
id="layer1">
<rect
style="fill:#aaccff;stroke-width:0.264583"
id="rect289"
width="61.665253"
height="54.114403"
x="33.978813"
y="94.38559" />
</g>
</svg>
'''
if __name__ == '__main__':
test_svg(xml)
Axy
Oct 19, 2022, 11:24:33 AM10/19/22
to
I mean, it's worth to look at BeautifulSoup source how do they do that.
With BS I work with attributes exactly as you want, and I explicitly
tell BS to use lxml parser.
Axy.
With BS I work with attributes exactly as you want, and I explicitly
tell BS to use lxml parser.
Axy.
Jon Ribbens
Oct 19, 2022, 12:15:45 PM10/19/22
to
On 2022-10-19, Robert Latest <bobl...@yahoo.com> wrote:
> If the XML input has namespaces, tags and attributes with prefixes
> in the form prefix:sometag get expanded to {uri}sometag where the
> prefix is replaced by the full URI.
>
> Which means that given an Element e, I cannot directly access its attributes
> using e.get() because in order to do that I need to know the URI of the
> namespace.
That's because you *always* need to know the URI of the namespace,
> If the XML input has namespaces, tags and attributes with prefixes
> in the form prefix:sometag get expanded to {uri}sometag where the
> prefix is replaced by the full URI.
>
> Which means that given an Element e, I cannot directly access its attributes
> using e.get() because in order to do that I need to know the URI of the
> namespace.
because that's its only meaningful identifier. If you assume that a
particular namespace always uses the same prefix then your code will be
completely broken. The following two pieces of XML should be understood
identically:
<svg xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape">
<g inkscape:label="Ebene 1" inkscape:groupmode="layer" id="layer1">
and:
<svg xmlns:epacskni="http://www.inkscape.org/namespaces/inkscape">
<g epacskni:label="Ebene 1" epacskni:groupmode="layer" id="layer1">
So you can see why e.get('inkscape:label') cannot possibly work, and why
e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.
The xml.etree author obviously knew that this was cumbersome, and
hence you can do something like:
namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
element = root.find('inkspace:foo', namespaces)
which will work for both of the above pieces of XML.
But unfortunately as far as I can see nobody's thought about doing the
same for attributes rather than tags.
Axy
Oct 19, 2022, 12:34:11 PM10/19/22
to
I have no idea why, I used to remove namespaces, following the advice
from stackoverflow:
https://stackoverflow.com/questions/4255277/lxml-etree-xmlparser-remove-unwanted-namespace
_ns_removal_xslt_transform = etree.XSLT(etree.fromstring('''
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="no"/>
<xsl:template match="/|comment()|processing-instruction()">
<xsl:copy>
<xsl:apply-templates/>
</xsl:copy>
</xsl:template>
<xsl:template match="*">
<xsl:element name="{local-name()}">
<xsl:apply-templates select="@*|node()"/>
</xsl:element>
</xsl:template>
<xsl:template match="@*">
<xsl:attribute name="{local-name()}">
<xsl:value-of select="."/>
</xsl:attribute>
</xsl:template>
</xsl:stylesheet>
'''))
xml_doc = _ns_removal_xslt_transform(
etree.fromstring(my_xml_data)
)
Later on, when I worked with SVG, I used BeautifulSoup.
Axy.
from stackoverflow:
https://stackoverflow.com/questions/4255277/lxml-etree-xmlparser-remove-unwanted-namespace
_ns_removal_xslt_transform = etree.XSLT(etree.fromstring('''
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="no"/>
<xsl:template match="/|comment()|processing-instruction()">
<xsl:copy>
<xsl:apply-templates/>
</xsl:copy>
</xsl:template>
<xsl:template match="*">
<xsl:element name="{local-name()}">
<xsl:apply-templates select="@*|node()"/>
</xsl:element>
</xsl:template>
<xsl:template match="@*">
<xsl:attribute name="{local-name()}">
<xsl:value-of select="."/>
</xsl:attribute>
</xsl:template>
</xsl:stylesheet>
'''))
xml_doc = _ns_removal_xslt_transform(
etree.fromstring(my_xml_data)
)
Later on, when I worked with SVG, I used BeautifulSoup.
Axy.
Robert Latest
Oct 19, 2022, 1:36:29 PM10/19/22
to
Jon Ribbens wrote:
> That's because you *always* need to know the URI of the namespace,
> because that's its only meaningful identifier. If you assume that a
> particular namespace always uses the same prefix then your code will be
> completely broken. The following two pieces of XML should be understood
> identically:
>
> <svg xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape">
> <g inkscape:label="Ebene 1" inkscape:groupmode="layer" id="layer1">
>
> and:
>
> <svg xmlns:epacskni="http://www.inkscape.org/namespaces/inkscape">
> <g epacskni:label="Ebene 1" epacskni:groupmode="layer" id="layer1">
>
> So you can see why e.get('inkscape:label') cannot possibly work, and why
> e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.
I get it. It does.
> That's because you *always* need to know the URI of the namespace,
> because that's its only meaningful identifier. If you assume that a
> particular namespace always uses the same prefix then your code will be
> completely broken. The following two pieces of XML should be understood
> identically:
>
> <svg xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape">
> <g inkscape:label="Ebene 1" inkscape:groupmode="layer" id="layer1">
>
> and:
>
> <svg xmlns:epacskni="http://www.inkscape.org/namespaces/inkscape">
> <g epacskni:label="Ebene 1" epacskni:groupmode="layer" id="layer1">
>
> So you can see why e.get('inkscape:label') cannot possibly work, and why
> e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.
> The xml.etree author obviously knew that this was cumbersome, and
> hence you can do something like:
>
> namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
> element = root.find('inkspace:foo', namespaces)
>
> which will work for both of the above pieces of XML.
use in my code rather than relying on the xml's generator to not change their
prefixes.
BTW, I only now thought to look at what actually is at Inkscape's namespace
URI, and it turns out to be quite a nice explanation of what a namespace is and
why it looks like a URL.
0 new messages