regex help

Gabriel Rossetti

unread,

Dec 16, 2009, 12:16:45 PM12/16/09

to pytho...@python.org

Hello everyone,

I'm going nuts with some regex, could someone please show me what I'm
doing wrong?

I have an XMPP msg :

the <parameter> node may be absent or empty (<parameter/>), the <x> node
may be absent. I'd like to grab everything exept the <payload> nod and
create something new using regex, with the XMPP message example above
I'd get this :

for some reason my regex doesn't work correctly :

r"(<message .*?>).*?(<mynode
.*?>).*?(?:(<parameters>.*?</parameters>)|<parameters/>)?.*?(<x .*/>)?"

I group the opening <message> node, the opening <mynode> node and if the
<parameters> node is present and not empty I group it and if the <x>
node is present I group it. For some reason this doesn't work correctly :

>>> import re
>>> s1 = "<message xmlns='jabber:client' to='no...@host.com'><mynode
xmlns='myprotocol:core' version='1.0'
type='mytype'><parameters><param1>123</param1><param2>456</param2></parameters><payload
type='plain'>...</payload></mynode><x xmlns='jabber:x:expire'
seconds='15'/></message>"
>>> s2 = "<message xmlns='jabber:client' to='no...@host.com'><mynode
xmlns='myprotocol:core' version='1.0'
type='mytype'><parameters/><payload
type='plain'>...</payload></mynode><x xmlns='jabber:x:expire'
seconds='15'/></message>"
>>> s3 = "<message xmlns='jabber:client' to='no...@host.com'><mynode
xmlns='myprotocol:core' version='1.0' type='mytype'><payload
type='plain'>...</payload></mynode><x xmlns='jabber:x:expire'
seconds='15'/></message>"
>>> s4 = "<message xmlns='jabber:client' to='no...@host.com'><mynode
xmlns='myprotocol:core' version='1.0'
type='mytype'><parameters><param1>123</param1><param2>456</param2></parameters><payload
type='plain'>...</payload></mynode></message>"
>>> s5 = "<message xmlns='jabber:client' to='no...@host.com'><mynode
xmlns='myprotocol:core' version='1.0'
type='mytype'><parameters/><payload
type='plain'>...</payload></mynode></message>"
>>> s6 = "<message xmlns='jabber:client' to='no...@host.com'><mynode
xmlns='myprotocol:core' version='1.0' type='mytype'><payload
type='plain'>...</payload></mynode></message>"
>>> exp = r"(<message .*?>).*?(<mynode
.*?>).*?(?:(<parameters>.*?</parameters>)|<parameters/>)?.*?(<x .*/>)?"
>>>
>>> re.match(exp, s1).groups()
("<message xmlns='jabber:client' to='no...@host.com'>", "<mynode
xmlns='myprotocol:core' version='1.0' type='mytype'>",
'<parameters><param1>123</param1><param2>456</param2></parameters>', None)
>>>
>>> re.match(exp, s2).groups()
("<message xmlns='jabber:client' to='no...@host.com'>", "<mynode
xmlns='myprotocol:core' version='1.0' type='mytype'>", None, None)
>>>
>>> re.match(exp, s3).groups()
("<message xmlns='jabber:client' to='no...@host.com'>", "<mynode
xmlns='myprotocol:core' version='1.0' type='mytype'>", None, None)
>>>
>>> re.match(exp, s4).groups()
("<message xmlns='jabber:client' to='no...@host.com'>", "<mynode
xmlns='myprotocol:core' version='1.0' type='mytype'>",
'<parameters><param1>123</param1><param2>456</param2></parameters>', None)
>>>
>>> re.match(exp, s5).groups()
("<message xmlns='jabber:client' to='no...@host.com'>", "<mynode
xmlns='myprotocol:core' version='1.0' type='mytype'>", None, None)
>>>
>>> re.match(exp, s6).groups()
("<message xmlns='jabber:client' to='no...@host.com'>", "<mynode
xmlns='myprotocol:core' version='1.0' type='mytype'>", None, None)
>>>

Does someone know what is wrong with my expression? Thank you, Gabriel

r0g

unread,

Dec 16, 2009, 12:22:25 PM12/16/09

to

Gabriel Rossetti wrote:
> Hello everyone,
>
> I'm going nuts with some regex, could someone please show me what I'm
> doing wrong?
>
> I have an XMPP msg :
>

<snip>

>
>
> Does someone know what is wrong with my expression? Thank you, Gabriel

Gabriel, trying to debug a long regex in situ can be a nightmare however
the following technique always works for me...

Use the interactive interpreter and see if half the regex works, if it
does your problem is in the second half, if not it's in the first so try
the first half of that and so on an so forth. You'll find the point at
which it goes wrong in a snip.

Non-trivial regexes are always best built up and tested a bit at a time,
the interactive interpreter is great for this.

Roger.

Intchanter / Daniel Fackrell

unread,

Dec 16, 2009, 12:38:42 PM12/16/09

to

I'll just add that the "now you have two problems" quip applies here,
especially when there are very good XML parsing libraries for Python
that will keep you from having to reinvent the wheel for every little
change.

See sections 20.5 through 20.13 of the Python Documentation for
several built-in options, and I'm sure there are many community
projects that may fit the bill if none of those happen to.

Personally, I consider regular expressions of any substantial length
and complexity to be bad practice as it inhibits readability and
maintainability. They are also decidedly non-Zen on at least
"Readability counts" and "Sparse is better than dense".

Intchanter
Daniel Fackrell

P.S. I'm not sure how any of these libraries are implemented yet, but
I'd hope they're using a finite state machine tailored to the parsing
task rather than using regexes, but even if they do the latter, having
that abstracted out in a mature library with a clean interface is
still a huge win.