Complex, for me, xpath syntax

[James B. Byrne]

I am using Nokogiri with Cucumber to write some tests. I have the
situation where a pair of values should appear on a web page one
time, and exactly one time only. These values are contained in
adjacent cells of an xhtml table. I have created a working test by
injecting into the table rows a span have a dynamically constructed
class value that gives me the requisite results.

This is however an extremely brittle method of testing and I am
researching whether there is a more flexible way. Now, I note that
xpath has the function contains(s1,s2) which returns true iff s2 is
a sub-string of s1. So, I can pass s2 as the argument for the test,
the trick I cannot perform is how do I get the text of the
appropriate node into s1.

The xhtml construct is similar to this

<table>
<thead></thead>
<tfoot></tfoot>
<tbody>
<tr>
<td class="username">xyz</td>
<td class="rolename">abc</td>

an xpath selector like this should work, I think (this is in Ruby.

uname = "xyz"
rname = "abc"

//tr[contains('s1u,#{uname}')]/following-sibling::*[contains('s1r,#{rname}')]"


My problem is that I cannot see what, if anything, I can substitute
for s1u and s1r to hold the text values. I tried this:

s1u = "//td[@class='username']"
s1r = "//td[@class='rolename']"
lnode = "//tr[contains('#{s1u},#{uname}')]/" +
"following-sibling::*[contains('#{s1r},#{rname}')]"

have_xpath(lnode)

But, as I guessed, this does not work as I cannot figure out the
proper syntax to return the text value for s1u and s1r, if for no
other reason. In any case, I need a count of occurrences, not a
Boolean response if at least one occurrence is found.

My main use of xpath is an infrequent requirement for testing
purposes so I have the barest knowledge of what it is capable of. I
would greatly appreciate it if someone could show me how to do this,
if indeed what I ask can be done at all.

Sincerely,



--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:Byr...@Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3

[James B. Byrne]

On Thu, November 19, 2009 12:21, James B. Byrne wrote:

>
> This is however an extremely brittle method of testing and I am
> researching whether there is a more flexible way. Now, I note that
> xpath has the function contains(s1,s2) which returns true iff s2 is
> a sub-string of s1. So, I can pass s2 as the argument for the test,
> the trick I cannot perform is how do I get the text of the
> appropriate node into s1.
>

Based on suggestions from a colleague I have come up with this, but
it still throws a syntax error. Any other suggestions?

resonse.body <<-ENDDOC

<table>
<thead></thead>
<tfoot></tfoot>
<tbody>
<tr>
<td class="username">xyz</td>

<td class="rolename">ABC</td>
</tr>
</tbody>
</table>
ENDDOC

uname = "xyz"
rname = "ABC"

cnode = "//*[count(.[contains(.,'#{uname}')]" +
"/following-sibling::*[contains(.,'#{rname}')]) = 1]"

doc = Nokogiri::HTML(response.body)

if doc.xpath(cnode)
...

[Aaron Patterson]

How about this?

require 'nokogiri'

doc = Nokogiri::HTML(<<-ENDDOC)

<table>
<thead></thead>
<tfoot></tfoot>
<tbody>
<tr>
<td class="username">xyz</td>
<td class="rolename">ABC</td>
</tr>
</tbody>
</table>
ENDDOC

uname = "xyz"
rname = "ABC"

cnode = "//*[contains(.,'#{uname}') and " +
"following-sibling::*[contains(.,'#{rname}')]]"

p doc.xpath(cnode).length
p doc.xpath(cnode)

--
Aaron Patterson
http://tenderlovemaking.com/

[James B. Byrne]

On Fri, November 20, 2009 12:29, Aaron Patterson wrote:

>
> How about this?
>
>> require "nokogiri"

=> []

>> doc = Nokogiri::HTML(<<-ENDDOC)
<table>
<thead></thead>
<tfoot></tfoot>
<tbody>
<tr>
<td class="username">xyz</td>
<td class="rolename">ABC</td>
</tr>

<tr>
<td class="username">pqr</td>

<td class="rolename">ABC</td>
</tr>

<tr>
<td class="username">lmn</td>
<td class="rolename">DEF</td>
</tr>
</tbody>
</table>
ENDDOC
=> #<Nokogiri::HTML::Document:0x15b49ebdff20 name="document"
children=[#<Nokogiri::XML::DTD:0x15b49ebdc85c name="html">,
#<Nokogiri::XML::Element:0x15b49ebdc5dc name="html"
children=[#<Nokogiri::XML::Element:0x15b49ebd7028 name="body"
children=[#<Nokogiri::XML::Element:0x15b49ebd4d14 name="table"
children=[#<Nokogiri::XML::Element:0x15b49ebd3b30 name="thead">,
#<Nokogiri::XML::Element:0x15b49ebd3798 name="tfoot">,
#<Nokogiri::XML::Element:0x15b49ebd2c80 name="tbody"
children=[#<Nokogiri::XML::Element:0x15b49ebd145c name="tr"
children=[#<Nokogiri::XML::Element:0x15b49ebd08cc name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebcfad0 name="class"
value="username">] children=[#<Nokogiri::XML::Text:0x15b49ebcf260
"xyz">]>, #<Nokogiri::XML::Text:0x15b49ebcf094 "\n ">,
#<Nokogiri::XML::Element:0x15b49ebcf030 name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebce284 name="class"
value="rolename">] children=[#<Nokogiri::XML::Text:0x15b49ebcc6c8
"ABC">]>, #<Nokogiri::XML::Text:0x15b49ebcc344 "\n ">]>,
#<Nokogiri::XML::Element:0x15b49ebc9838 name="tr"
children=[#<Nokogiri::XML::Element:0x15b49ebc8ac8 name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebc899c name="class"
value="username">] children=[#<Nokogiri::XML::Text:0x15b49ebc6a0c
"pqr">]>, #<Nokogiri::XML::Text:0x15b49ebc574c "\n ">,
#<Nokogiri::XML::Element:0x15b49ebc47ac name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebc3d98 name="class"
value="rolename">] children=[#<Nokogiri::XML::Text:0x15b49ebbd09c
"ABC">]>, #<Nokogiri::XML::Text:0x15b49ebbc9f8 "\n ">]>,
#<Nokogiri::XML::Element:0x15b49ebbc804 name="tr"
children=[#<Nokogiri::XML::Element:0x15b49ebbbc9c name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebbb9cc name="class"
value="username">] children=[#<Nokogiri::XML::Text:0x15b49ebb9f50
"lmn">]>, #<Nokogiri::XML::Text:0x15b49ebb9398 "\n ">,
#<Nokogiri::XML::Element:0x15b49ebb9334 name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebb91b8 name="class"
value="rolename">] children=[#<Nokogiri::XML::Text:0x15b49ebb8510
"DEF">]>, #<Nokogiri::XML::Text:0x15b49ebb83a8 "\n ">]>]>]>]>]>]>
>>
?> uname = "xyz"

=> "xyz"
>> rname = "ABC"

=> "ABC"
>>
?> cnode = "//*[contains(.,'#{uname}') and " +
?> "following-sibling::*[contains(.,'#{rname}')]]"
=> "//*[contains(.,'xyz') and following-sibling::*[contains(.,'ABC')]]"
>>
?> p doc.xpath(cnode).length
2
=> nil
>> p doc.xpath(cnode)
[#<Nokogiri::XML::Element:0x15b49ebd145c name="tr"
children=[#<Nokogiri::XML::Element:0x15b49ebd08cc name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebcfad0 name="class"
value="username">] children=[#<Nokogiri::XML::Text:0x15b49ebcf260
"xyz">]>, #<Nokogiri::XML::Text:0x15b49ebcf094 "\n ">,
#<Nokogiri::XML::Element:0x15b49ebcf030 name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebce284 name="class"
value="rolename">] children=[#<Nokogiri::XML::Text:0x15b49ebcc6c8
"ABC">]>, #<Nokogiri::XML::Text:0x15b49ebcc344 "\n ">]>,
#<Nokogiri::XML::Element:0x15b49ebd08cc name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebcfad0 name="class"
value="username">] children=[#<Nokogiri::XML::Text:0x15b49ebcf260
"xyz">]>]
=> nil

At least this does not throw a syntax error. But now I am confused
as to what it is doing. The node length reported is 2, which I
infer from the content displayed refers to the number of occurrences
of

... name="class" value="username">] children
[#<Nokogiri::XML::Text:0x15b49ebcf260 "xyz" ...

But I am looking for pairs of uname + rname. Whatever XPath
expression I end up with, with this source document I would expect
to see a value of 1 instead of 2.

Help?

[James B. Byrne]

I extracted the node string and reformatted it into something that I
could parse the structure from.

[#<Nokogiri::XML::Element:0x15b49ebd145c name="tr"
children=[#<Nokogiri::XML::Element:0x15b49ebd08cc
name="td" attributes=[#<Nokogiri::XML::Attr:0x15b49ebcfad0
name="class" value="username">]
children=[#<Nokogiri::XML::Text:0x15b49ebcf260 "xyz">]>,
#<Nokogiri::XML::Text:0x15b49ebcf094 "\n ">,

#<Nokogiri::XML::Element:0x15b49ebcf030 name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebce284 name="class"
value="rolename">]
children=[#<Nokogiri::XML::Text:0x15b49ebcc6c8 "ABC">]>,
#<Nokogiri::XML::Text:0x15b49ebcc344 "\n ">]>,

#<Nokogiri::XML::Element:0x15b49ebd08cc name="td"
attributes=[#<Nokogiri::XML::Attr:0x15b49ebcfad0 name="class"
value="username">]
children=[#<Nokogiri::XML::Text:0x15b49ebcf260 "xyz">]>]

So, what I see here is a 'tr' node containing the two dependent 'td'
nodes that match the criteria. I infer then that the length method
refers to these two siblings that were returned. If true, then the
inference one draws is that node.length will always equal (# of
matches found multiplied by the number of criteria nodes given).

Is this correct?

[James B. Byrne]

On Fri, November 20, 2009 15:15, James B. Byrne wrote:

> So, what I see here is a 'tr' node containing the two dependent 'td'
> nodes that match the criteria. I infer then that the length method
> refers to these two siblings that were returned. If true, then the
> inference one draws is that node.length will always equal (# of
> matches found multiplied by the number of criteria nodes given).
>
> Is this correct?

No, it is not. This node set will contain all of the descendant
bits of the parents whose children match the criteria. So, the
problem is how to count the parents.

[James B. Byrne]

Given this:

doc = Nokogiri::HTML(<<-ENDDOC)
<table>
<thead></thead>
<tfoot></tfoot>
<tbody>
<tr>
<td class="username">xyz</td>
<td class="rolename">ABC</td>

<td class="noname">in</td>
<td class="somename">out</td>
<td class="noname">xray</td>
</tr>
...

uname = "xyz"
rname = "ABC"
xname = "xray"

cnode = "//*[contains(.,'#{uname}') and " +

"following-sibling::*[contains(.,'#{rname}')] and " +
"following-sibling::*[contains(.,'#{xname}')]]"

Then
p doc.xpath(cnode).length => 2

p doc.xpath(cnode)
=> (formatted along [] pairs)
[#<Nokogiri::XML::Element:0x15b49e8623ac name="tr" children=
[#<Nokogiri::XML::Element:0x15b49e86212c name="td" attributes=
[#<Nokogiri::XML::Attr:0x15b49e862050 name="class"
value="username">] children=
[#<Nokogiri::XML::Text:0x15b49e861c18 "xyz">]>,
#<Nokogiri::XML::Text:0x15b49e861a9c "\n ">,
#<Nokogiri::XML::Element:0x15b49e861a4c name="td" attributes=
[#<Nokogiri::XML::Attr:0x15b49e8618f8 name="class"
value="rolename">] children=
[#<Nokogiri::XML::Text:0x15b49e8614c0 "ABC">]>,
#<Nokogiri::XML::Text:0x15b49e861358 "\n ">,
#<Nokogiri::XML::Element:0x15b49e861308 name="td" attributes=
[#<Nokogiri::XML::Attr:0x15b49e8611b4 name="class"
value="noname">] children=
[#<Nokogiri::XML::Text:0x15b49e860d7c "in">]>,
#<Nokogiri::XML::Text:0x15b49e860c14 "\n ">,
#<Nokogiri::XML::Element:0x15b49e860bc4 name="td" attributes=
[#<Nokogiri::XML::Attr:0x15b49e860a70 name="class"
value="somename">] children=
[#<Nokogiri::XML::Text:0x15b49e860638 "out">]>,
#<Nokogiri::XML::Text:0x15b49e8604d0 "\n ">,
#<Nokogiri::XML::Element:0x15b49e860480 name="td" attributes=
[#<Nokogiri::XML::Attr:0x15b49e86032c name="class"
value="noname">] children=
[#<Nokogiri::XML::Text:0x15b49e85fef4 "xray">]>,
#<Nokogiri::XML::Text:0x15b49e85fd8c "\n ">
]>,
#<Nokogiri::XML::Element:0x15b49e86212c name="td" attributes=
[#<Nokogiri::XML::Attr:0x15b49e862050 name="class"
value="username">] children=
[#<Nokogiri::XML::Text:0x15b49e861c18 "xyz">]>
]

I cannot fathom why I am seeing two 'td' elements with a value of
'xyz', one with all of its siblings and one without, all within a
'tr' location.

[Aaron Patterson]

Ah, I think I see now. For username, role name pairs, I would do
something like this:

doc = Nokogiri::HTML(<<-ENDDOC)
<table>
<thead></thead>
<tfoot></tfoot>
<tbody>
<tr>
<td class="username">xyz</td>
<td class="rolename">ABC</td>
</tr>
<tr>
<td class="username">pqr</td>
<td class="rolename">ABC</td>
</tr>
<tr>
<td class="username">lmn</td>
<td class="rolename">DEF</td>
</tr>
</tbody>
</table>
ENDDOC

p doc.css('.username').map { |x|
x.parent.children.find_all { |n| !n.blank? }.map { |y|
y.text
}
}

I don't know that xpath would be the perfect solution for this, but
the ruby code above should get you going.

[James B. Byrne]

Frankly, the approach I was attempting to take here was suggested to
me as an alternative to what I was doing: which was simply
generating a special id value made up of the concatenated values of
the values of interest. They suggested that the example I showed to
begin with would prove more flexible. After tussling with this for
a weekend despite your most generous advice, I came to the
conclusion that the game just was not worth the candle.

So, I have returned to my original formulation which, if brittle, at
least possesses the virtues of simplicity, clarity, and
effectiveness.

A useful exercise nonetheless. it will no doubt save me a lot of
grief in the future to have learned that //* returns both the
descendant AND self nodes. Which is perfectly clear if one uses
this construct without predicates since it must be so if it is to
return every node.

Thank you for your assistance.