JQuery compatibility?

[Jeroen van Dijk]

Hi Aaron et al,

I'm using nokogiri through webrat and I'm very happy about it.

Except for one thing. When I need to do a complex element query the
css selectors are not enough and I have to go down to the xpath route.
IMO xpath is harder to grasp then the css selectors. So I would like
to see the following pseudo selectors were added (following jQuery):

:contains('test') Selects all elements which contain the specified
text.
:has

How hard would this be and where would you suggest me to start?

It would be great if it could become part of nokogiri itself, but I'm
also happy to apply it through a separate gem.

Cheers,
Jeroen

[Mike Dalessio]

Jeroen,

Because Nokogiri is standards-compliant in both its xpath() and css() queries, we're not really interested in changing that behavior to support non-standard syntax. But I personally have no objection to a jquery() method that accepts an extended jquery syntax.

Currently XPath queries are parsed and interpreted by libxml2 without any intervention from Nokogiri, so this change isn't a simple extension of an existing parser.

That said, Nokogiri does support a visitor pattern within the syntax parsing system, used to interpret (or "pre-compile") query strings (which is how CSS queries are translated into XPath).

Nokogiri previously had a "compatibility layer" for Hpricot that did something very similar (since Hpricot had its own particular brand of query syntax). You should be able to check out what it looked like by reviewing the commit wherein the Hpricot-compat code was removed: 888b667ce46ad451d9eb8668965d900fef956b08

Let us know if you have more questions. I'd be interested in helping out.

-m

--

You received this message because you are subscribed to the Google Groups "nokogiri-talk" group.
To post to this group, send email to nokogi...@googlegroups.com.
To unsubscribe from this group, send email to nokogiri-tal...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nokogiri-talk?hl=.

--
mike dalessio
mi...@csa.net

[Jeroen van Dijk]

Thanks Mike. I'll look into the Hpricot code and see if I can figure
it out.

I'll post here when I have some progress and/or questions.

Jeroen

> > nokogiri-tal...@googlegroups.com<nokogiri-talk%2Bunsu...@googlegroups.com>
> > .

[Aaron Patterson]

On Fri, Nov 20, 2009 at 11:46 AM, Jeroen van Dijk
<jeroentj...@gmail.com> wrote:

> Hi Aaron et al,
>
> I'm using nokogiri through webrat and I'm very happy about it.
>
> Except for one thing. When I need to do a complex element query the
> css selectors are not enough and I have to go down to the xpath route.
> IMO xpath is harder to grasp then the css selectors. So I would like
> to see the following pseudo selectors were added (following jQuery):
>
> :contains('test') Selects all elements which contain the specified
> text.

contains() already works.

> :has

This should be fairly easy to implement (I think).

--
Aaron Patterson
http://tenderlovemaking.com/

[Jeroen van Dijk]

On Nov 23, 6:35 pm, Aaron Patterson <aaron.patter...@gmail.com> wrote:
> On Fri, Nov 20, 2009 at 11:46 AM, Jeroen van Dijk
>

> <jeroentjevand...@gmail.com> wrote:
> > Hi Aaron et al,
>
> > I'm using nokogiri through webrat and I'm very happy about it.
>
> > Except for one thing. When I need to do a complex element query the
> > css selectors are not enough and I have to go down to the xpath route.
> > IMO xpath is harder to grasp then the css selectors. So I would like
> > to see the following pseudo selectors were added (following jQuery):
>
> > :contains('test') Selects all elements which contain the specified
> > text.
>
> contains() already works.
>
> > :has
>
> This should be fairly easy to implement (I think).
>

I think I sort of have it working for the very simple case. So yes it
easy after staring at the code a few hours :)

However I didn't get nested function calls working. In the following
gist you see a simple hack to implement the :has function:
http://gist.github.com/241761
Note that I haven't actually checked if the xpaths are correct that is
something for the next phase. I first want to convert css selectors to
xpath without errors.

In the end I want to be able to do something like:
"li.date fieldset:has(legend label:contains('some label')) ol li
select"

So in this case the :has selector should be able to have nested
queries. Can you give an hint on where to continue? I don't think I
understand completely what the keywords :FUNCTION etc. do precisely. I
know I have to do something with nodes, but which keyword will cause
the nested node to be evaluated as a full selector?

The supported selector :not() does also not supported nesting (e.g.
"div:not(legend:not(span)) so I'm not sure if the architecture
actually allows nesting.

Looking forward to some hints.

Cheers,

Jeroen




> --
> Aaron Pattersonhttp://tenderlovemaking.com/

[Jeroen van Dijk]

I think I am able to solve the problem wasn't it for the issue below:

The generated tokenizer does not support nested queries (I think)

When I convert the following css selector "div:has(div.some_class)"

I get the error:

Nokogiri::CSS::SyntaxError: unexpected '.' after 'div'
/Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css/parser.rb:
78:in `on_error': using unsupported selectors - should be able to
convert "div:has(div.some_class)"
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/
1.8/racc/parser.rb:99:in `_racc_do_parse_c'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/
1.8/racc/parser.rb:99:in `__send__'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/
1.8/racc/parser.rb:99:in `do_parse'
/Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css/
generated_tokenizer.rb:30:in `parse'
/Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css/parser.rb:
70:in `xpath_for'
/Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css.rb:21:in
`xpath_for

Is there a way to overcome this?

Should I extend the parser?

Cheers Jeroen




On Nov 24, 10:40 am, Jeroen van Dijk <jeroentjevand...@gmail.com>

[Aaron Patterson]

On Wed, Nov 25, 2009 at 2:53 AM, Jeroen van Dijk
<jeroentj...@gmail.com> wrote:
> I think I am able to solve the problem wasn't it for the issue below:
>
> The generated tokenizer does not support nested queries (I think)
>
> When I convert the following css selector "div:has(div.some_class)"
>
> I get the error:
>
> Nokogiri::CSS::SyntaxError: unexpected '.' after 'div'
> /Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css/parser.rb:
> 78:in `on_error': using unsupported selectors - should be able to
> convert "div:has(div.some_class)"
>        /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/
> 1.8/racc/parser.rb:99:in `_racc_do_parse_c'
>        /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/
> 1.8/racc/parser.rb:99:in `__send__'
>        /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/
> 1.8/racc/parser.rb:99:in `do_parse'
>        /Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css/
> generated_tokenizer.rb:30:in `parse'
>        /Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css/parser.rb:
> 70:in `xpath_for'
>        /Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/css.rb:21:in
> `xpath_for
>
> Is there a way to overcome this?
>
> Should I extend the parser?

I think to properly support this, the yacc grammar needs to be
changed. Basically, we need to detect the :has() function and tell
the yacc grammar that a selector is allowed inside that function.

[Jeroen van Dijk]

Hi Aaron,

I saw that you added support for the :has selector. Thanks! It seems
to work :-)

I have been busy too. Since I am not too familiar with the Nokogiri
code I started with a JQuery-Nokogiri compatibility testsuite to see
what needs to be done.

If you run the following commands:

git clone git://github.com/jeroenvandijk/nokogiri_jquery.git
cd nokogiri_jquery
script/server -e cucumber
cucumber

You should be able to see the differences between JQuery and Nokogiri
selectors. It runs JQuery 1.3.2 through HtmlUnit and compares it with
the result of Nokogiri.

I think it would be nice to have at least resolved:
- support for nested selectors in the :not selector, similar to :has
(I think I'm able to do this with your :has example)
- support for :odd, :even, :lt, :empty, :hidden, :visible and the
others I have not yet specified in the feature (I can do this)

The testsuite also indicated some incompatibilities that are more
subtle:
- :eq, :gt, use a different index than JQuery. Nokogiri's index goes
from 1 to n while JQuery goes from 0 to n
- :last seems to be tricked by nested elements i.e. it return the last
element from a nested element instead of the last element in the
document
- li[name!='some value'] only returns something when there is a li
with an name other than 'some value'. JQuery also returns elements
that have no name attribute at all

I couldn't find a CSS3 specification that tells me whether either
JQuery or Nokogiri is wrong. What do you think?

Cheers, Jeroen

[Jeroen van Dijk]

PS.
If you are on OS X you also need to have the gem rjb installed:

sudo env JAVA_HOME=/Library/Java/Home gem install rjb

On Dec 1, 11:08 am, Jeroen van Dijk <jeroentjevand...@gmail.com>

[Aaron Patterson]

CSS spec seems to use 1 based indexes:

http://www.w3.org/TR/css3-selectors/#first-child-pseudo

So we did too.

> - :last seems to be tricked by nested elements i.e. it return the last
> element from a nested element instead of the last element in the
> document

This just sounds like a bug. Can you write up a short test to
illustrate and file it in the tracker?

> - li[name!='some value'] only returns something when there is a li
> with an name other than 'some value'. JQuery also returns elements
> that have no name attribute at all

!= is not defined in the spec, so I would lean toward JQuery behavior.
Again, file a ticket with a test, and I'll make it happen.

> I couldn't find a CSS3 specification that tells me whether either
> JQuery or Nokogiri is wrong. What do you think?

I'm pretty happy to go with JQuery behavior. I'm not sure what to do
about the gt() lt() indexes since it seems like the browser does 1
based pseudo selector indexes. I would consider changing it to work
like JQuery, but that would have to be a large release number since it
would break backward compatibility.

[Mike Dalessio]

This seems like a fundamental-enough difference that we might want to consider having a separate method for jQuery-compatible syntax. If we did that, css() would support only the CSS spec, and jquery() would support full jQuery synax along with the different index convention.

--
mike dalessio
mi...@csa.net

[Aaron Patterson]

On Thu, Dec 03, 2009 at 03:04:44PM -0500, Mike Dalessio wrote:
> On Thu, Dec 3, 2009 at 11:57 AM, Aaron Patterson
> <aaron.p...@gmail.com>wrote:

[snip]

> > I'm pretty happy to go with JQuery behavior. I'm not sure what to do
> > about the gt() lt() indexes since it seems like the browser does 1
> > based pseudo selector indexes. I would consider changing it to work
> > like JQuery, but that would have to be a large release number since it
> > would break backward compatibility.
> >
>
> This seems like a fundamental-enough difference that we might want to
> consider having a separate method for jQuery-compatible syntax. If we did
> that, css() would support only the CSS spec, and jquery() would support full
> jQuery synax along with the different index convention.

We already support extensions to css in the css method. We would have
to remove them in order to make it CSS spec only. That doesn't sound
good. I think we could support everything except the different index
numbers, then maybe make a jquery() method that matched exactly?

Thoughts?

[Mike Dalessio]

2009/12/4 Aaron Patterson <aa...@tenderlovemaking.com>

I don't have strong arguments against this, as long as we've got some descriptive test coverage.
 

--
mike dalessio
mi...@csa.net

[Aaron Patterson]

Ok. Jeroen, can you file tickets, one for each of the selector
differences between JQuery and Nokogiri? We won't be able to resolve
all of the differences for 1.4.1, but if we have tickets for them, we
can work on them for the next release.

In each ticket, links to the jquery docs would be good, failing tests
would be even better.