Does anyone know if the new WebDriver core will solve this problem for Selenium?
Also, can you specify single-window mode from the test script? Or must it be specified when you start Sel-RC Server? About 10% of my tests involve multiple windows but if I could specify it for my single window tests I'd still gain a significant performance improvement.
I've had this problem also for sometime and have just "put up with it" due to lack of time but I'd love to have a solution.
Jennifer Bevan <jennife...@gmail.com>
Sent by: selenium-...@googlegroups.com 12/11/2009 10:44 PM
|
|
--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To post to this group, send email to selenium-...@googlegroups.com.
To unsubscribe from this group, send email to selenium-develo...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/selenium-developers?hl=en.
Thanks Jen,
yes, I meant Sel-RC-Java 'test scripts' as actual test programs written in Java. I couldn't remember if those 'hooks' could be put in on a per-test bases.
tnx.
Jennifer Bevan <jennife...@gmail.com>
Sent by: selenium-...@googlegroups.com |
12/14/2009 05:00 PM
|
|
We've put the hooks in for in-test control of multi- vs. single-window mode into the Java RC client, and possibly (I'd have to check) the Python RC client, but you said "test script" -- are you using the HTML tests? If so, I believe you'd have to use the command-line flag "-singleWindow", at least for the time being.
As far as the Webdriver core's impact goes, the hypothesis is that it will help, although we do not yet know how much. The discussion so far has basically been to continue with the webdriver integration into Selenium 2.0 and then see what remains to be done to make the multi- vs. single- window performance more comparable.
-Jen
On Mon, Dec 14, 2009 at 2:36 AM, <PGran...@idc.com> wrote:
Sent by: selenium-...@googlegroups.com 12/11/2009 10:44 PM
|
|
yes, I meant Sel-RC-Java 'test scripts' as actual test programs written in Java. I couldn't remember if those 'hooks' could be put in on a per-test bases.
"Does anyone know if the new WebDriver core will solve this problem
for
Selenium? "
~ T
--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
Well, this is something that I am unable to wrap my head around & one
of the reasons to start this thread. If you see my comment with the
numbers I posted, simulated XPath using IE's javascript engine wasn' t
"that" slow when used directly. i.e. if I run javascript-xpath on an
HTML page directly (outside of Selenium) the numbers were rather
surprisingly fast. But inside Selenium it becomes exponentially slow.
And I asked this question before & hopefully some one can answer it -
why is it so slow inside Selenium - be it singleWindow or dual window?
Patrick
In the HTML page I include the javascript-xpath library & then I
evaluate XPath on page load using a function call.
As far as I know (at least based on the Perl driver's POD) there are 3
options that can be used - default, ajaxslt & javascript-xpath.
"default" option loads the Google's ajaxslt & javascript-xpath uses
Cybozu Labs' library. Google's ajaxslt is definitely slower than
javascript-xpath and so I did my testing with javascript-xpath. I used
the copy that is distributed with the RC & is located in core/xpath of
the slenium-server.jar file. BTW, it is the same version that can also
be found at the original author's site
http://svn.coderepos.org/share/lang/javascript/javascript-xpath/trunk/release/javascript-xpath-latest.js
(v0.1.11).
Patrick
I think I can do that, but I'll have to get to it next week - I am on
vacation from today afternoon.
I have a test page ready to demonstrate it. Where can I upload it?
Patrick
Filed the bug: http://code.google.com/p/selenium/issues/detail?id=307
If you need any further instructions, please let me know.
I took your test page to do a little test with a remote Selenium
Server with IE7 in single window mode and the Java client. This is the
code I used:
Selenium s = new DefaultSelenium("sel_host", 4445, "*iexploreproxy",
"http://test_host/test-files/test.html");
s.start();
long start = System.currentTimeMillis();
s.useXpathLibrary("javascript-xpath");
long end = System.currentTimeMillis();
System.out.println("Switched to javascript-xpath in " + (end -
start) + " ms.");
s.open("http://test_host/test-files/test.html");
s.waitForPageToLoad("20000");
start = System.currentTimeMillis();
int count = s.getXpathCount("//table[@class='tblList']/tbody/
tr").intValue();
end = System.currentTimeMillis();
System.out.println("Found " + count + " items in " + (end - start) +
" ms.");
s.stop();
This prints out:
Switched to javascript-xpath in 16 ms.
Found 17 items in 47 ms.
47 ms - 16 ms are 31 ms - that's the time that is needed for the xpath
evaluation. The xpath evaluation in Selenium is not just
document.evaluate, there is some other code run before (checking which
XPath engine to use) and after (converting the NodeList to a Node
array) this.
To run the xpath evaluation directly in the browser it takes 16 ms.
So, I do not see there so much room for improvement in single window
mode.
Sascha
> So, I do not see there so much room for improvement in single window
> mode.
>
Damn, I feel much better (well relatively). I missed that you were
testing this in single window mode and even though I did test it in
single window mode, my concern was more for the dual window (which is
the default). Can you run this same test in dual window mode for
comparison? May be I should've pointed it out in the bug I filed.
Just for reference here is my test, albeit in perl
use strict;
use warnings;
use Time::HiRes qw(sleep gettimeofday tv_interval);
use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;
use Data::Dumper;
my $sel = Test::WWW::Selenium->new( host => "localhost",
port => 4444,
browser => "*iexplore",
browser_url => "http://
localhost/" );
$sel->open_ok("/test.html");
$sel->use_xpath_library("javascript-xpath");
$start = [gettimeofday()];
print "XPath matches: ".$sel->get_xpath_count("//table
[\@class='tblList']/tbody/tr")."\n";
$time_spent = tv_interval($start)*1000;
print "Xpath: $time_spent [ms]\n";
And the output:
ok 1 - open, /test.html
XPath matches: 17
Xpath: 204.226 [ms]
1..1
So as you can see definitely worse than single window mode.
Sascha
I have to use multiWindow for my application. I want to help if you
guys are busy with other stuff. But, all I need is some document or
some primer on how the selenium server is desinged so that I get a
head start, more specifically what classes launch the browsers & how
the communication happens between these two browser windows. If you
can point me in that direction, I am willing to invest some time in
debugging this. I am not guaranteeing any results, but at least I can
try.
So, that brings up an interesting point - is there any way we can can
inject this javascript-xpath file to the test window? I think the
javascript-xpath is loaded only in the driver window. This hack would
definitely improve XPath performance a bit more.
Simon
How about letting the user decide that? In this case, for e.g.,
javascript-xpath does nothing for browsers which support DOM 3 XPath
natively and since it is just a script include tag, it does not alter
the behavior of the app under test in any way. If you look hard & long
enough, may be you can find some corner cases where this could
interfere with the app under test (may be namespace collision, which I
seriously doubt but just throwing it out there), but that is where we
let the user decide whether they want to include it or not for
performance reasons. And there is significantly noticeable
improvements in XPath processing if we do that.
function main() {
var expr = ".//tbody/tr";
var start_ = new Date();
var tblNode = document.evaluate('//table[@class=\'tblList\']',
document, null, 9, null).singleNodeValue;
for (var i=0; i<100; i++)
{
var result = document.evaluate('//table[@class=\'tblList\']/tbody/
tr', document, null, 6, null);
//var result = document.evaluate(expr, tblNode, null, 6, null);
}
alert('Found ' + result.snapshotLength + ' rows in ' + (new Date() -
start_) + ' [ms]');
};
I uncomment the respective for each run. So when I run the XPath
without using the table context node - document.evaluate('//table
[@class=\'tblList\']/tbody/tr', document, null, 6, null) the time for
execution to complete was 283ms but when I used the table context node
& used relative XPath document.evaluate(expr, tblNode, null, 6, null)
- the time dropped to 203ms.
The above test was of course done outside of Selenium but I am fairly
positive that we can get better improvements in XPath processing for
IE in Selenium (for multiWindow) if we can do two things :
1) Inject javascript-xpath in app under test window
2) Somehow facilitate context nodes usage for XPaths - this will be
very helpful especially if your pages are huge.
Xpathes that start with an ID selection are usually faster in IE as
Javascript XPath can use the native method getElementById to get it.
This should be something similar as the context node. The Selenium API
does not support the concept to return an Element reference to your
client (in comparison to WebDriver where you have your WebElement).
Sascha
On Jan 13, 1:27 am, Aditya Ivaturi <ivat...@gmail.com> wrote:
> Apart from injecting "javascript-xpath" in the target window, I think
> I might have stumbled on another way to squeeze out some more
> performance from the XPath processing on IE. I posted this on the
> users forumhttp://groups.google.com/group/selenium-users/browse_thread/thread/e5...
Even if we use ids for the element nodes, context nodes would still be
faster for those nodes with ids. But, I definitely understand that
architecture of Selenium is such that it doesn't provide reference
back to the client. So I guess, I am off to dig some more :).
I've been working on an alternative XPath engine over the past few
days, supported by refactoring some code in Core so that it (and other
future implementations) can be plugged in. The "new" engine delegates
to an existing engine (i.e. ajaxslt or javascript-xpath) for actual
evaluation; however, it has the following interesting characteristics,
specifically for optimizing multi-window mode by avoiding cross-window
RPC latency:
- It makes a mirror copy of the document in the remote window to a
frame in the local Selenium window. It uses this "test copy" to
construct an optimized expression for locating the desired element in
the remote window. For element counting, the remote window isn't even
consulted.
- It makes heavy use of caching, to avoid unnecessary document copies,
expression optimization, and actual evaluation. In particular, once
found, nodes are cached - until they become invalidated. Caches are
bounded in size.
- It uses jQuery for the purpose of parsing innerHTML content and
transforming it into DOM nodes for insertion into the mirror document.
Because we don't want to be evaluating application scripts in the
Selenium window, <script> elements are not included among the inserted
nodes. Therefore, this engine will not work for XPath's that look for
or include script element paths.
I'd like to welcome everyone to test out the new engine, which is NOT
turned-on by default. I don't have any extensive real world tests at
my disposal, so your numbers would be VERY informative for me -
especially in juxtaposition to numbers from using plain-old javascript-
xpath . If you pull the latest from the repo and build the Selenium
server (instructions here:
http://groups.google.com/group/selenium-developers/browse_thread/thread/c3e06311c9400145),
you should have it. But you will need to copy the engine
implementation file rpc-optimizing-user-extension.js and either add it
to the server server jar (I've been using
HAW BIN@HAWBIN-2b300780 /cygdrive/i/work/selenium-merged/common/src/js
$ jar uf ../../../build/selenium-server-standalone.jar core/scripts/
user-extensions.js
), or specify it with the -userExtensions switch.
Let me know what ya'll find out. :)
Haw-Bin
On Jan 13, 8:16 pm, Aditya Ivaturi <ivat...@gmail.com> wrote:
> If you guys haven't already lost patience :), here is another test
> that I did and its results -http://code.google.com/p/selenium/issues/detail?id=307#c8.
- What happens if the page uses ajax & constantly updates its data or
even layout? How, will this new change affect the XPath queries since
it is now going to be run against a copy? Let us say, if my test suite
includes first finding a button using XPath, then clicking it & then
find something else on that same page using XPath, how will the new
change handle that?
--aditya
On Jan 13, 8:34 pm, gyrm <hbc...@gmail.com> wrote:
> Thought I'd share ...
>
> I've been working on an alternative XPath engine over the past few
> days, supported by refactoring some code in Core so that it (and other
> future implementations) can be plugged in. The "new" engine delegates
> to an existing engine (i.e. ajaxslt or javascript-xpath) for actual
> evaluation; however, it has the following interesting characteristics,
> specifically for optimizing multi-window mode by avoiding cross-window
> RPC latency:
>
<snip>
1) Downloaded the latest source from http://selenium.googlecode.com/svn/trunk/.
I am at r8081.
2) In trunk run this command - rake selenium-server-standalone.
3) Copy common\src\js\core\scripts\rpc-optimizing-user-extension.js to
the build directory & rename it to user-extensions.js
4) java -jar selenium-server-standalone.jar -userExtensions user-
extensions.js.
5) I am using the same test page that I submitted to issue #307.
6) Here is my Perl test script:
<code>
use strict;
use warnings;
use Time::HiRes qw(sleep gettimeofday tv_interval);
use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;
#use Data::Dumper;
my $sel = Test::WWW::Selenium->new( host => "localhost",
port => 4444,
browser => "*iexplore",
browser_url => "http://
localhost/" );
my ($start, $end, $time_spent) = 0;
$sel->use_xpath_library("javascript-xpath");
$start = [gettimeofday()];
for (my $i=0; $i < 100; $i++)
{
$sel->get_xpath_count("//table[\@class='tblList'][1]/tbody/tr");
}
$time_spent = tv_interval($start)*1000;
print "Xpath: $time_spent [ms]\n";
$sel->close();
</code>
and here is my output:
ok 1 - open, /test.html
Xpath: 503481.153 [ms]
1..1
So it took almost 503 seconds to query "//table[\@class='tblList'][1]/
tbody/tr" 100 times. Am I doing some thing wrong?
Thanks for trying it out.
The extension doesn't replacement "javascript-xpath" - it actually
adds a new engine called "rpc-optimizing". So try to set that with
your call to use_xpath_library() .
As far as AJAX page updates, the engine doesn't permanently cache a
mirrored copy of the document. It always compares against the
innerHTML of the original*. If the markup in the original document
changes, it will update the mirrored copy - otherwise it won't. I am
happy to hear your experiences with AJAX changes, though, as I haven't
tested this myself yet.
Haw-Bin
*It's not a direct HTML-to-HTML comparison, because the copy may have
some <script> tags stripped out. Thus, the comparison goes through a
mapping from original to copy that is cached.
On Jan 14, 4:04 pm, Aditya Ivaturi <ivat...@gmail.com> wrote:
> Haw-Bin - may be you & I are doing some thing different because I got
> even horrible numbers with the new code. So let me list out what I am
> doing to verify that I am doing it right:
>
> 1) Downloaded the latest source fromhttp://selenium.googlecode.com/svn/trunk/.
On a side note, I also tried to offload my XPath query processing to
JavaScript directly using user-extensions and that allowed me to use
context nodes, which actually provides me performance close to or even
better than FF (with queries from Perl driver). But, you have to
realize that these are very specific set of XPath queries that are
specific to that page like the one with over 800 tables. So all in
all, I can mix & match these two solutions to gain a considerable
improvement in execution time on IE.
Obviously, it begs the question - will this new "rpc-optimizing"
option be available only through user-extensions?
I fully intend to fold rpc-optimizing into Core - I thought it would
be a good idea to vet it as an extension first. My only concern is
with the jQuery dependency - it should be somehow hidden from the
application under test, such that it doesn't conflict with or clobber
other instances of jQuery or any other js framework. I haven't done
any testing in this area yet, and am not sure what the best approach
might be.
Haw-Bin
Does this mean we would have jQuery /and/ prototype in a future core?
-adam
Right now I'm thinking of either extracting just the bits of jQuery
required for the limited functionality I need, OR creating some kind
of wrapper class that completely encapsulates the jQuery object, and
exposing a couple of methods that simply delegate to jQuery. Or maybe
some hybrid of these two approaches.
Haw-Bin
Haw-Bin
Haw-Bin