Watir is slow with nested frames

[JMI]

I examine the possibility of using Watir to automate my AUT which uses nested frames. To my deception I found that Watir is very slow to act on elements that are located in deeply nested frames. For example, to set a text field within the seventh frame, Watir needs some 15 seconds, while to set the same field within the first frame it needs only one second. This is very suprising to me, especially that a VB script that I created to do exactly the same is fast - about one second, independently of how deep the frame is nested.

 

Why Watir is slow with nested frames? Is this a known issue? Does a solution to this exist?

 

I use Ruby 1.8.7, Watir 1.9.2, IE8

　

　

　

[JMI]

And I join the demo files to show the issue. Copy the html files to a folder, open 7.html and run test_frame.rb

[Jarmo Pertman]

I'd ask why in the hell you've created such a website in the first
place? :P

Can't say for sure though why it's slower than with VB. This would
need some debugging to do.

Jarmo

On Jul 28, 5:53 pm, JMI <jurek.michal...@bluewin.ch> wrote:
> And I join the demo files to show the issue. Copy the html files to a
> folder, open 7.html and run test_frame.rb
>

>  0.html
> < 1KViewDownload
>
>  1.html
> < 1KViewDownload
>
>  2.html
> < 1KViewDownload
>
>  3.html
> < 1KViewDownload
>
>  4.html
> < 1KViewDownload
>
>  5.html
> < 1KViewDownload
>
>  6.html
> < 1KViewDownload
>
>  7.html
> < 1KViewDownload
>
>  text.html
> < 1KViewDownload
>
>  test_frame.rb
> < 1KViewDownload

[JMI]

>I'd ask why in the hell you've created such a website in the first
>place? :P

My AUT is not a website, it's a complex application that is deployed via an HTML server and can be accessed by thin clients. It is not my creation, it is my reality! The html pages I attached to this thread have been created just to illustrate the problem.

 

>Can't say for sure though why it's slower than with VB. This would
>need some debugging to do.

By digging thought the Watir surce code, (input_elements.rb) I found that the method set(value) of the class TextField invokes three other methods before actually setting the value of the field: assert_exists, assert_enabled and assert_not_readonly (very good!). There exists another method in the same class that does almost the same: value=(v). It sets the value of the field (as set(value) does), but does not call neither assert_enabled nor assert_not_readonly. And it is fast! (To try this, replace .set('123456789' by .value = '123456789' in my demo code). So now the question is: why the methods assert_enabled and assert_not_readonly (one of them or both) are slow for elements located within nested frames! (And yes, in VB, equivalents of those methods are fast).

 

To be continued...

 

 

[Jarmo Pertman]

Using #value= also does not trigger JavaScript events. That could be
also the reason. You could add timing statements around those
invocations to get the idea about the slowness most easily like this:
t = Time.now
assert_enabled
puts Time.now - t # how many seconds did it take time

Just add those statements in the source code and you'll get some idea,
what is the slow part. If you'd like more complex and thorough
solution, then you can use some profilers.

Jarmo

[JMI]

I am back from my hollidays and back to this performance problem.

 

I used a debugger (Eclipse) and the profiler (require 'profile') and found out that the problem comes from the method assert_exists of the class Element (in element.rb):

 

def assert_exists

  locate if respond_to?(:locate)

  unless ole_object

    raise UnknownObjectException.new(

                Watir::Exception.message_for_unable_to_locate(@how, @what))
end

The problem is that this method is called again and again by many methods, also by those called by assert_exists! As a result, for my simple example of 7 nested frames, this method was called 1536 times! The same elements were located again and again with exactly the same criteria (@how, @what and @container)! I modified the method assert_exists by adding four class variables that store the three location criteria and the OLE object found. Next time when the method assert_exists is called, the new criteria are compared with the previous ones. If they are the same, no need to 'locate' again, simply return the object found previously. After my modification, the method assert_exists is called only 17 times (always with my simple example), and the real location is done only 8 times (7 frames and one text field) - other 9 calls return the previously located objects. And everything takes less than 1 second (compared to 15 seconds before!).

Here is the modified method:

 

    def assert_exists
      if respond_to?(:locate)
        if (defined?(@@how) && defined?(@@what) && defined?(@@container) && defined?(@@o) &&
            @@how == @how && @@what == @what && @@container == @container)
          # The element has already been located -> take the already located OLE object
          @o = @@o

        else
          # Locate the element...
          locate
          # ...and save the information that served to locate it...
          @@how = @how
          @@what = @what
          @@container = @container
          # ...and also the located OLE object.
          @@o = @o
        end
      end
      unless ole_object
        raise UnknownObjectException.new(
                Watir::Exception.message_for_unable_to_locate(@how, @what))
      end
    end

Question:

As I am new to Watir (and Ruby), maybe I have ommited an important functionnality that may be broken by my modification.

Is my modification safe?

[Jarmo Pertman]

On Aug 22, 4:41 pm, JMI <jurek.michal...@bluewin.ch> wrote:
> Is my modification safe?

No. If you'd navigate away from that page and would like to locate an
element with the same locators on some other page then you'd fail.
There are some other approaches how to speed that up, but i'd like to
investigate why you're getting so many method executions in the first
place.

Jarmo

[JMI]

>> Is my modification safe?
>
>No. If you'd navigate away from that page and would like to locate an
>element with the same locators on some other page then you'd fail.

If I navigate away to another page, at least @container will not be the same, so the method 'locate' will be called and will do the full job.  So no, I would not fail.

 

>There are some other approaches how to speed that up, but i'd like to
>investigate why you're getting so many method executions in the first
>place.

You can use the HTML pages I posted at the beginning of this thread.

 

Jurek

[Jarmo Pertman]

On Aug 25, 12:49 pm, JMI <jurek.michal...@bluewin.ch> wrote:
> >> Is my modification safe?
>
> >No. If you'd navigate away from that page and would like to locate an
> >element with the same locators on some other page then you'd fail.
>
> If I navigate away to another page, at least @container will not be the
> same, so the method 'locate' will be called and will do the full job.  So
> no, I would not fail.

Oh, right. But then there's this case where you've located that DOM
element and it will be removed by JavaScript or what-not and added
back new DOM element with the same locators. I'm afraid that in this
case you won't find that element. Not sure though.

>
> >There are some other approaches how to speed that up, but i'd like to
> >investigate why you're getting so many method executions in the first
> >place.
>
> You can use the HTML pages I posted at the beginning of this thread.

Okay, will do that soon.

>
> Jurek

Jarmo

[Jarmo Pertman]

Okay, i have looked into this issue now. I've set a breakpoint just
before the #text_field invocation and didn't see as many invocations
as you did (didn't count them, but it was something in the lines of ~5
times). I don't understand where did you get that number 1536.

Anyway, i looked at the code and saw that everything that can be done
is already done. All caches are dangerous if not implemented properly
and your patch breaks the point of assert_exists:
1) the element is located at the first place
2) the element is removed from the DOM, e.g. assert_exists should fail
3) your patch returns the previously located element, e.g. the
assert_exists doesn't fail

Also, this makes #exists? also return "true" although it should return
"false".

In conclusion, i'd still say that the application which uses 7 nested
(!!!!) frames is really badly made and should be rewritten anyway. No
offence. So, it is just one really bad corner-case where it's slower
than usual, but unfortunately i don't see any good solution at this
moment how to solve it for everyone. I guess you can use #value for
that specific case, but i'm pretty sure you will also have slower
invocations with other methods.

Also, one solution would be to open that frame directly in your
browser instead of accessing it through all the other 6 frames.
Something like this with this concrete example or just create some
helper module, which could generate the correct url:
my_frame=ie.frame(:name, '7.6').frame(:name, '6.5').frame(:name,
'5.4').frame(:name, '4.3').frame(:name, '3.2').frame(:name,
'2.1').frame(:name, '1.0')
ie.goto ie.url.gsub(/\d+\.html$/, my_frame.src)
ie.text_field(:name, 'edit_box').set('123456789')

Jarmo

[Alister Scott]

+1 to working with the developers to get rid of 7 nested frames. That is ridiculous.

[JMI]

Le samedi 27 août 2011 14:55:20 UTC+2, Jarmo Pertman a écrit :

Okay, i have looked into this issue now. I've set a breakpoint just
before the #text_field invocation and didn't see as many invocations
as you did (didn't count them, but it was something in the lines of ~5
times). I don't understand where did you get that number 1536.

I got this number from the ruby profiler by adding requre 'profile' at the beginning of test_frame.rb.

When I count the calls of assert_exists programmatically, by adding $x+=1 in the body of assert_exists and puts $x at the end of the program, I get 512 (obviously the profiler counts three times too many).

But still this is much too many! You can also add puts #{@what} to the body of assert_exists to see exactly for which elements this method is called and recalled before the actual set is done.

Anyway, i looked at the code and saw that everything that can be done
is already done.

I am not so pessimistic.

 

All caches are dangerous if not implemented properly

I agree

 

and your patch breaks the point of assert_exists:
1) the element is located at the first place
2) the element is removed from the DOM, e.g. assert_exists should fail
3) your patch returns the previously located element, e.g. the
assert_exists doesn't fail

Also, this makes #exists? also return "true" although it should return
"false".

Two responses:

1) For cases with no frames: Each time assert_exists is called for a given element by the method set (or any other method that interacts with this element), it is first called for each frame containig this element, so surely the cache will change and the element will be relocated.

2) For cases with no frames: Each time when you interact with a different element (ie., with different locators), the cache changes. So to reproduce your scenario I should do something like that:

- set my_text_field to a value

- wait till my_text_field is removed from the DOM (without Watir interacting with any other element, as interacting would change the cache)

- set again my_text_field to a value (or do something else to my_text_field)

I can hardly imagine a need for such a test scenario in real life.

   

In conclusion, i'd still say that the application which uses 7 nested
(!!!!) frames is really badly made and should be rewritten anyway. No
offence.

No offence! I have already said it is not me who created the application!

 

So, it is just one really bad corner-case where it's slower
than usual, but unfortunately i don't see any good solution at this
moment how to solve it for everyone.

"Just one really bad corner-case", and I am just in it! 

 

I guess you can use #value for
that specific case, but i'm pretty sure you will also have slower
invocations with other methods.

Yes, the problem concerns every interaction with elements placed within nested frames.

Also, one solution would be to open that frame directly in your
browser instead of accessing it through all the other 6 frames.
Something like this with this concrete example or just create some
helper module, which could generate the correct url:
my_frame=ie.frame(:name, '7.6').frame(:name, '6.5').frame(:name,
'5.4').frame(:name, '4.3').frame(:name, '3.2').frame(:name,
'2.1').frame(:name, '1.0')
ie.goto ie.url.gsub(/\d+\.html$/, my_frame.src)
ie.text_field(:name, 'edit_box').set('123456789')

This would mean cutting the application into pieces (a piece per frame). And these pieces will most probably not behave as the whole application...

 

Jurek

[Jarmo Pertman]

On Aug 29, 6:47 pm, JMI <jurek.michal...@bluewin.ch> wrote:
> > Anyway, i looked at the code and saw that everything that can be done
> > is already done.
>
> I am not so pessimistic.

Thanks for not being! Your letter made me to look into it once again
and this time i paid more attention to code execution points and
noticed that there was indeed a bottleneck. In locator.rb in
FrameLocator there were these lines:
frames = @container.document.frames
@container.document.getElementsByTagName(tag).each do |ole_object|

@container.document went up in the @container's chain calling
assert_exists each time so it was done quite many times as you said.
I've now added a cache for @container.document during the time of
locating the element which speeds up your example in a magnitude of 10
times or more and the cache will be invalidated as soon as the element
has been located so there's not much of a danger of the cache being
out of sync.

The fix is in this commit (there's also some other refactorings
involved) https://github.com/bret/watir/commit/7ba7272b258005b0711fad61a7669632b05151e8
The lines you should be interested in are 6-8 and 118, 120 in
locator.rb. Try it out and let me know of your results.

> 2) For cases with no frames: Each time when you interact with a different
> element (ie., with different locators), the cache changes. So to reproduce
> your scenario I should do something like that:
> - set my_text_field to a value
> - wait till my_text_field is removed from the DOM (without Watir interacting
> with any other element, as interacting would change the cache)
> - set again my_text_field to a value (or do something else to my_text_field)
> I can hardly imagine a need for such a test scenario in real life.

It is quite possible in today's web 2.0 applications where DOM
elements are created and removed all the time by JavaScript so i
wouldn't be so sure that this is not a real life scenario.

> > In conclusion, i'd still say that the application which uses 7 nested
> > (!!!!) frames is really badly made and should be rewritten anyway. No
> > offence.
>
> No offence! I have already said it is not me who created the application!

I'm still not changing my opinion on that thing. If the application is
not done by you then maybe you can talk to the persons who did that?
If i'd have to test an application like that i'd really consider
changing my employer or projects :) Just a hint that life shouldn't be
too hard.

Again, thanks for not giving up on this problem which (hopefully) lead
to the fix :)

> Jurek

Jarmo

[JMI]

Le mercredi 31 août 2011 18:52:56 UTC+2, Jarmo Pertman a écrit :

On Aug 29, 6:47 pm, JMI <jurek.m...@bluewin.ch> wrote:
> > Anyway, i looked at the code and saw that everything that can be done
> > is already done.
>
> I am not so pessimistic.

Thanks for not being! Your letter made me to look into it once again
and this time i paid more attention to code execution points and
noticed that there was indeed a bottleneck. In locator.rb in
FrameLocator there were these lines:
frames = @container.document.frames
@container.document.getElementsByTagName(tag).each do |ole_object|

@container.document went up in the @container's chain calling
assert_exists each time so it was done quite many times as you said.
I've now added a cache for @container.document during the time of
locating the element which speeds up your example in a magnitude of 10
times or more and the cache will be invalidated as soon as the element
has been located so there's not much of a danger of the cache being
out of sync.

The fix is in this commit (there's also some other refactorings
involved) https://github.com/bret/watir/commit/7ba7272b258005b0711fad61a7669632b05151e8
The lines you should be interested in are 6-8 and 118, 120 in
locator.rb. Try it out and let me know of your results.

 

Just great!

Now (with Watir 2.0.1 and your corrections) the number of calls to assert_exists increases linearly with the number of frames - before (with Watir 1.8.7) it increased exponentially, as 2**n. For 7 nested frames of my simple example it means a factor of about 12.

 

> > In conclusion, i'd still say that the application which uses 7 nested
> > (!!!!) frames is really badly made and should be rewritten anyway. No
> > offence.
>
> No offence! I have already said it is not me who created the application!

I'm still not changing my opinion on that thing. If the application is
not done by you then maybe you can talk to the persons who did that?

The application is owned by a big IT company (some tens of thousands of people world wide) and it is customized for nearly a hundred of client companies (also world wide). Some 100 people work directly for the development and improvement of the application (for already some 20 years!) and some other hundreds for customizations. The structure of the web pages is deeply influenced by choices made some 15 years ago and we will have to live with them for some another couple of years. The frames were a performance problem a time ago (pages loaded slowly) but it was solved by dynamically loading only the modified frames (usually deep in the hierarchy). Actually the only problem with frames I know of is that of Watir - but - Hey! - you have just solved it!

 

Yes, I will talk about "that thing", but I will not put much of my energy into it! Nobody would want to change the GUI "just to modernize", with no other added value for the business.

 

If i'd have to test an application like that i'd really consider
changing my employer or projects :) Just a hint that life shouldn't be
too hard.

 

I do not test "the application" - we only test on-site customisations. Some years ago I created a key-word driven framework that uses a commercial test tool to interact with GUI (no problem with frames). Now the commercial tool evolves into a big you-do-not-have-to-know-how-to-code monster and I am thinking of migrating my framework into Ruby Watir. This is my personal spare-time project. So life is not too hard!

 

Thanks again, Jarmo!

 

Jurek