Query flakiness on Lolipop devices

135 views
Skip to first unread message

Hao

unread,
Apr 27, 2015, 8:29:58 PM4/27/15
to calabash...@googlegroups.com
Hi,

We've been experiencing a lot of flakiness on the chained queries on lolipop devices, particularly when there's non-element-blocking animation in the background - we are not always getting the results from queries on the same page view, for example, from the console:


irb(main):004:0> query("* marked:'test_list' child * index:0 descendant * id:'test_list_item'")

[

    [0] {

                     "class" => "android.widget.LinearLayout",

                       "tag" => nil,

               "description" => "android.widget.LinearLayout{233d1ca6 V.E...C. ........ 0,126-1248,386 #7f0a0348 app:id/test_list_item}",

                        "id" => "test_list_item",

                   "visible" => true,

                      "rect" => {

              "height" => 260,

               "width" => 1248,

                   "y" => 422,

                   "x" => 64,

            "center_x" => 688,

            "center_y" => 552

        },

                   "enabled" => true,

        "contentDescription" => nil

    }

]

irb(main):005:0> query("* marked:'test_list' child * index:0 descendant * id:'test_list_item'")

[]

irb(main):006:0> query("* marked:'test_list' child * index:0 descendant * id:'test_list_item'")

[]

irb(main):007:0> query("* marked:'test_list' child * index:0 descendant * id:'test_list_item'")

[

    [0] {

                     "class" => "android.widget.LinearLayout",

                       "tag" => nil,

               "description" => "android.widget.LinearLayout{233d1ca6 V.E...C. ........ 0,126-1248,386 #7f0a0348 app:id/test_list_item}",

                        "id" => "test_list_item",

                   "visible" => true,

                      "rect" => {

              "height" => 260,

               "width" => 1248,

                   "y" => 422,

                   "x" => 64,

            "center_x" => 688,

            "center_y" => 552

        },

                   "enabled" => true,

        "contentDescription" => nil

    }

]


Notice that the same query is sometimes returning empty results when they ran on the same view.

Here are some of the observations that I've made:

1. This only seems to happen on devices that are on Lolipop
2. This seems to more likely happen when there are animations in the background (not blocking any elements)
3. Chained queries are more likely to receive empty results compared to simpler queries (i.e. query("* marked:'test_list')

In our test runs, we would receive empty values for some queries when we are expecting values. And even when we put in wait_for's to wait until element_exist, the element may be there at the time, but when we do a touch() on the element, it could be missing at that moment, since the queries themselves are flaky in terms of giving results back.

I'm on the latest calabash-android build 0.5.8.

Has anyone experienced similar issues? Is there any way for me to debug and get more details on why this is happening?


Thanks a lot!
Hao

Hao

unread,
Apr 27, 2015, 8:51:52 PM4/27/15
to calabash...@googlegroups.com
Also, I tried to enable the DEBUG flags for calabash, and I'm pretty much seeing an empty json response:

irb(main):005:0> query("* marked:'test_list' child * index:0 descendant * id:'test_list_item'")

= Request


! CONNECT TO 127.0.0.1:34780

! CONNECTION ESTABLISHED

POST //map HTTP/1.1

Content-Type: application/json;charset=utf-8

User-Agent: HTTPClient/1.0 (2.4.0, ruby 2.0.0 (2014-05-08))

Accept: */*

Date: Tue, 28 Apr 2015 00:46:54 GMT

Content-Length: 138

Host: 127.0.0.1:34780


{"query":"* marked:'test_list' child * index:0 descendant * id:'test_list_item'","operation":{"method_name":"query","arguments":[]}}


= Response


HTTP/1.0 200 OK 

Content-Type: application/json;charset=utf-8

Date: Tue, 28 Apr 2015 00:46:53 GMT


{"results":[{"class":"android.widget.LinearLayout","tag":null,"description":"android.widget.LinearLayout{3e5ac387 V.E...C. ........ 0,111-1272,339 #7f0a0348 app:id/test_list_item}","id":"test_list_item","visible":false,"rect":{"height":228,"width":1272,"y":46,"x":393,"center_x":1029,"center_y":160},"enabled":true,"contentDescription":null}],"outcome":"SUCCESS"}! CONNECTION CLOSED

[

    [0] {

                     "class" => "android.widget.LinearLayout",

                       "tag" => nil,

               "description" => "android.widget.LinearLayout{3e5ac387 V.E...C. ........ 0,111-1272,339 #7f0a0348 app:id/test_list_item}",

                        "id" => "test_list_item",

                   "visible" => false,

                      "rect" => {

              "height" => 228,

               "width" => 1272,

                   "y" => 46,

                   "x" => 393,

            "center_x" => 1029,

            "center_y" => 160

        },

                   "enabled" => true,

        "contentDescription" => nil

    }

]

irb(main):006:0> query("* marked:'test_list' child * index:0 descendant * id:'test_list_item'")

= Request


! CONNECT TO 127.0.0.1:34780

! CONNECTION ESTABLISHED

POST //map HTTP/1.1

Content-Type: application/json;charset=utf-8

User-Agent: HTTPClient/1.0 (2.4.0, ruby 2.0.0 (2014-05-08))

Accept: */*

Date: Tue, 28 Apr 2015 00:46:56 GMT

Content-Length: 138

Host: 127.0.0.1:34780


{"query":"* marked:'test_list' child * index:0 descendant * id:'test_list_item'","operation":{"method_name":"query","arguments":[]}}


= Response


HTTP/1.0 200 OK 

Content-Type: application/json;charset=utf-8

Date: Tue, 28 Apr 2015 00:46:55 GMT


{"results":[],"outcome":"SUCCESS"}! CONNECTION CLOSED

[]

Mike Berkow

unread,
Apr 30, 2015, 11:01:17 AM4/30/15
to calabash...@googlegroups.com
I am experiencing similar issues here as well.  It seems that queries where there may be more than one element with the same ID may be the issue.

Rhy

unread,
Apr 30, 2015, 12:48:21 PM4/30/15
to calabash...@googlegroups.com
I'm finding the same thing querying arrays with Android 5.x simulators. 

first_claim_text = query(claim_date)[0]['text']

where 

element(:claim_date) { "* id:'cell_date'" }



On Monday, April 27, 2015 at 6:29:58 PM UTC-6, Hao wrote:

Hao

unread,
May 5, 2015, 6:29:27 PM5/5/15
to calabash...@googlegroups.com
Hmm, looks like multiple people are having the same issues, does anyone have any idea what might've caused it? Or is there any way for us to provide more info to root cause it?

Thanks

tobias....@xamarin.com

unread,
May 11, 2015, 5:06:43 AM5/11/15
to calabash...@googlegroups.com, evildr...@gmail.com
I am unable to reproduce this problem, but I would love to fix it. Can anyone provide me with a sample apk to reproduce? Feel free to email me an apk if you do not want to share it publicly. 

Brad Thompson

unread,
Jun 9, 2015, 1:21:39 PM6/9/15
to calabash...@googlegroups.com, evildr...@gmail.com
I just experienced this for the first time on my Note 4.  Basically I had a simple query which I performed to see if something was on the currently viewable screen.  Something like

query("* text:'Some Element'")

if this was not found I would alter it to do

query("all * text:''Some Element")
to confirm it does exist on view but needs a scroll.  Trouble is that first query above returns something even though it's off screen for some reason.

tobias....@xamarin.com

unread,
Nov 22, 2015, 7:51:48 PM11/22/15
to calabash-android
Hey everyone,

Could anyone having this issue please try running with this gem: https://rubygems.org/gems/calabash-android/versions/0.5.15.coordinate.pre.fix2

I think I have found the reason why this happens, and I hope this gem fixes it.

Thanks a lot!
Reply all
Reply to author
Forward
0 new messages