Interact Touch

[Frederik Bangle]

Hello Henrik & others,

I am using Automate since 2 years with big flows (>1k blocks each), without any issue so far. Lately, I noticed that sometimes my flows get stuck  either on an Interact Touch block, or Interact blocks. The log just shows "2022-01-04 : XXXXXX@1046 Interact Touch" and it stays like this without any error, and doesn't do the action, I just need to stop it. I do think it's related to Accessibility, so I use the root superuser shell Enable_accessibility_services, however it doesn't fix it totally, as it's still appear. The accessibility setting is always enabled, this bug appears only for Interact/Interact Touch blocks, and happens here and there for any of them, not one in particular. 

I also use dumpsys command lines for inspection, to get the exact layout as the one with Inspect is sometimes buggy. Thing is, am I the only one having this issue with those Interact blocks since 2-3 weeks ?

I've been adding some more superuser shell blocks with dumpsys UI Layout inspect  and I'm wondering if it's the cause. I just can't seem to figure it out...

Thanks!

[Henrik "The Developer" Lindqvist]

Hmm, seems like dispatching a gesture can fail or be denied, not just cancelled. That's probably what's happening, but the block get stuck wait for a completion or cancellation callback. I'll fix that.

[Frederik Bangle]

Thank you.

I only started seeing this last 2-3 weeks ~ while using Automate since a long time, why does this happen only now with the Interact / Interact Touch blocks ?

I thought it was linked to the Superuser commands I implemented (tap X Y) with Dumpsys, but I'm glad if it's not that.

If you need any details from the logs, even if there's no error, don't hesitate.

[Henrik "The Developer" Lindqvist]

Maybe you've updated to the alpha version which now targets Android 11?

[Frederik Bangle]

Automate version is 1.30, with Android versions going from 9 to 11 (multiple phones with this same bug, appearing at the same time)

[Frederik Bangle]

Note : it persists also with last version of Automate, and with the last version of Accessibility (Samsung devices). Didn't try with alpha of Automate tho

[Frederik Bangle]

Hello Henrik,

I have been doing some tests, and conclusion is that if there are 2 fibers running (for example after a FORK), with a Dumpsys Inspect (adb superuser block) AND at the same time on the other fiber, an Interact Block, the Interact Block will fail with "No accessibility service running".

I tried the same loop without Dumpsys, it worked 100x times, and with Dumpsys, accessibility service get all the time disabled.

Any thoughts about this ?

Thanks 

[Frederik Bangle]

I could see some workaround here, but couldn't make it work.

https://gitanswer.com/appium-testing-android-accessibility-services-javascript-68389530

[Henrik "The Developer" Lindqvist]

"No accessibility service running" is often caused by the app being killed, and due to an Android bug it's not being restarted properly:

https://issuetracker.google.com/issues/75414169

Sadly there's no workaround, except trying to minimize the chance of Automate being killed.

[Frederik Bangle]

When this happens, Automate is running fine and isn't being killed. The fiber with "UI_Automator Dump" superuser block keeps running fine, while the other fiber, with the Interact Touch block, stopped running ("No accessibility service running).

I found out this answer on github :

"It is not possible to test Accessibility Services in parallel with UiAutomator.

Both UiAutomator(including UiAutomator2) and AccessibilityService depends on same resources internally which are singleton in nature and UiAutomator API always holds lock on those resources.

As a result it just not possible to use both UiAutomator and AccessibilityService at a time."

Source : https://github.com/appium/appium/issues/4910

So, if as you said, there is no workaround : how can I inspect the whole window in the foreground and get all the XML with the Views, the ID, the yexts, the coordinates,... I want to be able to fetch all of them, and then read them in a file to get the coordinates (that was why I used "UI automator dump).

Thank you !

[Henrik "The Developer" Lindqvist]

I have no idea what dumpsys accessability does, it's probably doesn't require a running service.

It's currently not possible to get the XML of all windows, just the foreground one, using the Inspect layout block.