Swipe doesn’t seem to work in WebView context on Android using Selendroid

[Mario]

So I just updated to Appium 1.0.0. And after migrating my code to be compatible I noticed that the swipe doesn’t work in Hybrid or WebView context on Android

I’m using Selendroid to have compatibility with older Android APIs. 

I’ve tried the following code:

driver.swipe(540, 850, 540, 0, 1000);

to scroll down to the next element.

Selendroid comes back with a 500 response at this stage:

POST /wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform 500 16ms

debug: Appium request initiated at /wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform

debug: Request received with params: {"actions":[{"action":"press","options":{"y":850,"x":540}},{"action":"wait","options":{"ms":1000}},{"action":"moveTo","options":{"y":0,"x":540}},{"action":"release","options":{}}]}

debug: Proxying command to localhost:8080

info: Making http request with opts: {"url":"http://localhost:8080/wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform","method":"POST","json":{"actions":[{"action":"press","options":{"y":850,"x":540}},{"action":"wait","options":{"ms":1000}},{"action":"moveTo","options":{"y":0,"x":540}},{"action":"release","options":{}}]}}

info: [SELENDROID] replyWithServerError 500

info: [SELENDROID] /wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform

debug: Proxied response received with status 500: undefined

POST /wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform 500 11ms

debug: Appium request initiated at /wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform

debug: Request received with params: {"actions":[{"action":"press","options":{"y":850,"x":540}},{"action":"wait","options":{"ms":1000}},{"action":"moveTo","options":{"y":0,"x":540}},{"action":"release","options":{}}]}

debug: Proxying command to localhost:8080

info: Making http request with opts: {"url":"http://localhost:8080/wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform","method":"POST","json":{"actions":[{"action":"press","options":{"y":850,"x":540}},{"action":"wait","options":{"ms":1000}},{"action":"moveTo","options":{"y":0,"x":540}},{"action":"release","options":{}}]}}

info: [SELENDROID] replyWithServerError 500

info: [SELENDROID] /wd/hub/session/fe55616b-233c-64ab-6c45-d0179686ca15/touch/perform

debug: Proxied response received with status 500: undefined

On the client end I get a Command duration or timeout error with an Exception thrown pointing to the driver.swipe() line.

The way this worked for me before was to use seleniums TouchActions class to perform each step separately i.e. TouchActions.down(x, y).perform();, TouchActions.move(x, 0).perform(); TouchActions.up(x, 0).perform();

Using the same TouchActions with the AppiumDriver will produce some ClassCastExceptions

Does Anyone know how this is working now??  What’s the implementation I need?

This seems like an issue or incompatible APIs between Appium and Selendroid.

Any help is greatly appreciated. 

[Jonathan Lipps]

driver.swipe is a native-only command. It won't work when proxying to Chrome, since Chromedriver doesn't support it.

--
http://appium.io
---
You received this message because you are subscribed to the Google Groups "Appium-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to appium-discus...@googlegroups.com.
Visit this group at http://groups.google.com/group/appium-discuss.
For more options, visit https://groups.google.com/d/optout.

[Mario]

So what is the hybrid command we can use with Selendroid? Is there one? Or am I misinterpreting the whole point of java-client. i.e. Do I need to go back to using the WebDriver instead of AppiumDriver? 

Seems like there’s no way of scrolling on hybrid apps. At least none that I could find.

Thanks for the prompt reply Jonathan.

[Jonathan Lipps]

You should be able to use the same hybrid command with Selendroid

[Alex Fainshtein]

You can try to use the following:

JavascriptExecutor js = (JavascriptExecutor) driver;

HashMap<String, Double> swipeObject = new HashMap<String, Double>();

swipeObject.put("startX", 128.0);

swipeObject.put("startY", 943.0);

swipeObject.put("endX", 128.0);

swipeObject.put("endY", 255.0);

swipeObject.put("duration", 2.0);

js.executeScript("mobile: swipe", swipeObject);

worked for me in webview on appium 1.0

[Mario]

Thanks Alex,

Are you using AppiumDriver or WebDriver class as the backing driver implementation?

Jonathan,

The implementation that worked previously looked like this:

WebDriver driver = new SwipeableWebDriver(URL, capabilities);

where SwipeableWebDriver implementation looked like this:

public class SwipeableWebDriver extends RemoteWebDriver implements HasTouchScreen {

public RemoteTouchScreen touch;

public SwipeableWebDriver(URL remoteAddress, Capabilities desiredCapabilities) { 

super(remoteAddress, desiredCapabilities);

touch = new RemoteTouchScreen(getExecuteMethod());

}

@Override

public TouchScreen getTouch() {

return touch;

}
}

Is this still going to be supported in future version or is there a push to migrate everything to AppiumDriver and appiums java-client library?

Cheers,

Mario

[Mario]

@Jonathan,

I can’t use the same TouchActions class as I get the ClassCastException with AppiumDriver and HasTouchScreen that’s expected in TouchActions.

Tried using the TouchAction from the java-client library and that just fails when used with Selendroid. Here is the error log from Appium:

POST /wd/hub/session/b75de559-f4bb-e11a-39c1-a127838a536d/touch/perform 500 12ms

debug: Appium request initiated at /wd/hub/session/b75de559-f4bb-e11a-39c1-a127838a536d/touch/perform

debug: Request received with params: {"actions":[{"action":"press","options":{"y":850,"x":1079}}]}

debug: Proxying command to localhost:8080

info: Making http request with opts: {"url":"http://localhost:8080/wd/hub/session/b75de559-f4bb-e11a-39c1-a127838a536d/touch/perform","method":"POST","json":{"actions":[{"action":"press","options":{"y":850,"x":1079}}]}}

info: [SELENDROID] replyWithServerError 500

info: [SELENDROID] /wd/hub/session/b75de559-f4bb-e11a-39c1-a127838a536d/touch/perform

debug: Proxied response received with status 500: undefined

POST /wd/hub/session/b75de559-f4bb-e11a-39c1-a127838a536d/touch/perform 500 10ms

debug: Appium request initiated at /wd/hub/session/b75de559-f4bb-e11a-39c1-a127838a536d/touch/perform

debug: Request received with params: {"actions":[{"action":"press","options":{"y":850,"x":1079}}]}

debug: Proxying command to localhost:8080

I guess I can try extending the AppiumDriver and implementing the HasTouchScreen but I’m doubtful that would work.. I’ll try it later on tonight..

Cheers,

Mario

[Jonathan Lipps]

I don't think Selendroid supports the touch action library yet.

You'll need to look at selendroid's docs to see how to use it exactly.

[Mario]

Selendroid does support a touch library. However, it supports the Touch library that’s implemented in Selenium/WebDriver library (org.openqa.selenium.interactions.touch.TouchActions) . I’ve used it before to swipe/scroll and it worked. Selendroid doesn’t support the TouchAction in Appiums java-client library (io.appium.java_client.TouchAction) 

These two libraries have similar/same functionalities but their underlaying (JSON) commands are different and this is where a miscommunication occurs between Appium and Selendroid. I’ve tried the JavascriptExecutor method as well without any success.

Selendroid also needs a webdriver instance that implements the HasTouchScreen interface for the org.openqa.selenium.interactions.touch.TouchActions to work. AppiumDriver class doesn’t seem to provide this and when I try to extend the AppiumDriver class to implement the HasTouchScreen interface, it is still incompatible with the org.openqa.selenium.interactions.touch.TouchActions library. 

Maybe I’m doing something wrong here, still. If there are any more documents that I can read to get this working again I would be happy to do so, lads. IMHO, this is a bug. Or at least missed interim fix until Selendroid implements the same touch library as appium.

[Jonathan Lipps]

> Or at least missed interim fix until Selendroid implements the same touch library as appium.

Yes, this is the issue. In the meantime there should be a workaround to use the old TouchAction API. Jonah is there a way to make sure that AppiumClient can do that for Selendroid?

[Mario]

I managed to get around the compilation issues and class cast exceptions by implementing a new SwipeableWebDriver that extends AppiumDriver and implements the HasTouchScreen interface. This lets me use WebDriver and the old TouchActions implementation without the errors. This is the only way I got the scroll to work before. Now, I can see that the correct commands are being sent via JSON to selendroid. Hopefully this keeps working for a while longer.

Thanks lads.

Mario

[bootstrap online]

You should probably open an issue on https://github.com/appium/java-client

I think AppiumDriver could implement HasTouchScreen so you don't need
to subclass.

[Mario]

https://github.com/appium/java-client/issues/27

This should be enough but if you need more details I can provide later.

[Priyank Shah]

Thanks for rasing this issue.even i am facing same issue with native application. I am also running my test cases on real device in selendroid mode.

java.lang.ClassCastException: io.appium.java_client.AppiumDriver cannot be cast to org.openqa.selenium.interactions.HasTouchScreen
    at org.openqa.selenium.interactions.touch.TouchActions.<init>(TouchActions.java:37)
    at screens.FileChooserPage.selectFileFromList(FileChooserPage.java:42)
    at utils.Functions.enigmaSelectFile(Functions.java:316)
    at LaunchTest.testFileTransfer(LaunchTest.java:46)


Results :

Failed tests:
  LaunchTest.testFileTransfer:46 » ClassCast io.appium.java_client.AppiumD...

Thanks,
Priyank Shah

[Mario]

Hey Priyank,

So in the workaround I described above, you should subclass the AppiumDriver and implement the HasTouchScreen interface. The implementation should look something like this:

public class SwipeableWebDriver extends AppiumDriver implements HasTouchScreen {

public RemoteTouchScreen touch;

public SwipeableWebDriver(URL url, Capabilities caps) {

super(url, caps);

touch = new RemoteTouchScreen(getExecuteMethod());

}

public TouchScreen getTouch() {

return touch;

}

}

Then, you use the WebDriver to interface to interact with your app.

WebDriver driver = new SwipeableWebDriver(“http://localhost:4723/whatever”, caps);

this should help with your ClassCastException problem, however, if you’re testing a native app you don’t need any of the code above. Just use the AppiumDriver.swipe() method

[SOUVIK DUTTA]

Hi Mario,

I am trying to use you code to swipe in WebView, but it doesn't work. Please let me know, where I am going wrong. I am using normal Appium dependencies, do I need selendroid jar as well. Could you please list the JAR required if anything specific is needed.

=================================================
if(browser.equalsIgnoreCase("android")){
 DesiredCapabilities capabilities = new DesiredCapabilities();
 capabilities.setCapability(CapabilityType.BROWSER_NAME,"");
 capabilities.setCapability("deviceName","Android");
 capabilities.setCapability("device","Android");
 capabilities.setCapability("takesScreenshot","true");
 capabilities.setCapability("platformVersion","4.4.2");
 capabilities.setCapability("appPackage","uk.co.ee.myee");
 capabilities.setCapability("appActivity","uk.co.ee.myee.Launcher");
 //capabilities.setCapability("udid","320497a8f74a11c9");
 capabilities.setCapability("udid","989fb005");
 driver = new AppiumSwipeableDriver(new URL("http://127.0.0.1:4723/wd/hub"),capabilities);
=====================================================
public class AppiumSwipeableDriver extends AppiumDriver implements HasTouchScreen{
public RemoteTouchScreen touch;
public AppiumSwipeableDriver(URL URL, Capabilities Cap) {
super(URL, Cap);

touch = new RemoteTouchScreen(getExecuteMethod());

}
@Override

public TouchScreen getTouch() {
return touch;
}
}

=====================================================
 waitForVisible(By.xpath(OR.getProperty("wblL2Menu")),20);
 driver.findElement(By.xpath(OR.getProperty("wblL2Menu"))).click();
 driver.swipe(100, 200, 500, 200, 1000);
====================================================
Error

org.openqa.selenium.UnsupportedCommandException: unknown command: session/cb341a3868ff6d8c5ff192589d11d8ae/touch/perform
Command duration or timeout: 110 milliseconds
Build info: version: '2.42.2', revision: '6a6995d', time: '2014-06-03 17:42:03'
System info: host: 'SouvikDutta-PC', ip: '127.0.0.1', os.name: 'Windows 7', os.arch: 'x86', os.version: '6.1', java.version: '1.7.0_45'
Session ID: a759ac2b-fe87-4f79-af96-da09be2b3228

Thanks

Souvik

[Aniket Gadre]

I don't understand why is there a need to do a swipe on webview? Even if you are unable to see a specified object on screen (especially in a webview because your webpage is pretty long) and if you try to take any action on it, it should work. Thats what my experience is. Webviews generally auto scrolls to required element.

 

 

Best Regards,

Aniket Gadre

 

“Programming can be fun, so can cryptography; however they should not be combined.”

--

[Priyank Shah]

Hi Souvik,

You did not call perform method.
and it should be like

TouchActions action = new TouchActions(driver);
                action.flick(WebElement, 0, navigationDirection,
                        SCROLLINGSPEED).perform();


Thanks,
Priyank Shah

[SOUVIK DUTTA]

Hi Priyank,

Thanks for prompt response, could you please confirm that instead of swipe i need to use flick as mentioned by you .... right ???

@ Ankit : I have tried to reach the element directly, but it doesn't work.

Thanks

Souvik

[naddy095]

Hi,

Even I face same the problem, the webview element its not possible to interact with the elements that are not visible and it does not even swipe up by itself.Any work around for this.

Even if you flick to swipe how is one supposed to know when to stop flicking?

Thanks and regards.

[Aniket Gadre]

You need to put it in a loop...checking after every flick, whether you are able to see required object

 

Best Regards,

Aniket Gadre

 

“Programming can be fun, so can cryptography; however they should not be combined.”

[Priyank Shah]

       
I had written below code.. I think that can solve ur problem.. :)

        boolean isFileSelected = false;
        while (!isFileSelected) {
            if (fileList.get(0).getText().compareTo(fileName) > 0) {
                navigationDirection = SCROLLUP;

            }
            TouchActions action = new TouchActions(driver);

            action.flick(fileList.get(0), 0, navigationDirection,
                    SCROLLINGSPEED).perform();

            elementListView = driver.findElement(By
                    .className("android.widget.ListView"));
            fileList = elementListView.findElements(By
                    .className("android.widget.TextView"));

            for (WebElement el : fileList) {
                if (el.getText().equals(fileName)) {
                    el.click();
                    isFileSelected = true;
                    break;
                }
            }

But please, That worked for native app. I m not sure about hybrid app.

Thanks,
Priyank Shah

On Thursday, May 15, 2014 4:20:06 PM UTC+5:30, Mario wrote:

[Mario]

Hi Souvik,

All the libraries I’m using are the WebDriver with it’s dependencies and the Appium java library.

To your swipe problem.

As I said above driver.swipe() didn’t work for me on hybrid. I had to implement my own swipe which was basically using TouchActions. 

The gist of which is ta.down(x, y).perform(); ta.move(x, 0).perform(); ta.release(x, 0).perform();

Then you can put this code inside a loop to keep scrolling until you find the element you’re looking for.

Hope that helps,

Mario