Java Robot mouseMove not working when tomcat is running as windows service

[Ravi]

I am trying to run an automation script using Selenium and in this script Robot class's mouseMove method has been used to move mouse cursor to a specific location. When I run this script from eclipse as a Java application or using tomcat (this script is a part of larger framework, which is deployed on tomcat), it is working fine.

But if I run tomcat as a windows service, the same code is not working. There is no error in the server logs, but the mouse cursor is not moving. Please let me know if anyone knows a resolution for this issue.

Following are the lines of code from the script, which produce no result when tomcat is running as a windows service:

WebElement element = driver.findElement(By.id("headerForm:j_id26"));

Actions builder = new Actions(driver); 

builder.click(element).build().perform(); 

Point mousePt = element.getLocation();

int x = (int) mousePt.getX(); 

int y = (int) mousePt.getY(); 

Robot robot = new Robot(); 

robot.mouseMove(x+100, y+110); 

robot.mousePress(InputEvent.BUTTON1_MASK); 

robot.mouseRelease(InputEvent.BUTTON1_MASK); 

[David]

Under what account does the Tomcat window service run under?

I speculate that running as a windows service still requires an account with UI/desktop access in order for the Java robot code to work, and not sure that will even work well, but that would at least be the basic requirement. After all, when not run as a windows service but under normal user account, the code is running with UI/desktop access.

Running as windows service is also akin to like locking the desktop or disconnecting remote desktop session, which are also known to trigger automation problems.

So at a minimum don't run the windows service under system or non-user account. And be sure the user account it is running under is logged in to desktop and desktop is not locked, etc. If windows services can be associated to user account sessions, then make sure it is session 0 (local desktop).

[Ravi]

Thanks for your reply David.

Tomcat windows service is running as Local System user. A Web application is deployed on this tomcat and through this application, the execution is redirected to Selenium grid (in a separate thread). Therefore, the actual execution is happening on a different machine.

I think Robot class can only be used in a script that is run locally and will not work in selenium grid. When I run tomcat is logged in user and try to run same execution from another machine, it does not work.

[David]

Ah, that makes sense then. You'd have to rework the design of the Tomcat app to make it work correctly. I blogged about this type of issue with some suggested workarounds but no implementation/solution that is ready made:

http://autumnator.wordpress.com/2011/12/22/autoit-sikuli-and-other-tools-with-selenium-grid/

you could also opt not to use Grid or use it in a way that where the Robot code (and perhaps Selenium code) runs locally on the target node rather than the code executing from Tomcat server with the Selenium portion farmed out to Grid. i.e. Tomcat instead calls Java code that runs locally on the target node via Java remoting or other technique, etc.