SikuliX Community or "Official" Library

[Clark Burbidge]

Kudos to all involved here for all the great work on Robot Framework and the surrounding community! Anyone named Clark gets extra kudos (plus bonus for umlauts or their Finnish cousins)!! I'm new to testing and RF and have enjoyed getting involved on the fringes. It seems to me that web testing is a strong part of RF, but that GUI testing is lagging behind. I could be very wrong and would love to be corrected. My GUI testing is lagging and could use some work. Because of my impression & research into alternatives I think that a Sikuli Library could help build the GUI testing functionality of RF.

I am posting hoping to generate some discussion about SikuliX and the possibility of a community or official library. I have found many tries (probably successes that have stumped me) and several good ideas around Sikuli, testing, making GUI testing more robust, logging and debugging, etc., but nothing quite as strait forward as say the Selenium2Library.

Is there a reason this is missing? It looks to me that SikuliX is a bit of moving target but also that RaiMan has interest in making SikuliX into a more useful tool for testing and RF. See his recent update. I wish I had the programming skills to jump in and write/modify/update a library myself, but is there something out there that I am missing as to why it doesn't exist yet? Can we work together to get it done? I'm learning and do well if prodded in the right direction.

Here are a list of examples I've found so far:

• http://blog.mykhailo.com/2011/02/how-to-sikuli-and-robot-framework.html - THE post on Sikuli and RF integration
  
• https://github.com/Tset-Noitamotua/Sikuli-and-Robot-Framework-Integration/tree/Windows-8.1 - Redo of the above link but on Windows 8.1 and more
  
• https://github.com/alistair-broomhead/jython-sikuli-server - Alistair has 3 versions
  
• https://github.com/jatalahd/SikuliRobotLibrary - jatalahd has one in Java
  
• https://github.com/smysnk/sikuli-framework - An abstraction for robust and OO interactions
  
• http://www.8bitavenue.com/2012/04/sikuli-selenium-robot-framework-integration/ - a limited set of keywords for Sikuli that I actually have working
  
• https://github.com/ljszalai/rfsikuli - Abandoned fork of https://github.com/daluu/simplesikuli from http://code.google.com/p/simplesikuli/wiki/RobotFrameworkIntegration for the Sikuli Java API which I think has changed now in SikuliX v1.1.

As well as some good related ideas:

• https://github.com/smysnk/sikuli-framework - Ideas on more robust testing with SikuliX 
  
• https://code.google.com/p/rguils/wiki/RobustGUIAutomation - Ways to better treat image change and robust framework
  
• http://blog.mykhailo.com/2011/02/how-to-sikuli-and-robot-framework.html - At the very end of the post, better logging, debugging and integration with the RF log.
• http://www.sikulix.com/support.html - SikuliX support for RF in its IDE that I saw today and need to try out

Anyone have any more?

As I said up top, thanks to all involved and for all the great work so far! I'd love to hear everyone's thoughts and help where I can if there is a movement in this direction.

Cheers,

Clark

[Rajesh Taneja]

Thanks for starting this discussion Clark.

I can see Robot website recommend https://github.com/rainmanwy/robotframework-SikuliLibrary#readme.

But it is very limited and search is on screen, which is not affective as well. 

While https://github.com/jaredfin/SikuliXRobotLibrary is much better and has exposed all sikuli api's.

Which one do you recommend ?

[Hélio Guilherme]

Thank you Clark for posting the resources.

I do use SikuliLibrary, even if it is a bit outdated. I specially like the Remote Robot use, meaning that I don't even have to install Jython, just Java. And the library takes care of starting the Java side.
The most missing feature, is text recognition. Even tried Tesseleract (or something like that). Because of this missing/not functional feature, I don't invest much in this kind of test automation. I don't want to take image snippets for any element I need to interact with.

My project is a Java Applet (Oracle WebForms) and I must use Internet Explorer with it.

I'll keep following these Sikuli/RF ecosystem because they are good projects (too good to fade away).

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-users+unsub...@googlegroups.com.
To post to this group, send email to robotframework-users@googlegroups.com.

Visit this group at https://groups.google.com/group/robotframework-users.

For more options, visit https://groups.google.com/d/optout.

--

My Favorite Open Source Projects
awsome-lists gretl meld robotframework wxPython

[David]

I originally created simplesikuli library as a proof of concept for Sikuli usage in a simplified wrapper that could work with RF as well as executed remotely via the RF Java remote server interface (whether for use with RF or something else). Sadly since then, Sikuli had major changes and my library is no longer compatible with recent versions of it. I've since not touched Sikuli, although I'm still interested in the tool for GUI testing use. It's both great and not so great that there's so many options for Sikuli with RF and no main/official library for it. All this is still progress over not having any options at all.

Alternatively, another good approach for Sikuli that would also work with RF would be to instead of having a Sikuli library to have a Sikuli RemoteWebDriver interface and RF users can connect to it via Selenium2Library (or Appium related RF libraries). This solves a bigger objective in that more people can use Sikuli w/o the Sikuli API (or having to use it in Java or python) since they can control it via WebDriver API. I brought this up in various forums including the Sikuli one. Not sure what progress has been made since on that. I also started a project on it to build a proof of concept (using Jython 1st before trying native Java API), but never got around to spending time to make progress on it: https://github.com/daluu/SikuliDriverServer. 

If you ask me now about making better use of Sikuli for GUI automation for the community, I would rather work towards a SikuliWebDriver interface that could then be used with RF than to make a specific RF library for Sikuli so as to offer greater adoption and integration with test tooling, frameworks, and clients. Think of it as the Appium approach, where there's also an Appium RF library, as opposed to making separate RF libraries for XCUITest, UIAutomator, Selendroid, etc.

On a related note, for GUI testing, aside from Sikuli, there's alternative tooling options one can consider, that could also be integrated with RF. Not sure if anyone wrote RF libraries for these already or not:

• Microsoft WinAppDriver (may already be part of Appium now?)
• https://github.com/asweigart/pyautogui
• https://github.com/autopilot-rs/autopy
• Appium for Mac: https://appium.io/docs/en/drivers/mac/
• another Mac GUI tool: https://github.com/pyatom/pyatom

[David]

Btw, I almost forgot, I was involved in an Appium discussion thread long time back, and as a result, image recognition based element location strategy (for what I assume is generic cross platform GUI automation support) is on Appium roadmap, but it'll be a long way coming: https://github.com/appium/appium/issues/943. If that's available, there may be less need for Sikuli.

[Hi-Fi]

Especially if you run RF with Java, there's https://github.com/Hi-Fi/remotesikulilibrary.

That also works as remote, but client might need a bit writing because of e.g. image base64 en- and decoding.