Using Selenium over the internet

[Bilal Abdulkany]

Hi,

I am new to Selenium Grid and RC. We have a requirement to use Selenium Grid over the internet. That is, Have a centralized hub inside a network and the nodes would connect to the Hub through the internet.

Is it possible to achieve this? What are the security issues, since this is somewhat a secure website.

Appreciate your support in this.

Thank you!

[Krishnan Mahadevan]

Am no expert in the security domain, but this sounds like a recipe for disaster.

Why would you want to do something like this ? Can you please help elaborate your use-case ?

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/588e60eb-7baf-4170-9580-8968329aebcb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bilal Abdulkany]

Well, we are using Selenium in tomcat web application to automate the login to another of their website. Using selenium, we will manipulate the javascript in that website using the Selenium RC. So there will be an installation setup given to the users that would install the node services in their machines. The hub would connect to the node and perform the operations after the customer logs in.

[David]

Curious, what "operations" are performed after customer logs in and hub connects to the node? This doesn't seem like a good design.

[Bilal Abdulkany]

This would perform some javascript manipulations on that webpage. This is the only design that would complete the client's requirement lol

[David]

Why would they need javascript manipulation on the page? That just seems strange.

If they needed to do something against the website (submit data/form, register user, login), all that can be done with pure HTTP requests and cookies simulating a browser session. Javascript on a webpage either makes AJAX calls you can simulate with HTTP or it's all local client side javascript code that displays/runs on browser (no interaction with server). This latter case makes no sense other than something like automating client side browser actions for a demo or like automated game playing with a bot, etc.

Can you elaborate more perhaps?

[Bilal Abdulkany]

It's like this. The webpage I spoke about is cross-platform. We cannot send such requests to it. What I meant by js manipulation is to hide the logos and manipulate the html elements in it. What we developed is a web portal with a single sign on. We use the login information in the portal to log into that web page and manipulate it's html elements. This web page I spoke about is a product that was used by the company and we can do whatever we want with it.

[Bilal Abdulkany]

Yes. We are using selenium Grid to automate this website. So if anyone has used the Nodes over internet, please let me know of it's feasibility.

Thank you.

On Monday, April 27, 2015 at 10:52:07 AM UTC+5:30, Brian Smith wrote:

It seems to me you're attempting to utilize Selenium and WebDriver to automate administration or day-to-day tasks through a web interface, which does not include 'testing' and is not what the platform is intended for.

[Jim Evans]

I'm compelled to jump in here and mention that this comment is diametrically opposed to the project maintainers' attitude, that the library is "a browser automation library," not "a library for automated testing."

[Brian Smith]

Ok, in fairness I should say it's primary use is intended for testing automation. I developed a Java UI application using Selenium for a client in the past to drive pulling massive amounts of data in a web admin application as it was tedious and prone to human error.

[David]

Yes. We are using selenium Grid to automate this website. So if anyone has used the Nodes over internet, please let me know of it's feasibility.

I doubt anyone has really done this for testing or other uses beyond a demo/trial test. I don't see why it's not possible though. The hub and node just must be able to communicate with each other. Meaning they can ping each other, you should ensure the needed ports are open, that would be at a minimum the port that the node registers to the hub with. For security, you may want to ensure that the path the node connects to hub and vice versa is routed through a VPN connection or an SSL tunnel.

But regarding your setup, do you really need a hub? One that's internal? Why not have the hub and node together within the same network? Each client/customer installs and runs their own hub & node. Your tomcat app as the Selenium client just connects to the customer's hub over the internet. This approach is simpler to manage and debug in terms of networking. And if you were using Selenium 2 WebDriver, you could do away with need for hub if it's not really needed and just run the node as a standalone RemoteWebDriver node/connection (no Grid setup).

But again, regarding using nodes over the internet, I think the best use case you might find (if you search around) might be one where the hub is internal (or not) and the nodes are deployed on Amazon EC2 cloud. I know some folks have done that with JMeter distributed load testing, I'm assuming someone may have taken similar approach for Selenium grid. You just then need to adapt that concept to yours that doesn't use EC2.

[Bilal Abdulkany]

Hi,

But if the hub is internal, how can we give the selenium commands to it? We implemented each node with an associated username and password, and to prevent the hub from throwing random test scenarios on every node, we implemented a custom capability matcher. Also to direct it to the specific node, we used an attribute 'applicationNode' in the node configuration which has a unique name for that node. When the node is up and running, the node connects to the hub and when the user clicks a specific link, the web application interacts with the hub and directs the commands to the node as to what should happen.

[David]

Curious who "architected" this design for you? I would assume this "architect" would know whether his/her own design should work or not, so that you don't actually have to post the original question of this thread here other than to ask if anyone else has done something similar before.

And pardon my ignorance, I don't work with Grid a whole lot, but I don't recall grid & node requiring login authentication, so that must be a custom implementation? If that is actually available with grid hub/node by default, please point me to the info about that.

Is your setup described as a central hub in your network, and a unique node at each client site, and the custom mapping of node configs allows you to route specifically to each client node? So your hub sits between web app and client node, acting as proxy to route action to client node from web app? If so, an alternate design could be to remove this "extraneous" proxy, and move the node routing logic away from hub into the web app itself. The web app probably already has some of that logic that it passes to the hub to tell the hub what "capability" it needs for where to run. So, when user clicks link, the web app should know which specific client (node) needs to execute the commands (this could be tied to the user's login or other unique identifiers). So the web app then sends the Selenium commands directly to the client's "node". The client's "node" in this case is an abstract term. The node itself can be a hub at client site that talks to the real node to execute commands, or instead of hub + node, it's just a RemoteWebDriver server instance running at client site (no hub, just a node in RemoteWebDriver non-Grid mode). And each client site could be deployed like this.

In the original setup or my described alternate setup, you still have to find a way to "secure" the connection between your site/end (e.g. your web app or your hub as endpoint) and the client's site/end (their node as endpoint). I just think from a networking topology perspective, the alternate setup is simpler as it is better to use less connections (or shorter routes) and less middleman proxies.

A reason you may want to stick with your hub design is if you need to check status and manage the execution of commands across multiple clients (assuming they take a while or you need to restart nodes), where the hub may give you a nice GUI dashboard for that vs without a hub. Otherwise, the hub just seems like an extra layer for not much gain.

[Bilal Abdulkany]

Hi,

I architectured this design for the requirement of the client. I hope to pursue this as research now :) 

Well the login credentials are for the username, password and also the unique application node name to register to the hub via the tomcat web application. 

Yes, the hub acts as a proxy and this design is to check status and manage execution of commands across multiple clients. This just acts as a remote control and you are correct it provides a dashboard of all the clients registered to it. The security is https on Tomcat and also added SSL certificates on the application server to prevent Man in the middle attacks.