Grid 4.0

[Simon Stewart]

An introduction to the new Grid

I'm going to assume you all know how to build selenium and are familiar with how it all works, so this is just an introduction to building and setting up the new Grid. There's still work to be done (some of which is detailed at the bottom), but this is enough to show the idea working as planned…. I'll also be writing a wiki page, but want to stabilise things first before pushing this in front of the world :)

As ever, I'm always happy to hear feedback, either privately or on one of the many channels we use to talk about this stuff.

As with the original selenium standalone/grid binary, everything is in one place:

./buckw build grid-tng

Or, if you’d like to build without the alias path so you know where the code is:

./buckw build //java/server/src/org/openqa/selenium/grid:selenium

Now it’s built, time to run it!

java -jar buck-out/gen/java/server/src/org/openqa/selenium/grid/selenium.jar 

This should give you a list of all the commands that the binary is aware of. As the output says, you can run these by appending the command name and any extra flags you’d like.

First off, we need a “session map” server. Essentially, this acts as a hash of “session id” to “where on earth is the session running”, though there’s some additional information kept in there too.

java -jar buck-out/gen/java/server/src/org/openqa/selenium/grid/selenium.jar sessions

The next thing we need is a server that can distribute new sessions to nodes within the grid. You start this using:

java -jar buck-out/gen/java/server/src/org/openqa/selenium/grid/selenium.jar distributor

As you’ve started each of these, they’ve output the ports that they’re listening on. This is important, because the next step is to add a node. A node is responsible for actually running a session, and it needs two bits of information: where the distributor is (so it can let it know that there’s a new node in the world) and where the session map is (so later requests can be directed efficiently). Assuming you’ve used the default ports, you can run it using:

java -jar buck-out/gen/java/server/src/org/openqa/selenium/grid/selenium.jar node --sessions http://localhost:5556 --distributor=http://localhost:5553 --detect-drivers

That “detect-drivers” is needed so that the node will automatically check the system for which drivers it can use. At some point, we’ll allow you to configure this more meaningfully :)

So, you’ve now got the basic system up and running. However, it is unlikely you’d want to expose your internal infrastructure in this way: we also want a gateway that local ends can connect to and communicate with. This is what the router is for:

java -jar buck-out/gen/java/server/src/org/openqa/selenium/grid/selenium.jar router --sessions http://localhost:5556 --distributor=http://localhost:5553

This will be listening on port 4444, as you’d expect, and so you can now start a webdriver session by pointing at “http://localhost:4444”

The router is effectively stateless, so if you’re using something like k8s you can have a whole fleet of them fronting your grid. When a selenium command reaches the router, it finds the node running the session and forwards the message directly to that node, bypassing the distributor entirely. It caches session locations too, so often the only thing it needs to do is forward a request directly to the server that will process it — this should help us to scale grid to gigantic sizes.

Of course, starting all these things together is a massive PITA, so, just like before, you can spin up the entire grid in a single process using:

java -jar buck-out/gen/java/server/src/org/openqa/selenium/grid/selenium.jar standalone --detect-drivers

Unlike in previous versions of selenium, this will actually use exactly the same components as above, but this time simply linking them all together in the same process rather than starting up a small fleet of servers.

There are obviously a huge number of things to do on the new Grid:

• Right now, the order of start up matters. Nodes don’t try and attach to the grid repeatedly, and there are no health checks
  
• There are no health checks at all, for that matter
  
• Integrate Open Tracing and structured logging into the Grid. It should be possible to host this stuff in a cloud provider, point Honeycomb or DataDog at the logs and get traces out without any additional work by hard-pressed SREs.
• It should be possible to add Selenium 3 Grid nodes to the grid by pointing them at the distributor instead of a normal hub
  
• Protocol conversion needs to be pulled out into a filter
  
• RC support also needs to be spun out that way too.
  
• I want to add a redis-backed session map (probably using jedis) so that we can deploy this thing with some resiliency.
• Finish rolling in the scheduler branch to the tree so that we can actual distribute work fairly.
• Allow for configs to be created. I really want some way to define a patchset for JSON, but I'm happy to purloin bits from other successful grid implementations.
• Request retries would be nice --- if this server is down, go to this one.

I'm sure there's more, but that's enough for now :)

Happy hacking, folks!

Simon

[⇜Krishnan Mahadevan⇝]

Simon 

A quick basic question. In the earlier version I had to spin off two JVMs to get a hub and node working. 

With the new setup it looks I may need 4 jvms to get the setup running ( sessions/distributor/node/router). 

Is my understanding correct ? 

I know, looking at the code I can get this answer but still thought I would ask. Which would be the component that a test will interact with to create a new session. Is it the router or is it the sessions. I am guessing that it must be the router. 

--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-develo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/CAOrAhYHusSOsXrL6V1n_w47XbhZTVggUVkiDCWfog2DrVz%3D1Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[Simon Stewart]

Hi Krishnan,

Good question. The short answer is that there are currently three ways to run the new Grid:

* All in one JVM, via "standalone"

* In two JVMs, via "hub" and "node"

* In four JVMs, via "router", "distributor", "sessions", and "node"

The second mode was added so that users who were familiar with the old grid could get up and running nice and fast :)

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/CANikZL%3Dk4XipVQ_L5f7y1j4o0QUj6bKP9j%3DGZLxnBXYAGYggrA%40mail.gmail.com.

[Paul Bruce]

Simon,

Amazing work! Per our discussions back in Chicago, I'm just now getting back to this work (happy holidays!) to refactor a Custom Proxy Class written for SeGrid2 for this new 4.0 model.

After digging, I don't see anything like DefaultRemoteProxy in the new stuff. Can you point me in the right direction or provide guidance?

My goal is to embed hooks for various lifecycle events (such as beforeSession, beforeCommand, afterSession, afterCommand overrides) using the 4.0 model. Is there a new way to do this at the Node level?

Much appreciated,

//Paul

[Simon Stewart]

Hi Paul,

Happy New Year to you too! The issue is #6666, and although it's assigned to Alexei right now, I'm sure he'd appreciate a hand with it. The equivalent of the RemoteProxy is the Node abstract class, so that's a good place to start looking.

We're still nailing down the model we'll use, so expect some flux. Notably, I'm about to start exploring a more event-driven approach. Hopefully I can land that with minimal disruption :) For the events you mention, the command hooks are probably best in the Node, and the Session hooks in the Distributor (though the Node may also be a good choice)

If you need a hand, we're almost always hanging out on the #selenium IRC channel on Freenode, and that's mirrored to Slack too. Feel free to ping us there too.

Cheers,

Simon

--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-develo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/f4f772df-c069-461f-afc9-23206edf36ce%40googlegroups.com.