Selenium java: helper servlet - NodeConfigurationBasedServlet

[Praveen Tiwari]

Gents,

Sometimes an extra servlet at node requires access to node's configuration but there is nothing to get it without some dirty workaround.

If there could be an abstract servlet implementation similar to RegistryBasedServlet, it'd become easier to access Node's config (something like NodeConfigBasedServlet). This can be done the same way as RegistryBasedServlet is done, i.e, by adding NodeConfig in the ServletContextHandler's attribute while starting the node (for example SeleniumServer is instantiated with GridNodeConfiguration).

If it's doable, I am ready to send a PR. Please let me know if there is another way to access node's config at node if am overlooking something.

Thanks very much,

Praveen

[Krishnan Mahadevan]

Praveen,

 

Can you please help elaborate your use-case ?

Right now, you should be able to build a RegistryBasedServlet, wire it into Selenium Hub and then given a node id (which is typically its IP and port), invoke getRegistry().getAllProxies().getProxyById(id).getConfig()

 

You don’t need to be going to the node to get hold of the configuration, because the node at the time of registration forwards its configuration to the Hub.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-develo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/CAOuKoX3HO9Dvf95pGj5rqXkimFJkgYbCH0AjgxyPDT7jTOSSBA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Praveen Tiwari]

Krishnan,

Thanks for getting back. Yes i know that but my question was different. I know how to get all that at "hub" server but i was finding some clean way at "node". Please note that hub and node aren't on the same server and JVM (most cases and mine is no exception). The registry is instantiated for hub, not node thus you can't get any of that at node. Hope it makes sense.

btw, i wrote a workaround of it. I extended the GridLauncher class, checked that the current role == NODE and read the command line supplied options to build GridNodeConfguration which worked well but it still a workaround. I read that new PRs for old grid are stopped.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/3BF75E94-133C-47B6-A89B-ECA53773A24E%40gmail.com.

[Krishnan Mahadevan]

>>>> I read that new PRs for old grid are stopped.

 

Yes, officially Selenium3 development has stopped and as such PRs would be accepted only for Selenium4.

 

>>>> I know how to get all that at "hub" server but i was finding some clean way at "node".

 

That was why I was asking what exactly is your use-case ? Anyone who is working with the Selenium Grid, ideally speaking should not have any need to directly talk to a selenium node.

 

>>>>> btw, i wrote a workaround of it. I extended the GridLauncher class, checked that the current role == NODE and read the command line supplied options to build GridNodeConfguration which worked well but it still a workaround.

 

Yes, it’s kind of a workaround for sure because now you would need a user to be able to talk to both the hub and the node directly. But if you resort to the approach that I had highlighted, you would just be talking to the Hub alone and not to the node.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/CAOuKoX1R26B3o1ENfvHFe31MA3K3eXZUF-K_uAiyM1ST6Z4OfQ%40mail.gmail.com.

[Praveen Tiwari]

ideally speaking should not have any need to directly talk to a selenium node.

Really? what if you've a feature rich servlet at node that require access to 'custom' data sent in nodeConfig at the time of servlet instantiation? It's not an external user or process that is accessing node directly, its a servlet that is loaded together with node server, resides inside it and need access to the data of its own container, whats wrong in it?

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/2FADD8F6-DDB7-42D9-A86C-79CCF4874B49%40gmail.com.

[⇜Krishnan Mahadevan⇝]

Not sure where you are going with this. But I would reiterate what I mentioned earlier. 

You really don't have a need to talk to node level servlets from an end users test. All of those can be done at the hub level without losing any functionality. 

Moreover this would be easier from a maintenance perspective as well because you need to deploy your custom servlets to just one machine. 

The only time when you need custom servlets at the node are ( to name a few )

1. Support remote file downloading ( can still be abstracted at the hub )

2. Admin actions such as shutdown node. 

3. Video recording ( can still be abstracted at the hub )

And you still haven't told us what is the problem statement that you are trying to solve wherein you need to be able to read a node configuration at the node via a plain vanilla servlet ?

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/CAOuKoX1Q88HvybuXvwSOev%2BGm-MSWkSzwdhQFn3QggGVvJ%3Dy%3DA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Praveen Tiwari]

When did I say that the node servlet is directly accessed? Seems like you don't have in depth knowledge of how and can a node work. Let me explain.

When you spin up a node, it starts a servlet container and loads predefined and extra servlets into it. Extra servlets in many implementations may not have dependencies but in some they may. If they have dependencies, there are two ways to resolve them. Either the servlet will lookup for the dependencies or hub provide them.

GridNodeConfiguration is one of those dependencies that the custom servlet would like to have during instantiation (when the node server is loading into JVM, note that node is not completely built yet) and thus can't wait for the hub to send it via one of PUT/POST request. It's also inefficient to get it from hub when the node should be able to access it from within its own container without wasting bandwidth. One of the reasons why someone would want to resolve such a dependency during instantiation is to perform expensive tasks that load into JVM certain classes that will stay there until the life of node. This will offload further sessions from performing those expensive tasks.

I know all tests send requests to hub which is the only endpoint, client is aware of. Node's servlets are always contacted by hub (because only the hub knows where are nodes located) and my setup is no exception. My custom servlet is at node because it need to access the test running in a browser just like why a video recording servlet should be at node.

I don't really think you can limit situations a custom servlet should be placed at node, there could be numerous reasons like,

- it needs to watch the browser

- it needs to save the logs before a shutdown request

- it needs to watch for node server resources like CPU/RAM usage and alert hub to take some action like spinning up a new node or shut it down

- it needs to create a persistent connection with a service to relay some information time to time.

and many more depending on the complexity of a setup

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/CANikZL%3D8h4EnGyEANL_c-nFLz5GU69MDQB8CmuZr6iPQUoXF9Q%40mail.gmail.com.

[Krishnan Mahadevan]

Praveen,

 

>>>> When did I say that the node servlet is directly accessed?

 

Which is why I have asking from the beginning, “What is your use case”. But you seem to be alluding the question for reasons better known to you [ Not my concern ]

 

>>>> Seems like you don't have in depth knowledge of how and can a node work. Let me explain.

 

Sure. You could say that. Considering the fact that I have only built a self-healing grid which is part of SeLion, and also just managed to just create an end-to-end documentation for the selenium grid, yeah I guess I still have a lot to learn ☺

But then again, we aren’t here to figure out all of that are we. But thanks for the explanation though. Appreciate it!

 

>>>>> It's also inefficient to get it from hub when the node should be able to access it from within its own container without wasting bandwidth

 

It’s a one-time request that a node makes to the hub (just as how the registration request goes out). So not sure what bandwidth wastage are you referring to here. The node servlet would just need to contact the hub once to do it.

 

>>>> My custom servlet is at node because it need to access the test running in a browser just like why a video recording servlet should be at node.

 

Well, all you had to do was say this at the beginning and the ambiguities would have been ruled out.

 

>>>>> it needs to watch for node server resources like CPU/RAM usage and alert hub to take some action like spinning up a new node or shut it down

 

I wouldn’t put in observability as one of the attributes that is to be done by the node. There are more sophisticated ways of doing this and I believe the Selenium Hub and Node also exposes some JMX beans which can be queried by tools such as Prometheus and an alerting mechanism could be setup which could inform the Hub of this and the hub could take corrective action.

 

All said and done, I believe your question has been answered.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/CAOuKoX3Sx0UnjWjN6Vzwv0T7BMmyEnUf02WYJGsjW7%2Bf6eYBhQ%40mail.gmail.com.