Using Selenium Grid with AWS ECS

[Chris Smith]

Has anyone got selenium grid working with AWS ECS? Kubernetes has a guide  https://github.com/kubernetes/kubernetes/tree/master/examples/selenium that I tried to adapt for ECS but am having some difficultly with port mappings. 

[Dylan Reichstadt]

Hey Chris,

I haven't moved our containers over to ECS yet, but I'm assuming you've seen the Selenium Docker repository?

https://github.com/SeleniumHQ/docker-selenium

These are pretty easy to run on EC2. It might be worth first trying it on EC2, and then working on adapting it to ECS.

The only thing I could imagine making the move to ECS difficult would be setting the REMOTE_HOST parameter to your box's IP. I believe REMOTE_HOST is needed as these containers will be running on different machines.

ECS I don't believe has a way to set the box's IP as an environment variable directly. More context on that here.

You may have to modify the Dockerfile when setting the REMOTE_HOST to:

echo `curl -s http://169.254.169.254/latest/meta-data/local-ipv4`

to get the host's IP address.

If you have more questions around Docker and Selenium Grid, I'd suggest watching this from Selenium Conf

https://www.youtube.com/watch?v=S4OkrnFb-YY

[Chris Smith]

Hey Dylan,

Happy to report I successfully got selenium grid running on ECS. I'm currently using 2.53.1 instead of 3.0.0 because of this issue https://github.com/SeleniumHQ/docker-selenium/issues/307#issuecomment-254314297

Like you said, I needed to get the EC2 instances external IP in my entry point script 

export EC2_HOST=$(wget -O - http://169.254.169.254/latest/meta-data/local-ipv4 2> /dev/null)
export REMOTE_HOST="http://$EC2_HOST:5555"
bash -x /opt/bin/entry_point.sh

Right now I'm forced to use a statically defined port mapping which I don't like though. I'm looking for ways to allow ECS to dynamically assign me a port mapping but retrieve it from within the container. It looks like docker doesn't expose this currently with access to either the remote api or querying the ECS api.  

Chris

[Steven Senatori]

Hy Chris, just wondering if you found a solution or workaround of some sort in regards to ECS not supporting dynamic port mapping? Currently facing a similar issue since i'd like to be able to scale up the nodes as needed.

[Chris Smith]

Yes I ended up implenting a solution for this. I posted a gist in this issue on the ECS agent repo https://github.com/aws/amazon-ecs-agent/issues/151#issuecomment-268048078

Using that script you can pass the correct port to selenium

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/PYX1fYqNGqc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/16d5756b-7b67-45ce-997e-85702c4468e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mahesh Goud]

Hey Chris,

I am trying to set up the remote hub using the Entrypoint but i am not successful, appreciate if you can help me. below is my Entrypoint.

Entrypoint	["/bin/bash","-c","-e HUB_PORT_4444_TCP_ADDR=xx.xxx.xx.xx","-e HUB_PORT_4444_TCP_PORT=4444","-e SE_OPTS=\"-host xx.xxx.xx.xx -port 5900\""]

Thanks,

Mahesh

[Danijel Hrvaćanin]

Hi.

I am trying to run selenium hub and node on AWS ECS, but somehow I got stuck becuse the Hub host is set to "null" even I am passing HUB_PORT_4444_TCP_ADDR value to docker image and later the node is not able to register itself on hub. I have also used ALB (HTTP/HTTPS), but maybe I have set it up wrong. It would be nice if someone already did manage to set Selenium on ECS and could post some YAML or JSON snippet as how it is working. Or atleast some partial steps what needs to be done for defining ECS Tasks and Services, and ALB.

Best regards.

[Diego Fernando Molina]

Hi Danijel,

Someone asked something similar recently in the repo, please check if my answer also helps you:

https://github.com/SeleniumHQ/docker-selenium/issues/607

[Roberto Rivera]

My team recently managed to get selenium working on ECS. Ended up submitting a PR to selenium to expose a -remoteHost flag.

The gist of it is that the REMOTE_HOST env var -- that could be used to override the node's address -- was removed and it's been recently added back as a command-line argument.

We used a modified version of Chris Smith's gist to get the port mappings, and with that flag we can get the server to connect to the nodes.

Reference implementation was open-sourced: https://github.com/RetailMeNotSandbox/ecs-selenium