Testing a Django application running inside a Docker container using Selenium

[Omer Zak]

I am developing a Django application inside a Docker container (why? See my blog article at https://tddpirate.zak.co.il/2018/07/14/i-got-the-ability-to-work-with-heroku-using-my-debian-stretch-system/)

Now I want to test its browser interactions using Selenium.

To keep the container lightweight (without browser or X-Window), I run Selenium and the browser driver (chromedriver in my case) on the host.

To get it to work, I created:

class LiveServerInsideContainerTestCase(LiveServerTestCase):

    # We need the following to get Selenium to communicate with our

    # Live Server, which needs to work for us to be able to

# authenticate users in the test.

    host = '172.17.0.2'  # TODO: bring in argument properly

    port = 5000   # Would yield: self.live_server_url = "http://172.17.0.2:5000"

where the IP of the outside computer is 172.17.0.1 and the IP of the running container itself is 172.17.0.2.

I force the port to 5000 because Docker maps container's port 5000 to host's port 5000, So that the Selenium standalone server will be able to drive the browser using http://localhost:5000 as the schema+host+port parts of the URL.

The chromedriver is activated using:

./chromedriver --whitelisted-ips=172.17.0.2 &

(works for version 2.39, not for the most recent version 2.40)

The Selenium standalone is activated using:

java -jar selenium-server-standalone-3.13.0.jar  &

The problem:

The tests using Selenium are too flaky.

When they work, they pass.

When they fail, they fail due to OSError: [Errno 98] Address already in use

A sample stacktrace is as follows.

ERROR: setUpClass (activists.tests.browser_test_admin.AdminTests)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/developer/.local/share/virtualenvs/memb-Wv4pRnjx/lib/python3.6/site-packages/django/test/testcases.py", line 1297, in setUpClass

    raise cls.server_thread.error

  File "/home/developer/.local/share/virtualenvs/memb-Wv4pRnjx/lib/python3.6/site-packages/django/test/testcases.py", line 1227, in run

    self.httpd = self._create_server()

  File "/home/developer/.local/share/virtualenvs/memb-Wv4pRnjx/lib/python3.6/site-packages/django/test/testcases.py", line 1241, in _create_server

    return ThreadedWSGIServer((self.host, self.port), QuietWSGIRequestHandler, allow_reuse_address=False)

  File "/home/developer/.local/share/virtualenvs/memb-Wv4pRnjx/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 66, in __init__

    super().__init__(*args, **kwargs)

  File "/usr/lib/python3.6/socketserver.py", line 453, in __init__

    self.server_bind()

  File "/usr/lib/python3.6/wsgiref/simple_server.py", line 50, in server_bind

    HTTPServer.server_bind(self)

  File "/usr/lib/python3.6/http/server.py", line 136, in server_bind

    socketserver.TCPServer.server_bind(self)

  File "/usr/lib/python3.6/socketserver.py", line 467, in server_bind

    self.socket.bind(self.server_address)

OSError: [Errno 98] Address already in use

Possible solutions:

1. Set up a virtual machine and run my application together with Selenium inside it.

2. Set up the Docker container in such a way that all of its ports above 1024 or so are mapped to the host's ports.

However, before giving up the container's advantage of lightweightness, or figuring out how to map all container's ports, I'd like to know if anyone has another solution to the problem.

Thanks,

--- Omer Zak

-- 
The brain does not use addresses.  www.werner-seyfried.com
My own blog is at https://tddpirate.zak.co.il/

My opinions, as expressed in this E-mail message, are mine alone.
They do not represent the official policy of any organization with which
I may be affiliated in any way.
WARNING TO SPAMMERS:  at https://www.zak.co.il/spamwarning.html

[Shlomi Fish]

Hi Omer!

the plaintext part of your E-mail message is unusable - broken link, wrong
wordwrapping, etc. Perhaps see the advice at
https://github.com/shlomif/how-to-share-code-online and give us a link.

Regards,

Shlomi Fish

On Sun, 15 Jul 2018 09:27:14 +0300
Omer Zak <w...@zak.co.il> wrote:

> I am developing a Django application inside a Docker container (why?
> See my blog article at https://tddpirate.zak.co.il/2018/07/14/i-got-the
> -ability-to-work-with-heroku-using-my-debian-stretch-system/)

> Now I want to test its browser interactions using Selenium.To keep the

> container lightweight (without browser or X-Window), I run Selenium and
> the browser driver (chromedriver in my case) on the host.
> To get it to work, I created:class
> LiveServerInsideContainerTestCase(LiveServerTestCase):    # We need the
> following to get Selenium to communicate with our    # Live Server,
> which needs to work for us to be able to    # authenticate users in the
> test.    host = '172.17.0.2'  # TODO: bring in argument
> properly    port = 5000   # Would yield: self.live_server_url = "http:/
> /172.17.0.2:5000"
> where the IP of the outside computer is 172.17.0.1 and the IP of the

> running container itself is 172.17.0.2.I force the port to 5000 because

> Docker maps container's port 5000 to host's port 5000, So that the
> Selenium standalone server will be able to drive the browser using http
> ://localhost:5000 as the schema+host+port parts of the URL.

> The chromedriver is activated using:./chromedriver --whitelisted-
> ips=172.17.0.2 &(works for version 2.39, not for the most recent

> version 2.40)
> The Selenium standalone is activated using:java -jar selenium-server-
> standalone-3.13.0.jar  &
> The problem:The tests using Selenium are too flaky.When they work, they

> pass.When they fail, they fail due to OSError: [Errno 98] Address

> server_bind    self.socket.bind(self.server_address)OSError: [Errno 98]

> Address already in use
> Possible solutions:1. Set up a virtual machine and run my application

> together with Selenium inside it.2. Set up the Docker container in such

> a way that all of its ports above 1024 or so are mapped to the host's
> ports.
> However, before giving up the container's advantage of lightweightness,
> or figuring out how to map all container's ports, I'd like to know if
> anyone has another solution to the problem.

> Thanks,--- Omer Zak

>
> -- 
> The brain does not use addresses.  www.werner-seyfried.com
> My own blog is at https://tddpirate.zak.co.il/
>
> My opinions, as expressed in this E-mail message, are mine alone.
> They do not represent the official policy of any organization with which
> I may be affiliated in any way.
> WARNING TO SPAMMERS:  at https://www.zak.co.il/spamwarning.html
>

--
-----------------------------------------------------------------
Shlomi Fish http://www.shlomifish.org/
Perl Humour - http://perl-begin.org/humour/

* rindolf demands equal rights for years, minutes, hours and days to also get
YAAKOV’S GREAT HUGE LOVE.
— http://www.shlomifish.org/humour/fortunes/sharp-perl.html

Please reply to list if it's a mailing list post - http://shlom.in/reply .

[Omer Zak]

Hello Shlomi,

In spite of the problems with my previous E-mail, this mailing list
helped me solve the problem.
Udi Oron suggested that I try pyenv instead of running inside a Docker
container.

It turned out to be the solution to my pain point of running Python 3.6
in a Debian Stretch PC - and without having to use a container.
So my Selenium tests are no longer flaky.

For future users, the pyenv documentation is:

Repository:
https://github.com/pyenv/pyenv

Article about pyenv usage:
https://www.ostechnix.com/pyenv-python-version-management-made-easier/

--- Omer Zak

--
Not voting in an election is like voting for the candidate, whom you
hate the most.

[Tal Einat]

+1 for pyenv.

--

You received this message because you are subscribed to the Google Groups "PyWeb-IL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyweb-il+unsubscribe@googlegroups.com.
To post to this group, send email to pywe...@googlegroups.com.
Visit this group at https://groups.google.com/group/pyweb-il.
For more options, visit https://groups.google.com/d/optout.

[alonn]

I like and use pyenv locally but not being able to run the tests in a container in a reliable way is a big problem for CI environment, Can you please elaborate - 

is the missing port on the container or the host? perhaps this is happening because the test framework is trying to connect to the port before the process in the container finished initializing? For this kind of problems (waiting for port in another docker to be available) you can use dockerize -wait feature. 

Also I would run both selenium and app in docker containers (separate.. not the same) and use docker compose or a k8 solution (locally it might be with minikube) to connect them, exposing ports consistently. Using selenium on the host is fragile

+1 for pyenv.

To unsubscribe from this group and stop receiving emails from it, send an email to pyweb-il+u...@googlegroups.com.

[Omer Zak]

Hello Alon,

DISCLAIMER: I am writing the following from memory, so some details may
not be right.
Basically, when running tests involving Selenium, I use
LiveServerTestCase.
When it runs tests, it starts a Web server, which requests port 0 and
therefore gets assigned a free port. However, when running in a
container, we must use a fixed port. I chose port 5000 and derived the
following class:

class LiveServerInsideContainerTestCase(LiveServerTestCase):
    host = '172.17.0.2' # Docker container's IP address
    port = 5000

Then sometimes, port 5000 is free and sometimes it is in use. Probably
when tests are run in parallel.

Therefore, the solution of running the Selenium server in a separate
container won't solve the problem. Maybe starting a separate container
for each test case (and a separate instance of the live server)?

A better solution would be to run both Selenium server and the Django
application in the same Docker container. However, it would require the
container to include a full Web browser and the full X-Window
environment including a display device (real or fake).

--- Omer Zak

Everyone needs a hug.

[Maxim Kovgan]

hi, Omer.

I am possibly missing something, but pyenv, or virtualenv, or anaconda or whatever, are unrelated directly off your problem.

They help you in direction of isolating which packages you're using in your application vs. how it interacts with system python packages, but I do not see how they can help you solve this particular issue.

To your issue:

Which version of django are you using ?

IF it's below 1.11, then please consider: https://code.djangoproject.com/ticket/28212

and the documentation discusses the parallelism issues:

https://docs.djangoproject.com/en/2.0/topics/testing/tools/#django.test.LiveServerTestCase

Good night.

M.

--
You received this message because you are subscribed to the Google Groups "PyWeb-IL" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyweb-il+u...@googlegroups.com.
To post to this group, send email to pywe...@googlegroups.com.
Visit this group at https://groups.google.com/group/pyweb-il.
For more options, visit https://groups.google.com/d/optout.

--

Maxim Kovgan

[alonn]

1. Can you please share the docker file?

2. Also please share how exactly you are running the tests? Maybe I'm missing something - Does the TestCase run from "outside" the container? on the host?

3. I don't think you should do tests in parallel with Selenium.. and live server, you'll hit other problems related to state changed by one tests affecting another test

4. Also no need to set the host at localhost. this is fragile (the network address is an implementation detail of docker, and might change) basically you can just use localhost and just set the port, to the port you binded the docker container to

5. Now. if you are getting the error from outside it might mean:another instance of the tests is binding to the port - see my comment about not doing parallel with selenium. OR another unrelated process is binding to the port, which in that case running with "docker-compose" and an "inner" network" between the containers would probably solve. BUT if you are getting this error from "within" the container, then it's probably a parallelization problem catching the port. 

[Omer Zak]

Hello Maxim,

I use Django 2.0.7 with Python 3.6.5.

Since pyenv helped me run Python 3.6.5 in Debian Stretch (Debian 9.5),
I do not need containers anymore.

The reason I am still having this conversation is because Alonn wants
to solve for all the problem of running Selenium based tests with
containerized Django installations. This would allow people do CI/CD of
Django based projects.

-=-=-=-=-=-

Hello Alonn,

The Dockerfiles and scripts for building the image and running the
container are available from:
https://gitlab.com/TDDPirate/heroku_on_debian

The Django application itself is available from:
https://gitlab.com/TDDPirate/hamakormembership (the instructions are
not up-to-date; the most recent commit needs no standalone Selenium
server).

The commit which was designed to run inside the heroku_on_debian
container and run tests using a standalone Selenium server is:
https://gitlab.com/TDDPirate/hamakormembership/tree/a7d0ccd2b18c57fcafc
f291c863d461b6eafc744
I did not tag it, as I did not see any need to release this version to
users.

I believe that the ultimate solution to the problem of running Selenium
tests (which involve the Google Chrome browser - YMMV when testing on
other browsers) on a containerized application is to:

1. Allow the user to specify to chromedriver and to the LiveServer (in
LiveServerTestCase) a list or a range of ports, from which LiveServer
and chromedriver would be free to choose the first free one.
Currently, you either specify a single port (like I did) or allow any
port to be used (by specifying port 0).

2. Have the Docker image expose all ports in that list/range and bind
them to host's ports with the same number. Example (for allowing 3
Selenium tests to run in parallel):

In the Dockerfile:
EXPOSE 5000
EXPOSE 5001
EXPOSE 5002

In the start_herokudjango.sh script, change line 33 from:
-p 5000:5000 \
to:
-p 5000:5000 -p 5001:5001 -p 5002:5002 \

(DISCLAIMER: I did not actually test the above suggestion. Please let
me know if the syntax is wrong and/or there is a shorter way to say
it.)

By the way, make sure you do not use version 2.40 of chromedriver - it
has a regression when trying to specify a port. I used version 2.39.

--- Omer Zak

The key to making programs fast is to make them do practically nothing.
Mike Haertel (original author of GNU grep)