Pyinotify and runserver performance with django channels in docker for mac

[qnub]

Not sure what exactly wrog here, but installing pyinotify not helps to lower CPU usage in docker on mac OS. As i understand after googling installing of `pyinotify` helps in case of running dev server with `runserver` manage command for general django configuration (without channels, but i not checked it because i need channels), but looks like it's changes nothing when you use `runserver` with `channels`.

What i've made — i've created docker (using latest docker for mac) container with image ubuntu/xenial x64 to execute project. And each page (even in admin panel) loads extremely long (about couple of minutes). If i use `--noreload` option it works fast (couple of seconds or less for page load). I mean loading without cache. I've checked in docker container with `inotifywait` and looks like signals works correctly for file changes, but project works slow even with installed `pyinotify` globally and in project virtual environment.

[Andrew Godwin]

Can I ask what channel layer you are using? That's what affects the speed of messages being transported.

Andrew

On Thu, Jun 1, 2017 at 1:36 AM, qnub <qnu...@gmail.com> wrote:

Not sure what exactly wrog here, but installing pyinotify not helps to lower CPU usage in docker on mac OS. As i understand after googling installing of `pyinotify` helps in case of running dev server with `runserver` manage command for general django configuration (without channels, but i not checked it because i need channels), but looks like it's changes nothing when you use `runserver` with `channels`.

What i've made — i've created docker (using latest docker for mac) container with image ubuntu/xenial x64 to execute project. And each page (even in admin panel) loads extremely long (about couple of minutes). If i use `--noreload` option it works fast (couple of seconds or less for page load). I mean loading without cache. I've checked in docker container with `inotifywait` and looks like signals works correctly for file changes, but project works slow even with installed `pyinotify` globally and in project virtual environment.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/61650182-e93b-4a68-ade8-58c3ea18ef31%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[qnub]

Yes, because it's dev environment with single node but with `rundelay` we use `asgi_ipc.IPCChannelLayer`

On Thursday, June 1, 2017 at 11:03:09 PM UTC+6, Andrew Godwin wrote:

Can I ask what channel layer you are using? That's what affects the speed of messages being transported.

Andrew

On Thu, Jun 1, 2017 at 1:36 AM, qnub <qnu...@gmail.com> wrote:

Not sure what exactly wrog here, but installing pyinotify not helps to lower CPU usage in docker on mac OS. As i understand after googling installing of `pyinotify` helps in case of running dev server with `runserver` manage command for general django configuration (without channels, but i not checked it because i need channels), but looks like it's changes nothing when you use `runserver` with `channels`.

What i've made — i've created docker (using latest docker for mac) container with image ubuntu/xenial x64 to execute project. And each page (even in admin panel) loads extremely long (about couple of minutes). If i use `--noreload` option it works fast (couple of seconds or less for page load). I mean loading without cache. I've checked in docker container with `inotifywait` and looks like signals works correctly for file changes, but project works slow even with installed `pyinotify` globally and in project virtual environment.

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

[Andrew Godwin]

OK, can you try using the Redis one instead and seeing if that's faster? Docker for Mac has a bit of an odd filesystem and kernel and it's possible the IPCLayer is not working well underneath it.

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/937cb35a-8e97-450d-9e0c-c2ce766d7e78%40googlegroups.com.

[qnub]

Thank you, finally i've checked it and looks like it helps (hope it works faster not because of my hardware upgrade but because of Reis usage). So with redis page loads in 10 seconds instead of 1,5 minutes with IPC.

On Thursday, June 1, 2017 at 11:33:28 PM UTC+6, Andrew Godwin wrote:

OK, can you try using the Redis one instead and seeing if that's faster? Docker for Mac has a bit of an odd filesystem and kernel and it's possible the IPCLayer is not working well underneath it.

Andrew

On Thu, Jun 1, 2017 at 10:11 AM, qnub <qnu...@gmail.com> wrote:

Yes, because it's dev environment with single node but with `rundelay` we use `asgi_ipc.IPCChannelLayer`

On Thursday, June 1, 2017 at 11:03:09 PM UTC+6, Andrew Godwin wrote:

Can I ask what channel layer you are using? That's what affects the speed of messages being transported.

Andrew

On Thu, Jun 1, 2017 at 1:36 AM, qnub <qnu...@gmail.com> wrote:

Not sure what exactly wrog here, but installing pyinotify not helps to lower CPU usage in docker on mac OS. As i understand after googling installing of `pyinotify` helps in case of running dev server with `runserver` manage command for general django configuration (without channels, but i not checked it because i need channels), but looks like it's changes nothing when you use `runserver` with `channels`.

What i've made — i've created docker (using latest docker for mac) container with image ubuntu/xenial x64 to execute project. And each page (even in admin panel) loads extremely long (about couple of minutes). If i use `--noreload` option it works fast (couple of seconds or less for page load). I mean loading without cache. I've checked in docker container with `inotifywait` and looks like signals works correctly for file changes, but project works slow even with installed `pyinotify` globally and in project virtual environment.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/61650182-e93b-4a68-ade8-58c3ea18ef31%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

[Andrew Godwin]

10 seconds is still very slow, on my computer it takes around 300 milliseconds (0.3 seconds) for the worst case render. You must have something else installed/configured that is affecting it?

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/58f6c5ce-62bf-4eda-a0ed-58fd5cf65f24%40googlegroups.com.

[qnub]

On developers machines we use postgres (and BDR on production, but i have not performance issues on production anymore), redis (now on developers too, on production it was from the start) and elasticsearch. All this things executed in own docker containers (on developer machines and on production). But all even elasticsearch should not have affect because now i'm talking about loading page in Django admin. So here is nothing to update in elasticsearch (BTW, i've made async ES indexation using channels :D) and all what is using is dev instance running with `manage.py runserver` (daphne and 4 workers by default as i understand), `manage.py rundelay` in separated process (both in same container running with supervisor), redis for channels and postgres 9.4.

Oh, i can link packages installed inside project's docker:

        pt-transport-https 

        ca-certificates 

        cron 

        g++ 

        gcc 

        gfortran 

        git 

        gosu 

        inotify-tools 

        iputils-ping 

        libatlas-base-dev 

        libfreetype6-dev 

        libinotifytools0-dev 

        libjpeg8-dev 

        liblcms2-dev 

        libmagickwand-dev 

        libopenjpeg-dev 

        libpq-dev 

        libwebp-dev 

        libxml2-dev 

        libxslt1-dev

        locales 

        ltrace 

        net-tools 

        python-dev 

        python-pip 

        python3-pip 

        python3.5 

        python3.5-dev 

        sendmail 

        software-properties-common 

        strace 

        sudo 

        supervisor 

        wget 

        zlib1g-dev

I have posted here before my issue with CPU usage by channels, but it was on production and because of used RedisLocal layer version, so many messages hangs and overflow channels. Now production is blazing fast.