Using django on kubernetes

75 peržiūros
Praleisti ir pereiti prie pirmo neskaityto pranešimo

Andréas Kühne

neskaityta,
2018-10-30 04:28:292018-10-30
kam: django...@googlegroups.com
Hi all,

I have created a SPA with angular on the frontend and django rest framework on the backend. It also has celery to do background tasks. Everything is working as intended and it is running pretty smoothly.

We have deployed it on kubernetes - so the frontend (with nginx) is running in one pod, the backend is running in another and celery is running on a third. Everything is connected and works. My question is more about the resources settings for django. Does anyone have any experience in setting up this? Currently I am running without resource limitations - which means that the kubernetes master doesn't know how much resources the django pod needs.

There has to be someone more who has done this and has setup the resource limits correctly - I would like some inspiration. I don't know how much django requires....

Andréas

PASCUAL Eric

neskaityta,
2018-10-30 06:55:042018-10-30
kam: django...@googlegroups.com

Hi,


Django is not really the key factor here, and the amount of consumed resources depends mainly on what your business logic does on top of Django.


IMHO the only way to properly estimate the limits is observing the metrics and trends of the related pods for a significant period and with an application load representative of the operational conditions. You'll have then rough estimates of how much RAM, CPU,... is used by the pods, and be able to set the limits correctly.


Maybe not question was not exactly this.


Eric


From: django...@googlegroups.com <django...@googlegroups.com> on behalf of Andréas Kühne <andrea...@hypercode.se>
Sent: Tuesday, October 30, 2018 9:27:35 AM
To: django...@googlegroups.com
Subject: Using django on kubernetes
 
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCf2d5fkhcG_g6oy_PY7yLcLtsV8fSsc%3D%2BFFOfdmGrkThA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Jason

neskaityta,
2018-10-30 07:16:402018-10-30
kam: Django users
using infrastructure monitoring such as new relic can lend valuable insight into what resources pods are using vs their default allocation

Andréas Kühne

neskaityta,
2018-10-30 08:26:022018-10-30
kam: django...@googlegroups.com
Thanks for the replys,

I am of course monitoring the resources - but I am a bit afraid that the memory usage for example might change over time? Ah well - I will have to monitor this for an extended period of time and see what happens then :-)

Regards,

Andréas


Den tis 30 okt. 2018 kl 12:17 skrev Jason <jjohn...@gmail.com>:
using infrastructure monitoring such as new relic can lend valuable insight into what resources pods are using vs their default allocation

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

PASCUAL Eric

neskaityta,
2018-10-30 10:09:462018-10-30
kam: django...@googlegroups.com

but I am a bit afraid that the memory usage for example might change over time?


It will of course, but apart if you have some memory leak somewhere, it will tend to stabilize over time.


Ah well - I will have to monitor this for an extended period of time and see what happens then :-)


For sure. I can't see any other way to have a realistic assessment otherwise.


Once you'll get some figures, you can elaborate the limits by adding a bit of headroom (let's say 20% more for instance). You can reduce this amount if you are short, or if you observe that the situation is rather stable, but in this case there's a risk for K8S to kill and restart your pods too often. In any case, it's a matter of compromises.


Regards


Eric


Sent: Tuesday, October 30, 2018 1:25:11 PM
To: django...@googlegroups.com
Subject: Re: Using django on kubernetes
 

Dan Davis

neskaityta,
2018-10-30 13:56:332018-10-30
kam: Django users
Andreas,

I don't know terribly much about Kubernetes, only Docker, however it seems that Kubernetes must natively provide some metrics collection, i.e https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/.   It would be nice to correlate particular views and their arguments with resource use.   If you are using a process model, not a threading model, then I think the Linux system call getrusage() could do that, providing that it is supported in Kubernetes containers.    I'm not even sure those are the same as Docker containers even though Kubernetes can run Docker images.  Maybe you can educate me!

Anyway, the package django-statsd might provide some help collecting APM data without something like NewRelic, but if you can use a real APM, do it.

PASCUAL Eric

neskaityta,
2018-10-30 17:59:182018-10-30
kam: Django users

I'm not even sure those are the same as Docker containers even though Kubernetes can run Docker images.


Kubernetes is an orchestrator for Docker containers, not a container engine. You can run the same images in K8S managed containers or on your local Docker engine, using docker-compose for instance. I currently work on a project related to a services platform based on micro-services deployed in Docker containers. I test the images locally on my machine either in docker-compose assemblies or in Minikube (for validating the K8S descriptors involved in deployment, configuration,...) and then I deploy the stuff on GCP.


As already mentioned, K8S provides tools (indicators, graphs,...) to monitor the resources used by pods. I would not use Linux metrics, if ever they were representative when collected from inside a pod, since the containers are running on VMs and they can be spread over different nodes if your configuration involves multi-nodes load balancing.


Eric


From: django...@googlegroups.com <django...@googlegroups.com> on behalf of Dan Davis <dans...@gmail.com>
Sent: Tuesday, October 30, 2018 6:56:32 PM
To: Django users

Subject: Re: Using django on kubernetes
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

Dan Davis

neskaityta,
2018-11-01 16:14:362018-11-01
kam: Django users
Thanks, would you say that running Minikube is the best way to learn Kubernetes at a significant level, or would you recommend a small Virtualbox/vagrant setup that really is a multi-node Kubernetes?   I don't really have enough cores and memory for the later anyway, but I could just start-up workloads on Amazon KCS or GCP to learn, if Minikube is not enough.
To post to this group, send email to djang...@googlegroups.com.

PASCUAL Eric

neskaityta,
2018-11-02 14:39:062018-11-02
kam: Django users

My opinion is that Minikube is the easiest way, since it is quite simple to set up and run. Behind the curtains it uses a VirtualBox machine BTW if I'm not mistaken (at least under Linux).


You can of course set up a multi-node stack using several VMs (after all, this is exactly what hosters do) but I'm afraid you'll fight a lot before having it run properly.


If you have access to GCP resources, you can go for it of course. But you'll have to master a couple of GCP specific topics in addition to Kubernetes ones.


So the bottom line if I were you : start learning by practising with Minikube, and when you'll get used to K8S concepts and YAML descriptors, move a step forward with GCP for instance if you need to.


Regards


Eric


Sent: Thursday, November 1, 2018 9:14:36 PM
To post to this group, send email to django...@googlegroups.com.
Atsakyti visiems
Atsakyti autoriui
Persiųsti
0 naujų pranešimų