Locust master doesn't see slaves in Kubernetes cluster
mder...@gmail.com
I don't know why (I'm going crazy trying to understand the reason) but from today I have a problem with my Locust master that always returns this error:
[2018-01-09 14:17:26,379] locust-master-deployment-262643481-9kfw8/INFO/locust.main: Starting web monitor at *:8089
[2018-01-09 14:17:26,381] locust-master-deployment-262643481-9kfw8/INFO/locust.main: Starting Locust 0.8.1
[2018-01-09 14:18:23,948] locust-master-deployment-262643481-9kfw8/WARNING/locust.runners: You are running in distributed mode but have no slave servers connected. Please connect slaves prior to swarming.
From two weeks I was able to get working Locust cluster without problem using Kubernetes 1.8.1
Yesterday I deleted my locust cluster and today I created a new one with Kubernetes 1.8.4
And I always get this damn error.
My deployment is very easy:
1) a config file for locust task
###########################################
apiVersion: v1
kind: ConfigMap
metadata:
name: locust-new-configmap
data:
basic.py: |
from locust import HttpLocust, TaskSet, task
def index(l):
l.client.get("/")
def contatti(l):
l.client.get("/contatti")
def privacy(l):
l.client.get("/privacy")
class UserTasks(TaskSet):
tasks = [index,contatti,privacy]
class WebsiteUser(HttpLocust):
"""
Locust user class that does requests to the locust web server running on localhost
"""
host = "http://127.0.0.1:8089"
min_wait = 2000
max_wait = 5000
task_set = UserTasks
###########################################
2) a deployment for locust master
###########################################
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: locust-master-deployment
spec:
replicas: 1
template:
metadata:
labels:
name: locust
role: master
spec:
hostAliases:
- ip: "123.123.123.123"
hostnames:
- "www.mysite.it"
volumes:
- name: locust-volume
configMap:
name: locust-new-configmap
containers:
- name: locust
image: mylocust-image/locust:v0.8.1
env:
- name: LOCUST_MODE
value: master
- name: LOCUST_LOCUSTFILE_PATH
value: "/locust-tasks/basic.py"
- name: LOCUST_TARGET_HOST
value: "https://www.mysite.it"
volumeMounts:
- name: locust-volume
mountPath: /locust-tasks
ports:
- containerPort: 8089
- containerPort: 5557
- containerPort: 5558
###########################################
3) a deployment for locust slave
###########################################
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: locust-slave-deployment
spec:
replicas: 10
template:
metadata:
labels:
name: locust
role: slave
spec:
hostAliases:
- ip: "123.123.123.123"
hostnames:
- "www.mysite.it"
volumes:
- name: locust-volume
configMap:
name: locust-new-configmap
containers:
- name: locust
image: mylocust-image/locust:v0.8.1
env:
- name: LOCUST_MODE
value: slave
- name: LOCUST_MASTER
value: locust-master
- name: LOCUST_LOCUSTFILE_PATH
value: "/locust-tasks/basic.py"
- name: LOCUST_TARGET_HOST
value: "https://www.mysite.it"
volumeMounts:
- name: locust-volume
mountPath: /locust-tasks
###########################################
4) a load balancer
apiVersion: v1
kind: Service
metadata:
name: locust-loadbalancer
labels:
name: locust
spec:
type: LoadBalancer
selector:
name: locust
role: master
ports:
- port: 8089
protocol: TCP
name: master-web
- port: 5557
protocol: TCP
name: master-port1
- port: 5558
protocol: TCP
name: master-port2
I tried also with Kubernetes 1.7.11 but no success :-(
mder...@gmail.com
FROM python:3.6.2
# Install packages
COPY requirements.txt /tmp/
RUN pip install --requirement /tmp/requirements.txt
# Add tasks directory
COPY locust-tasks /locust-tasks
# Set the entrypoint
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 5557 5558 8089
[docker-entrypoint.sh]
#!/bin/bash
LOCUST_MODE=${LOCUST_MODE:="standalone"}
LOCUST_MASTER=${LOCUST_MASTER:=""}
LOCUST_LOCUSTFILE_PATH=${LOCUST_LOCUSTFILE_PATH:="/locust-tasks/basic.py"}
LOCUST_LOCUSTFILE_URL=${LOCUST_LOCUSTFILE_URL:=""}
LOCUST_TARGET_HOST=${LOCUST_TARGET_HOST:="https://example.com"}
if [ ! -z "$LOCUST_LOCUSTFILE_URL" ]; then
LOCUST_LOCUSTFILE_PATH="/locust-tasks/locustfile.py"
curl $LOCUST_LOCUSTFILE_URL -o $LOCUST_LOCUSTFILE_PATH
fi
LOCUST_PATH="/usr/local/bin/locust"
LOCUST_FLAGS="-f $LOCUST_LOCUSTFILE_PATH --host=$LOCUST_TARGET_HOST"
if [[ "$LOCUST_MODE" = "master" ]]; then
LOCUST_FLAGS="$LOCUST_FLAGS --master"
elif [[ "$LOCUST_MODE" = "slave" ]]; then
LOCUST_FLAGS="$LOCUST_FLAGS --slave --master-host=$LOCUST_MASTER"
fi
exec $LOCUST_PATH $LOCUST_FLAGS
[requirements.txt]
Flask==0.12.2
gevent==1.2.2
greenlet==0.4.12
itsdangerous==0.24
Jinja2==2.9.6
locustio==0.8.1
MarkupSafe==1.0
msgpack-python==0.4.8
pyzmq==16.0.2
requests==2.18.4
Werkzeug==0.12.2
[locust-tasks/basic.py]
from locust import HttpLocust, TaskSet, task
def index(l):
l.client.get("/")
def stats(l):
l.client.get("/stats/requests")
class UserTasks(TaskSet):
# one can specify tasks like this
tasks = [index, stats]
# but it might be convenient to use the @task decorator
@task
def page404(self):
self.client.get("/does_not_exist")
mder...@gmail.com
In order to work with the newer version of Kubernetes is required that the metadata.name of master deployment is equal also in slave deployment and loadbalancer service.
So the code becomes for example:
[master deployment]
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: locust-master-deployment
spec:
replicas: 1
template:
metadata:
labels:
name: locust
role: master
...
[slave deployment]
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: locust-slave-deployment
spec:
replicas: 10
template:
metadata:
labels:
name: locust
role: slave
spec:
....
containers:
- name: locust
image: mylocust-image/locust:v0.8.1
env:
- name: LOCUST_MODE
value: slave
- name: LOCUST_MASTER
value: locust-master-deployment
- name: LOCUST_LOCUSTFILE_PATH
value: "/locust-tasks/basic.py"
- name: LOCUST_TARGET_HOST
value: "https://www.mysite.it"
...
[load balancer service]
apiVersion: v1
kind: Service
metadata:
name: locust-master-deployment
labels:
name: locust
...