--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.
If the system lacks resources, Pods will remain "Pending" until resources become available. Cluster scalers may use pending pods as a signal that the cluster size should be increased.
On Mon, Oct 10, 2016 at 5:58 AM, Diego Rodríguez Rodríguez <diego...@gmail.com> wrote:
Hello, I have a doubt about how Kubernetes' jobs are handled.
I have a queue to submit certain amount of incoming tasks (Celery and RabbitMQ take care of this). Each one of this tasks is, in fact, a workflow which will be executed in a worker (Celery worker with a DAG executor inside). Each step of the workflow is a Docker image with an input file and an output file.
My question is, if I submit jobs from the workflow engine directly to the Kubernetes API, what happens I at some point there are no more resources? Will the remaining tasks be kept or will they be lost? My goal is to treat Kubernetes' jobs as a black box to submit works to. This works are of a very different and heterogeneous nature and I wouldn't need to bother with what is inside them because they are dockerized and executed by Kubernetes at some point.
To sum up, I already have the layer of Celery workers with a DAG executor inside which knows the right order of the tasks and knows how to manage everything concerning the workflow. These components will submit jobs (through Kubernetes API) and will wait for them to be executed and then continue with the remaining tasks asking Kubernetes to run them until the workflow ends.
I have read about a somehow related issue in Github:
https://github.com/kubernetes/kubernetes/pull/17787
https://github.com/kubernetes/kubernetes/pull/18827
I couldn't determine if this is closed or it is coming in a future release.
Thanks in advance!
--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
I am making reference to http://kubernetes.io/docs/api-reference/extensions/v1beta1/operations/ on the 'watch changes to an object of kind Job' section.
To be honest I have never used something like this `watch` parameter before, so I may be misunderstanding it, but I am afraid I am not.
--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/qd5Ua2UqQNo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
def get_job_state(job_id):# handle this
response = requests.get('{}{}/{}'.format('https://kubernetes',
ENDPOINT, job_id),
headers={'Authorization': 'Bearer {}'.format(
TOKEN
)},
verify=CA_CERT_PATH)
if response.ok:
res_dict = json.loads(response.text)
resource_version = res_dict['metadata']['resourceVersion']
while not (res_dict['status'].get('succeeded', False) or
res_dict['status'].get('failed', False)):
response = requests.get('{0}{1}/{2}?watch=true&resourceVersion={3}'
.format('https://kubernetes',
ENDPOINT, job_id,
resource_version),
headers={'Authorization': 'Bearer {}'.
format(
TOKEN
)},
verify=CA_CERT_PATH)
if response.ok:
res_dict = json.loads(response.text)
else:
It was just that, but I feel like I would need a library to rely on as soon as possible (I am doing everything by hand) because the actual ones are not mature enough. When do you plan to release it? I am very interested.