Deploying kubernetes on a 20 node cluster

[Sidharta Seethana]

Hello,

So far, I have been playing with a tiny kubernetes setup (vagrant - 3-4 minions). I now have access to a small, on-premise test cluster (~20 nodes) on which I want to deploy Kubernetes : 

- The cluster is running an older version of Centos (6.3). The kernel has been upgraded to 2.6.32-504.1.3.el6.x86_64 in order to enable docker support. At this point, I'l be unable to upgrade/change the operating system. 

- One node in the cluster has pdsh setup so that remote shell commands can be run in parallel on the cluster 

- there is no fleet/salt-like system installed on this cluster

- Also, the configured centos repos don't seem to have kubernetes/etcd/cadvisor etc available. However, Salt/Docker (docker-io) packages seem to be available. 

Given this "state" of the cluster : 

     a) how do I go about setting up/configuring kubernetes? Would salt be the best bet? If not, what else? 

     b) Is there any documentation on how to go about setting up networking on such a cluster ( weave? flannel? etc ) ? 

None of the docs in the getting-started-guides folder seem to be a fit. I could attempt to setup each component separately in the "right" sequence - but it seems like a time-consuming task. Has anybody setup such a cluster before? Any information/pointers to docs would be great!

thanks,

-Sidharta

[Brendan Burns]

Eric Paris' ansible + kubernetes:

https://github.com/eparis/kubernetes-ansible

might be a good place to start from.

Also, what is the network for these machines?  Is it a single dumb switch?  Is there a router involved?  That's going to determine what you need to do to get networking working.

Please keep us up to date as you start trying this out.  we're happy to help w/ the debugging as you get closer to a working setup.

--brendan

--
You received this message because you are subscribed to the Google Groups "Containers at Google" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-contain...@googlegroups.com.
To post to this group, send email to google-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-containers.
For more options, visit https://groups.google.com/d/optout.

[Sidharta Seethana]

Thanks, Brendan. I'll take a look at the ansible play books. 

About the networking setup : yes, there is a router in front and not a dumb switch. 

I'll keep you guys posted. 

[Brendan Burns]

ok, with the router, you'll need to program it so that it sends the appropriate subnets to each machine.  Or you'll need to use a virtualization layer like weave or flannel.  Likely easier to use weave/flannel, but you'll pay a price in network performance.

--brendan

[Sidharta Seethana]

I can't mess with the router configuration for the time being. I'll take a look at flannel, thanks. 

[Paris, Eric]

The Fedora or RHEL7 kube rpms should install and work just fine, you
could rebuild the fedora srpms and copy around the result yourself.
You can even build kube from source and copy the binaries to each
machine. Up to you (ansible makes that pretty manageable)

Flannel is very good at getting you a basic overlay network which kube
pods can use to talk to each other with darn near 0 config.
Connectivity to things inside the pod from outside isn't so easy,
we're still batting around ideas there.

On Tue, Dec 9, 2014 at 7:07 PM, Sidharta Seethana

[Sidharta Seethana]

Thanks, Eric. I'll take a look at the spec files you pointed to in the other thread and figure out next steps.