Can I launch Google Container Engine (GKE) in Private GCP network Subnet?

[cybage.a...@gmail.com]

I'm trying to launch Google Container Engine (GKE) in Private GCP network Subnet.

I have created custom Google Cloud VPC, then I have created custom Private Network Access Subnet too under that VPC.

1) When I create GKE cluster with Private Subnet, still my Kubernetes nodes assigned with Public IP. Why it is so ? As per Google Document private instance should get Private IP.

2) If I create cluster in Private, can I connect my container application to Google SQL instance ?

3) Is any recommendation to launch GKE cluster should launched in Public Subnet only, not in Private Subnet ?

[Tim Hockin]

GKE relies on public IPs to access the hosted master, for now.

> --
> You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
> To post to this group, send email to kubernet...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.

[cybage.a...@gmail.com]

Thank you very much Tim, for your reply!!

So for now GKE cluster we have to launch in Public Subnet. Right?

Observation : We have launched GKE cluster in Private Subnet, but GKE Cluster is behaving same as if its launch in Public Subnet.

[Tim Hockin]

How did you create these private/public subnets? GKE should not allow
you to create a cluster in a Network that doesn't have a default route
to internet. Routes are per-Network, not per-SubNetwork.

[Cybage ALM]

I have created custom Google Cloud VPC, then created custom Private Network Access Subnet under that VPC.

Below are Setting of Private Subnet

Then while creating GKE Cluster we selecting Network as Custom Created VPC and Subnetwork which we have created Private Subnet as above.

These are steps we have followed to launch cluster in Private Subnet.

On Wed, Jul 19, 2017 at 10:32 PM, 'Tim Hockin' via Kubernetes user discussion and Q&A <kubernet...@googlegroups.com> wrote:

How did you create these private/public subnets?  GKE should not allow
you to create a cluster in a Network that doesn't have a default route
to internet.  Routes are per-Network, not per-SubNetwork.

On Tue, Jul 18, 2017 at 11:27 PM, nnile...@gmail.com
<cybage.a...@gmail.com> wrote:
> On Tuesday, July 18, 2017 at 8:52:47 PM UTC+5:30, Tim Hockin wrote:
>> GKE relies on public IPs to access the hosted master, for now.
>>
>> On Tue, Jul 18, 2017 at 1:46 AM, nnile...@gmail.com
>> <cybage.a...@gmail.com> wrote:
>> > I'm trying to launch Google Container Engine (GKE) in Private GCP network Subnet.
>> >
>> > I have created custom Google Cloud VPC, then I have created custom Private Network Access Subnet too under that VPC.
>> >
>> > 1) When I create GKE cluster with Private Subnet, still my Kubernetes nodes assigned with Public IP. Why it is so ? As per Google Document private instance should get Private IP.
>> >
>> > 2) If I create cluster in Private, can I connect my container application to Google SQL instance ?
>> >
>> > 3) Is any recommendation to launch GKE cluster should launched in Public Subnet only, not in Private Subnet ?
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.

>> > To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.
>> > To post to this group, send email to kubernetes-users@googlegroups.com.

>> > Visit this group at https://groups.google.com/group/kubernetes-users.
>> > For more options, visit https://groups.google.com/d/optout.
>
> Thank you very much Tim, for your reply!!
>
> So for now GKE cluster we have to launch in Public Subnet. Right?
>
> Observation : We have launched GKE cluster in Private Subnet, but GKE Cluster is behaving same as if its launch in Public Subnet.
>
> --
> You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.

> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/YK0JNgTIHyI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.

[Tim Hockin]

Private Google Access is not a private subnet.  That simply allows your VMs to access google service without a public IP.  You still have to make VMs without a public IP, which GKE does not support yet.

[dbg...@gmail.com]

On Friday, July 28, 2017 at 11:52:27 AM UTC+5:30, Tim Hockin wrote:
> Private Google Access is not a private subnet.  That simply allows your VMs to access google service without a public IP.  You still have to make VMs without a public IP, which GKE does not support yet.
>
>
> On Wed, Jul 19, 2017 at 12:14 PM, Cybage ALM <cybage.a...@gmail.com> wrote:
>
> I have created custom Google Cloud VPC, then created custom Private Network Access Subnet under that VPC.
>
>
> Below are Setting of Private Subnet
>
>
>
>
>
>
>
>

> Then while creating GKE Cluster we selecting Network as Custom Created VPC and Subnetwork which we have created Private Subnet as above.
>
>
> These are steps we have followed to launch cluster in Private Subnet.
>
>
>
>
>
>
>
>
> On Wed, Jul 19, 2017 at 10:32 PM, 'Tim Hockin' via Kubernetes user discussion and Q&A <kubernet...@googlegroups.com> wrote:
>
>
> How did you create these private/public subnets?  GKE should not allow
>
> you to create a cluster in a Network that doesn't have a default route
>
> to internet.  Routes are per-Network, not per-SubNetwork.
>
>
>
>
>
>
>
> On Tue, Jul 18, 2017 at 11:27 PM, nnile...@gmail.com
>
> <cybage.a...@gmail.com> wrote:
>
> > On Tuesday, July 18, 2017 at 8:52:47 PM UTC+5:30, Tim Hockin wrote:
>
> >> GKE relies on public IPs to access the hosted master, for now.
>
> >>
>
> >> On Tue, Jul 18, 2017 at 1:46 AM, nnile...@gmail.com
>
> >> <cybage.a...@gmail.com> wrote:
>
> >> > I'm trying to launch Google Container Engine (GKE) in Private GCP network Subnet.
>
> >> >
>
> >> > I have created custom Google Cloud VPC, then I have created custom Private Network Access Subnet too under that VPC.
>
> >> >
>
> >> > 1) When I create GKE cluster with Private Subnet, still my Kubernetes nodes assigned with Public IP. Why it is so ? As per Google Document private instance should get Private IP.
>
> >> >
>
> >> > 2) If I create cluster in Private, can I connect my container application to Google SQL instance ?
>
> >> >
>
> >> > 3) Is any recommendation to launch GKE cluster should launched in Public Subnet only, not in Private Subnet ?
>
> >> >
>
> >> > --
>
> >> > You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
>

> >> > To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
>
> >> > To post to this group, send email to kubernet...@googlegroups.com.

>
> >> > Visit this group at https://groups.google.com/group/kubernetes-users.
>
> >> > For more options, visit https://groups.google.com/d/optout.
>
> >
>
> > Thank you very much Tim, for your reply!!
>
> >
>
> > So for now GKE cluster we have to launch in Public Subnet. Right?
>
> >
>
> > Observation : We have launched GKE cluster in Private Subnet, but GKE Cluster is behaving same as if its launch in Public Subnet.
>
> >
>
> > --
>
> > You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
>

> > To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
>
> > To post to this group, send email to kubernet...@googlegroups.com.

>
> > Visit this group at https://groups.google.com/group/kubernetes-users.
>
> > For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
>
> You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
>
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/YK0JNgTIHyI/unsubscribe.
>

> To unsubscribe from this group and all its topics, send an email to kubernetes-use...@googlegroups.com.
>
> To post to this group, send email to kubernet...@googlegroups.com.

>
> Visit this group at https://groups.google.com/group/kubernetes-users.
>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
>

> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
>
> To post to this group, send email to kubernet...@googlegroups.com.

>
> Visit this group at https://groups.google.com/group/kubernetes-users.
>
> For more options, visit https://groups.google.com/d/optout.

Are there any near plan to have GKE working in Private network ? I don't want to expose my containers to public IPs

[Tim Hockin]

On Fri, Oct 13, 2017 at 3:17 AM, <dbg...@gmail.com> wrote:
> On Friday, July 28, 2017 at 11:52:27 AM UTC+5:30, Tim Hockin wrote:
>> Private Google Access is not a private subnet. That simply allows your VMs to access google service without a public IP. You still have to make VMs without a public IP, which GKE does not support yet.
>

> Are there any near plan to have GKE working in Private network ? I don't want to expose my containers to public IPs

We are evaluating how best to support this. In the mean time, it's
important to note that none of your containers are exposed by default,
they do not have external IPs, and with the exception of the nodes'
SSH port, all the default GCP firewalls default to "closed". The only
"public" traffic required is GKE masters <-> nodes, and that is only
"public" in name. The traffic stays withing Google's network.

Tim

[adit...@media.net]

I would like to give this thread a bump and love to know if there is any update.
It is not uncommon to allow access to a service by whitelisting the public ip. Each kubernetes node having its own public ip makes a mess. Right now, only solution seems to be running a NAT instance[1]. GCP doesn't provide NAT gateway as service either, so one would have to deal with scaling and high availability themselves.

[1] https://cloud.google.com/solutions/using-a-nat-gateway-with-kubernetes-engine

[Tim Hockin]

NB there are two issues here:

1) how to run a cluster where the VMs have no public IP, and the node
<-> master comms are private IP.

2) how to run a cluster with long-term-stable egress IPs.

They are not the same issue, despite being related :)

Tim

[manjo...@google.com]

Hi,

GKE now supports private clusters :-)
https://cloudplatform.googleblog.com/2018/03/kubernetes-engine-private-clusters-now.html

Hope that helps!

[adit...@media.net]

Hey, this is great news. Thanks for update.

[Tim Hockin]

Private cluster is private by default.  You can not access the master from the internet.  You can specifically change that with the master authorized networks feature, or you can access it from within your VPC network.

On Thu, Mar 29, 2018 at 10:42 PM Vinita <vjo...@etouch.net> wrote:

Hi,

I am trying to use private cluster. I am able to create private cluster but kubectl commands are not working. I am seeing connection time out error as below -

kubectl run nginx --image=nginx --replicas=2error: failed to discover supported resources: Get https://104.154.200.217/api: dial tcp 104.154.200.217:443: i/o timeout
Am I missing something. I am seeing this issue in my SDK as well as Cloud shell.Thanks

[Vinita]

Hello Tim,

Thank for your reply. I tried to access mater from a VM in the same VPC network by adding it's internal IP to master authorized network but I could not access it. I was able to access it if I add external IP of the VM in authorized network. Is this expected behavior? 

Thanks,

Vinita