Re: [kubernetes/kubernetes] kubectl port-forward should allow forwarding to a Service (#15180)

[Michail Kargakis]

@kubernetes/sig-cli-feature-requests

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[Shahidh K Muhammed]

We are looking at this issue as it is a pressing use case for us. Currently we use port-forward to a pod, but when he service restarts, the tunnel breaks. I'm thinking of possible ways to implement a persistent tunnel to a service.

Taking service name as an argument and getting​ corresponding pod based on that to forward the port seems like the easiest one to implement. Restarts and multiple pods have to be handled though.

What about the kubectl proxy command? I'm quite confused since this one is more closer to proxying services to the client machine. Would adding port support here makes more sense than extending port-forward? Something like kubectl proxy namespace/postges --port 5432:5432

Long time k8s user, but new to the codebase. Sorry if I'm missing something out

[Jean Mertz]

This would definitely be a nice feature to have. We currently have to do things like:

kubectl port-forward $(kubectl get pod -o=name -lcomponent=... | awk -F/ '{print $2}' | head -1) PORT

to get a single pod. It would be better (and also more resilient, since the above command always takes the first pod, no matter the state) to go through a service instead.

And as @shahidhk stated, obviously having the benefits of keeping a connection alive, even when a pod restarts would be a great feature to have.

[Alexandre Bourget]

This is an everyday feature, I use that as often as get pods .. any hack would be appreciated !

[Julian V. Modesto]

@deads2k what was the result of your related work in OpenShift mentioned in #36819? Is it still related and likely the best solution for this use case?

[Maksim Kostromin]

yeah, it would be nice to have service port forwarding.. these days with minikube when minikube service web-app or minikube service web-app --url doesn't work, I'm using pods port forwarding:

terminal:

cat web-app-pod.yml
# ...
metadata:
  name: web-app
# ...

export WEBAPP_POD=$(kubectl get pods -n $NAMESPACE | grep web-app | awk '{print $1;}')
kubectl port-forward -n $NAMESPACE $WEBAPP_POD 8080

browser / client:

open http://localhost:8080

[Edgar]

It was never mentioned here but was exactly what I was looking for: https://docs.giantswarm.io/guides/accessing-services-from-the-outside/#api-access

Accessing a random pod through the service via URL.

First, simply use kubectl proxy --port 8002.
Then you can access your apps like this: http://localhost:8002/api/v1/proxy/namespaces/NAMESPACE/services/SERVICE_NAME:SERVICE_PORT/

[Dmitry Shaposhnik]

@mediafreakch Proxy can do for HTTP services, but it won't work for raw TCP/UDP (postgres, powerdns, etc).

[Shiyang Wang]

cc @vfreex

[Joe Beda]

Proxy also does deep content inspection to rewrite URLs. This is appropriate for thinks like the UI but will lead to surprises for others use cases: https://github.com/kubernetes/apimachinery/blob/18a564baac720819100827c16fdebcadb05b2d0d/pkg/util/proxy/transport.go#L71

FWIW, I have a side project that is super early to implement this client side. If it proves out I'll release it and we can look to perhaps move it into kubectl.

[Yuxiang Zhu]

IMO it is more reasonable to implement this feature in port-forward not proxy.
The syntax would be like kubectl -s SERVICE [LOCAL_PORT]:[SERVICE_PORT] [...]
-p option is deprecated for specifying the pod name, probably it should be brought back.

[Shawn Hsiao]

My attempt to support this is in PR #59705, would appreciate comments and reviews.

[Shawn Hsiao]

Want to bring #59733 to the attention of people who are interested in this issue.

[Kubernetes Submit Queue]

Closed #15180 via #59809.

[Joshua Dotson]

@phsiao. Thanks! Will your solution allow cluster admins to scope port forwarding abilities using RBAC?

[Joshua Dotson]

Just a note: There is a lot of synergy between this issue and #43962

[Shawn Hsiao]

@josdotso the implementation did not change how it connects to the api, so the same RBAC rule should continue to work. The change would require permission to look up pods given a resource name, so that needs to be adjusted as needed.

[William Yang]

would love to have this feature 👍

[Emad Mohamadi]

I assume a very nice idea in this context (while we are port-forwarding to a service) is to watch on the pod we have chosen to forward to and when it goes down, try to connect to another ready replica.
Now we will choose a pod without checking its readiness and after connecting if it gets terminated, there is no failover.

[mxxk]

@phsiao thanks for enabling service-level port forwarding in your PR #59809. However, it seems that kubectl port-forward svc/my-service only forwards traffic to one pod, even when the service has multiple endpoints. Is this the intended behavior? It seems a bit confusing, as it is different from how services load-balance between multiple pods.

By contrast, using kubectl proxy as described by @mediafreakch it is possible to reach different pods, as expected.

All of this is in Kubernetes v1.9:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.5", GitCommit:"32ac1c9073b132b8ba18aa830f46b77dcceb0723", GitTreeState:"clean", BuildDate:"2018-06-21T11:46:00Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.7-gke.3", GitCommit:"9b5b719c5f295c99de68ffb5b63101b0e0175376", GitTreeState:"clean", BuildDate:"2018-05-31T18:32:23Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

[Shawn Hsiao]

@mxxk port-forward was originally designed to attach to one pod for debugging and other purpose, the improvement in #59809 was really to help with service discovery so you don't need to lookup the pod name first if you don't care which one you attach to if you have multiple qualifying ones. port-forward does more than just http connections, so the idea of load balancing does not apply and multiple active endpoints are not supported. There is a separate feature request for re-attach if the active pod terminates, but there has not been work on it.

[mxxk]

Thanks for clarifying @phsiao. Perhaps this point can be further clarified in the help page for the port forward command? The current wording

https://github.com/kubernetes/kubernetes/blob/8a5e9ecf8febdfe5ca72d4d99340ce22cbf55cbb/pkg/kubectl/cmd/portforward.go#L76-L77)

simply says "a pod selected by the service", but it would be great if it explicitly states that one pod selected by the service is chosen and all traffic is forwarded there for the entire lifetime of the port forward command 🙂

[Paul Myburgh]

Oh my... I have been smacking my head against the wall for the past few hours wondering why on earth, when I am port-forwarding to a service, I am only sending traffic to a single pod.

Dug through all service documentation to see if I had suddenly gone crazy and didn't know how to configure a simple service correctly.

Please, for the sake of all future head smackers, make it way more clear that port-forwarding to a service still actually is using a single pod - and does not function as one might expect.

[kfox1111]

Yeah. and worse, the port forward is not re-evaluated when the pod dies. so, do a rolling upgrade and the port forward breaks. FYI.

Don't get me wrong, I'm very happy that you no longer have to go hunt for a pod name to do the port forward. :) but there are some remaining issues with the current implementation.

[Paul Myburgh]

Yea, I think the easiest way to prevent the confusion is to actually just output a simple message to the console when starting the port-forward command when it includes a service. Something like:

port-forward allows the use of services for convenience purposes only. Behind the scenes connects to a single pod directly. Connection will be dropped should this pod die.

Would have saved a lot of time 😅

[Daren Jacobs]

So, having read this, is there any way to get persistence of port-forward through the connected pod dropping?

[Oleg Aldekein]

There is also a solution to set up port forwarding to a service without selector/pod (e.g. ExternalName service) by deploying a proxy pod inside K8s:

    kubectl -n production run mysql-tunnel-$USER -it --image=alpine/socat --tty --rm --expose=true --port=3306 tcp-listen:3306,fork,reuseaddr tcp-connect:your-internal-mysql-server:3306
    kubectl -n production port-forward svc/mysql-tunnel-$USER 3310:3306

In the example above the MySQL server at your-internal-mysql-server:3306 will be available on localhost:3310 on your machine.

[Phillip Wittrock]

That is clever

[Tim Hockin]

https://github.com/kubernetes-retired/contrib/tree/master/for-demos/proxy-to-service

:)

On Fri, Sep 20, 2019 at 8:41 AM Phillip Wittrock <notifi...@github.com>
wrote:

> That is clever
>
> —
> You are receiving this because you authored the thread.

> Reply to this email directly, view it on GitHub

> <https://github.com/kubernetes/kubernetes/issues/15180?email_source=notifications&email_token=ABKWAVEXH4LLARO334HK5PDQKTVMLA5CNFSM4BRIKA62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7HDAXI#issuecomment-533606493>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABKWAVBCC4PS7B7UPEGJNV3QKTVMLANCNFSM4BRIKA6Q>
> .

[Victor Trac]

I'll throw in there one more solution: we sometimes set up pritunl as a pod inside of k8s, configured to set the client's DNS server to the kube dns cluster IP, with a default route over the VPN. When a developer connects to the pritunl VPN, their machine is able to access any service by k8s service name & cluster IP/port, just like any other pod in the cluster.

[Lukasz]

I use

kubectl port-forward svc/foobar 1234:http
curl http://localhost:1234/api/v1/

—
You are receiving this because you are on a team that was mentioned.

Reply to this email directly, view it on GitHub, or unsubscribe.

[Marcos Corona]

I use for HTTP

kubectl port-forward svc/foobar 1234:http
curl http://localhost:1234/api/v1/

I use this with TCP! <3 thx

[Andrei Kvapil]

There is also a solution to set up port forwarding to a service without selector/pod (e.g. ExternalName service) by deploying a proxy pod inside K8s:

    kubectl -n production run mysql-tunnel-$USER -it --image=alpine/socat --tty --rm --expose=true --port=3306 tcp-listen:3306,fork,reuseaddr tcp-connect:your-internal-mysql-server:3306
    kubectl -n production port-forward svc/mysql-tunnel-$USER 3310:3306

In the example above the MySQL server at your-internal-mysql-server:3306 will be available on localhost:3310 on your machine.

Hi thanks for sharing, I made a kubectl plugin for that:

kubectl proxy-port-forward 3306:your-internal-mysql-server:3306

does exactly same

https://github.com/kvaps/kubectl-proxy-port-forward

—

Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are on a team that was mentioned.Message ID: <kubernetes/kubernetes/issues/15180/1739711642@github.com>