how does kubernetes work to access multiple nodes service by proxy service?
1 view
Skip to first unread message
Clare
Nov 5, 2014, 11:44:07 PM11/5/14
to
Background:
1. two server: 192.168.230.3,192.168.230.4, both run kubernetes proxy service.
2. two pods, every pod has only one container, pod1 is in 192.168.230.3 and runs web service, pod2 is in 192.168.230.4 and runs db service, pod1's service portal is 172.18.0.2:40000, pod2's service portal is 172.18.0.3:45000, pod1's endpoint is 172.17.0.2:8080, pod2's endpoint is 172.17.0.3:5432.
my question:
if pod1's container wants to access pod2's db container to do something, it must cross two hosts to achieve this. so, could please some guys to answer me how does kubernetes work to access multiple nodes service by proxy service? what's work flow of accessing service? if can give me the detail access workflow, that's better. thanks a lot in advance.
Brendan Burns
Nov 6, 2014, 12:00:37 AM11/6/14
to google-c...@googlegroups.com
Hello,
Here is how it works.
There is a service proxy on each host machine, it creates an unique for each service:
pod-service-1:12345
pod-service-2:23456
This is present on _all_ host machines.
When pod #1 sends traffic to 172.18.0.3:45000 (pod-service-2), there are IP tables rules on the host which catch this packet, and rewrite it to be localhost:23456 so that it goes to the service proxy that is running locally.
The service proxy then terminates the TCP connection, and initiates its own TCP connection to 192.168.230.4 (pod 2), the service proxy forwards traffic until the connection is terminated.
Thus, though it looks like you're bouncing between two hosts, you're actually only connecting to localhost, and then making a single outbound connection to the pod that implements the service.
I hope that helps explain things, let us know if you have further questions.
--brendan
On Wed, Nov 5, 2014 at 8:44 PM, Clare <yangzh...@gmail.com> wrote:
Background:1. two server: 192.168.230.3,192.168.230.4, both run kubernetes proxy service.
2. two pods, every pod has only on container, pod1 is in 192.168.230.3 and runs web service, pod2 is in 192.168.230.4 and runs db service, pod1's service portal is 172.18.0.2:40000, pod2's service portal is 172.18.0.3:45000, pod1's endpoint is 172.17.0.2:8080, pod2's endpoint is 172.17.0.3:5432.
my question:if pod1's container wants to access pod2's db container to do something, it must cross two hosts to achieve this. so, could please some guys to answer me how does kubernetes work to access multiple nodes service by proxy service? what's work flow of accessing service? if can give me the detail access workflow, that's better. thanks a lot in advance.
--
You received this message because you are subscribed to the Google Groups "Containers at Google" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-contain...@googlegroups.com.
To post to this group, send email to google-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-containers.
For more options, visit https://groups.google.com/d/optout.
Clare
Nov 7, 2014, 1:52:20 AM11/7/14
to google-c...@googlegroups.com
hi brendan,
thanks your reply.
As you said "The service proxy then terminates the TCP connection, and initiates its own TCP connection to 192.168.230.4 (pod 2), the service proxy forwards traffic until the connection is terminated." I want to know:
1) How the service proxy initiates its own TCP connection to 192.168.230.4(pod 2), if there are more minions which also run the same service with pod 2, how to chose one to connect.
2) Whether service proxy knows how to chose one minion to connect, if know, how does it do?
thanks,
clare
在 2014年11月6日星期四UTC+8下午1时00分37秒，Brendan Burns写道：
Daniel Smith
Nov 7, 2014, 12:45:12 PM11/7/14
to google-c...@googlegroups.com
On Thu, Nov 6, 2014 at 10:52 PM, Clare <yangzh...@gmail.com> wrote:
hi brendan,thanks your reply.As you said "The service proxy then terminates the TCP connection, and initiates its own TCP connection to 192.168.230.4 (pod 2), the service proxy forwards traffic until the connection is terminated." I want to know:1) How the service proxy initiates its own TCP connection to 192.168.230.4(pod 2), if there are more minions which also run the same service with pod 2, how to chose one to connect.
The proxy has a list of "endpoints" for the service (which it gets from apiserver, which populates them by running the service's selector on pods). The proxy chooses randomly from this list. (or is it round-robin? I forget.)
Hopefully that answers your question.
Tim Hockin
Nov 7, 2014, 1:05:50 PM11/7/14
to google-c...@googlegroups.com
round-robin
Clare
Nov 8, 2014, 9:33:51 AM11/8/14
to
hi Daniel
在 2014年11月8日星期六UTC+8上午1时45分12秒，Daniel Smith写道：
thank you very much.
even though proxy maintains all service endpoint, if one pod needs to access other pods which place in different hosts, because these endpoints are private, that pod can't access directly them. i set up two experiments via https://github.com/GoogleCloudPlatform/kubernetes/tree/master/examples/guestbook, overview as:
Experiment No 1:
service name service portal
service name service listen port(generated randomly)
frontend 192.168.230.3:40261
redismaster 192.168.230.3:48345
redisslave 192.168.230.3:36076
Experiment No 2:
service name service portal
service name service listen port(generated randomly)
frontend 192.168.230.3:40261
redismaster 192.168.230.3:48345
redisslave 192.168.230.3:36076
#################################
service name service portal
service name service listen port(generated randomly)
frontend 192.168.230.4:32932
redismaster 192.168.230.4:37737
redisslave 192.168.230.4:35083
1) if php frontend and redismaster are in the same host:192.168.230.3, i can open http://192.168.230.3:8000 to submit messages.
2) if php frontend and redismaster are not in the same host, php in 192.168.230.3, redis master in 192.168.230.4, i still can open open http://192.168.230.3:8000, but can't submit messages, and error log from 192.168.230.3 as follow:
[Sat Nov 08 21:44:11 2014] [error] [client 192.168.230.1] PHP Fatal error: Uncaught exception 'Predis\\Connection\\ConnectionException' with message 'Error while reading line from the server [tcp://172.18.0.17:10000]' in /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php:141\nStack trace:\n#0 /vendor/predis/predis/lib/Predis/Connection/StreamConnection.php(208): Predis\\Connection\\AbstractConnection->onConnectionError('Error while rea...')\n#1 /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php(130): Predis\\Connection\\StreamConnection->read()\n#2 /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php(122): Predis\\Connection\\AbstractConnection->readResponse(Object(Predis\\Command\\StringGet))\n#3 /vendor/predis/predis/lib/Predis/Client.php(246): Predis\\Connection\\AbstractConnection->executeCommand(Object(Predis\\Command\\StringGet))\n#4 /vendor/predis/predis/lib/Predis/Client.php(228): Predis\\Client->executeCommand(Object(Predis\\Command\\StringGet))\n#5 /app/index.php(32): Predis\\Client->__call('get', Array)\n#6 /app/index.php(32): Predis\\Client->get('messa in /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php on line 141, referer: http://192.168.230.3:8000/
from error log, we can find php fronted service to access redis master service via tcp://172.18.0.17:10000, by Brendan's answer, when sends traffic to 172.18.0.17:10000, it will rewrite it to 192.168.230.3:48345, and initiates a TCP connection to access redis master service which is in 192.168.230.4, but it seems it doesn't work. cloud please some guys help explain this?
在 2014年11月8日星期六UTC+8上午1时45分12秒，Daniel Smith写道：
Tim Hockin
Nov 8, 2014, 10:34:56 AM11/8/14
to google-c...@googlegroups.com
Step back from the example and simplify. If you log in to two machines and manually docker run -ti ubuntu, can those two containers ping each other?
If not, your environment has not satisfied the basic networking requirement.
1) if php frontend and redismaster are in the same host:192.168.230.3, i can open http://192.168.230.3:8000 to submit messages.2) if php frontend and redismaster are not in the same host, php in 192.168.230.3, redis master in 192.168.230.4, i still can open open http://192.168.230.3:8000, but can't submit messages, and error log from 192.168.230.3 as follow:[Sat Nov 08 21:44:11 2014] [error] [client 192.168.230.1] PHP Fatal error: Uncaught exception 'Predis\\Connection\\ConnectionException' with message 'Error while reading line from the server [tcp://172.18.0.17:10000]' in /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php:141\nStack trace:\n#0 /vendor/predis/predis/lib/Predis/Connection/StreamConnection.php(208): Predis\\Connection\\AbstractConnection->onConnectionError('Error while rea...')\n#1 /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php(130): Predis\\Connection\\StreamConnection->read()\n#2 /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php(122): Predis\\Connection\\AbstractConnection->readResponse(Object(Predis\\Command\\StringGet))\n#3 /vendor/predis/predis/lib/Predis/Client.php(246): Predis\\Connection\\AbstractConnection->executeCommand(Object(Predis\\Command\\StringGet))\n#4 /vendor/predis/predis/lib/Predis/Client.php(228): Predis\\Client->executeCommand(Object(Predis\\Command\\StringGet))\n#5 /app/index.php(32): Predis\\Client->__call('get', Array)\n#6 /app/index.php(32): Predis\\Client->get('messa in /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php on line 141, referer: http://192.168.230.3:8000/from error log, we can find php fronted service to access redis master service via tcp://172.18.0.17:10000, by Brendan's answer, when sends traffic to 172.18.0.17:10000, it will rewrite it to 192.168.230.3:48345, and initiates a TCP connection to access redis master service which is in 192.168.230.4, but it seems it doesn't work. cloud please some guys help explain this?
在 2014年11月8日星期六UTC+8上午1时45分12秒，Daniel Smith写道：
Reply all
Reply to author
Forward
0 new messages