confusion on how app in a warden container connects service intances with a port

[Hongliang Sun]

Here is the background knowledge I have learned:
DEA starts a warden container with two ports mapped in the host, one is container_port, the other is console_container_port. Howerver, when DEA starts an app, only container_port is used as an environment variable to be transported to the app, no other ports are transported.

When an apps is started by a start_script with start_command, the app will start to listen on the container_port to serve requests.

Then my confusion comes.

If the app running in the container needs to connect a service instance via JDBC, there must be a TCP connection from warden container to service instance.
The connection over TCP will hold a source IP(container_ip) and a port(maybe by random ). This connection request can reach the host, then be forwarded to the service instance after being dealed by SNAT. Source IP will be changed into host_ip of DEA.

When service instance returns the response, the destination IP in the response is host_ip of DEA and destination PORT is the port generated randomly above. But in the background knowledge of warden container I have learned, there is only one port(container_port) mapped between container and host, actually realized by a DNAT rule in iptables. Therefore, in theory, the response from service instance with the randomly generated port will not be sent to the container.

Then problems comes up that no services can work in Cloud Foundry.

And I am definitely sure Cloud Foundry works perfectly with their services. There must be some mistakes in my understandings.

Can someone tell me what I missed or what I have mistaken for.

Thanks in advance.


Hongliang Sun
Zhejiang University, China

[Wayne E. Seguin]

Essentially warden container routes only restrict *incomming* new connections.

The container can reach out to any IP and establish a TCP connection at will and the reverse traffic will be allowed because it initiated from within.

Does that help with your understanding?

--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/65dbdaab-77bf-4de2-9703-d4169cee530a%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[James Bayer]

hongliang,

is this a theoretical question or has cloud foundry apps stopped working for you recently, possibly due to the new application security groups [1] [2] introduced in v175?

[1] https://docs.google.com/a/gopivotal.com/document/d/1519mrEKPuJmDKR3G-adV6eQFXkxY4cTObtzQrCCE2vM/edit

[2] https://github.com/cloudfoundry-community/cf-docs-contrib/wiki/v175#upgrade-notes

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CABz%3D7E9mEV252DHTYYJrP9NZ3bkVkL%2B5jiDYH5kdk81a8K9ziA%40mail.gmail.com.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Thank you,

James Bayer

[Hongliang Sun]

James, thanks for your reply.

Actually it is a theoretical question. I have read source code implementation of dea_ng and warden, and just cannot explain the above case using the knowldge I have concluded.

Security groups is a great policy to isolate network access. And I do not think it has something to do with the question I put forward.

[Hongliang Sun]

Thanks for your reply, Wayne,

I think we have made an agreement on "The container can reach out to any IP and establish a TCP connection at will ",

However I am still confused with "the reverse traffic will be allowed because it initiated from within", can you explain more on this, or tell me where I am wrong below?

My point is that no matter what the reverse traffic is,  it is a TCP packet with a destination port. When this packet reached DEA_host, the network kernel rule will handle this. And only TCP packets with specific port in DEA_host will be handled by DNAT, then the TCP packet will be forwarded to the container. 

However an app starts an JDBC connection, it maybe fetch a random port, and this port has not been mapped in DEA_host. 

[Matthew Sykes]

Warden configures firewall rules to allow data to flow through previously established connections and to receive related communications.  When containers establish outbound connections, responses related to those automatically allowed.

Different protocols have different algorithms for 'related' and many require additional kernel modules to support them but basic tcp request/response is (of course) part of the core.

You can get detailed information by reading tutorials on iptables and netfilter.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/0529a192-709e-4fa8-9590-dd114ba1ca0b%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--
Matthew Sykes
matthe...@gmail.com

[Hongliang Sun]

Hi, Matthew,

      I think I have got what you said, and now I can find the network setting when creating a warden container at  https://github.com/cloudfoundry/warden/blob/master/warden/root/linux/net.sh#L100 

      I really appreciated that you gave me this helpful guide. Maybe I should spend more time analyzing warden container , iptables and netfilters. 

     

      :)

Hongliang Sun 

Zhejiang University, China