Connecting containers, am I missing something really obvious?

[Ian Flynn]

Previously I was running a couple Rails projects and static websites on a few different vms. Over the past day or two, I've been playing with moving them to rkt containers running on a CoreOS vm. No Kubernetes, no etcd, just one CoreOS vm running the rkt containers from systemd. When it comes to linking containers together, I know the accepted way is to either run related apps in the same pod, or else use Kubernetes, flannel + consul, something like that, but I wanted to try using pure rkt just to have a better understanding of how everything works before diving into frameworks that take care of everything for me.

I have a Postgresql container starting first in it's own pod, with my Rails app (in a separate pod/container) defined in systemd to start only after Postgres is up and running. Easy enough, but how to pass the ip and port of the Postgres container to the Rails app? Sure I could use --port to expose the postgres on the host, then tell the rails app to just connect to localhost, but that doesn't feel very containerized... what if I want to run more than one postgres instance one day? What if exposing the port causes a security concern? I want to run one postgresql instance and be able to add / remove apps that talk to it without having to teardown the postgresql pod. So here's what I did, just as an experiment you understand. I created my own Postgres ACI derived from the official one like so:

 acibuild begin docker://postgres:10-alpine 

 acbuild set-name custom-postgres 

 acbuild mount add sock /var/run/postgresql

 acbuild write custom-postgresql.aci

So now I have a postgresql aci that exposes the /var/run/postgresql socket in a volume. I run it like

 

rkt --insecure-options=image run --volume sock,kind=host,source=/var/rkt_volumes/postgresql/sock custom-postgresql.aci

I can now mount that same volume in my Rails app, and have it connect by socket. Seems to work well. For this one app, everything is working great. When I want to add an app that requires a database I look up the ip of the pod running postgres, psql into into it, create a new user and password for the app, then create the new pod passing in those credentials on the command line.

Still, it feels a little hacky. Without going to some kind of orchestration system (Kubernetes or what-have-you) is there a better way to do this? My understanding is that with the default networking given to pods, there is no way for apps running in two separate pods to connect to one-another without sharing a volume with a socket in it, or one of them exposing a host port. Is this accurate? Am I missing something obvious? What's the next logical step, I think flannel is capable of doing the kind of more advanced networking I'm looking for, should I learn that?

Thanks so much for any advice or guidance you might have.

[Alban Crequy]

Hi,

Using unix sockets in a shared volume is fine, but you can also allow
IP connectivity using a bridge instead of the default network
configuration:
https://github.com/rkt/rkt/blob/master/Documentation/networking/examples-bridge.md

Then, instead of letting rkt assign IPs to the pods automatically, you
can choose the IPs yourself:
https://github.com/rkt/rkt/blob/master/Documentation/networking/overriding-defaults.md#example-passing-arguments-to-two-different-networks

Then, you can tell the rail app pod to connect to the IP you have
decided for the postgres pod.

You could define a bridge called pgsql-bridge in /etc/rkt/net.d/ and
only use it in pods that are allowed to talk to the posgres pod.

Of course, that's not ideal to have to write IPs yourself. As you
pointed out, that's something where Kubernetes can help with
Kubernetes Services :)

Alban

> --
> You received this message because you are subscribed to the Google Groups
> "rkt-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to rkt-dev+u...@googlegroups.com.
> To post to this group, send email to rkt...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/rkt-dev/302b3e97-a7cf-4ab7-87e9-3ede1edd833e%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.