Block/arch diagram for pod with multiple containers

[Sankaranarayanan Venkatasubramanian]

Hello,

Could you please help me understand how multiple containers within a single pod works in gVisor - with a block/arch diagram? Since sentry works (and gets mapped) in both gr0 and gr3, does multiple copies of sentry exist for each container in gr3?

Also, There are tons of discussions on gVisor KVM flow. Would you be able to share a sequence diagram for a Go noob like me?

Thanks,

Sankar

[Kevin Krakauer]

Hi Sankar,

I don't have a diagram, but I can answer your question about multiple containers and pods assuming we're talking about Kubernetes. Each pod is a single gVisor sentry. Each container is a process (or set of processes) within that sentry. Containers are separated within the pod by some namespaces implemented by the sentry. This corresponds nicely to the Kubernetes concept of a pod, where containers in the same pod share resources and are not strongly isolated. But pods are strongly isolated.

Where the sentry is mapped and what platform it's running (KVM or systrap) doesn't affect how multiple containers run.

Kevin

--
You received this message because you are subscribed to the Google Groups "gVisor Users [Public]" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gvisor-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gvisor-users/d121b2a6-97a4-44b8-8a4b-2a209fa24d47n%40googlegroups.com.

[Sankaranarayanan Venkatasubramanian]

Thanks for the response, Kevin. Got it, I was missing the below one.

     >>  Containers are separated within the pod by some namespaces implemented by the sentry.

Could you help me locate the relevant part in the code?

Sankar 

[Ayush Ranjan]

You can follow the logic in runsc/boot/loader.go:startSubcontainer(). For instance, if you inspect the OCI runtime spec passed to runsc in a multi-container scenario (like in GKE), it will have a section like this:

```

    "namespaces": [
      {
        "type": "pid"
      },
      {
        "type": "ipc",
        "path": "/proc/104921/ns/ipc"
      },
      {
        "type": "uts",
        "path": "/proc/104921/ns/uts"
      },
      {
        "type": "mount"
      },
      {
        "type": "network",
        "path": "/proc/104921/ns/net"
      },
      {
        "type": "cgroup"
      }
    ],

```

This will cause the new subcontainer to have create/join these namespaces. For example, this subcontainer would run in a new pid namespace. But like Kevin said, such namespace isolation between the containers is implemented by the sentry. The sentry process itself (which is shared among all containers in a pod) is running in the same namespace while servicing requests from multiple containers.

- Ayush

[Sankaranarayanan Venkatasubramanian]

Thanks, Ayush!

[Sankaranarayanan Venkatasubramanian]

One more question, Ayush/Kevin. Based on the above, is it right interpretation that the containers are running in the same process address space as sentry, and it is a virtual isolation between containers that is provided by sentry for all the containers running inside the pod?

~Sankar V

[Kevin Krakauer]

The specific way containers/processes get memory depends on the
platform (systrap or KVM), but separate containers/processes cannot
access each other's memory nor the sentry's memory. The sentry can
read containers/processes memory though, just as a typical kernel
would.

On Tue, Feb 20, 2024 at 6:41 AM Sankaranarayanan Venkatasubramanian

> To view this discussion on the web visit https://groups.google.com/d/msgid/gvisor-users/cc3e96d6-480a-460e-816c-60d9508af1e3n%40googlegroups.com.