appc + oci harmonization progress

[Brandon Philips]

Hello All-

As we look to get an initial draft of the OCI spec released in the coming weeks I want to outline how the harmonization with the Application Container Spec (appc) is proceeding. Harmonizing OCI with the appc spec is part of our draft charter. 

Until now the OCI spec has been focused on the runtime spec and I think we need to figure out what the next priority is for us after that. Hopefully looking at the harmonization work left to do with appc will help inspire those priorities.

# Intro

The standard shipping container defines a specification for the size, shape and identity of the object that can be moved around with various transport methods whether it be ship, truck or train. By analogy the specification for a standard software container needs to define how a container is packaged into a single object, identified uniquely, distributed over various transport methods and then finally ran on a computer system.

The Application Container Spec ([appc][1]) defines a specification for all of these software container concerns: packaging, identity, distribution, and runtime. And over time, our goal, is to ensure the OCI spec has these same properties.

[1]: https://github.com/appc/spec

## Packaging Progress

appc defines the concept of an [Application Container Image (ACI)][1] which is a single object (compressed tar file) that contains a root filesystem and host-independent runtime configuration (args, env variables, mount points, etc).

In the OCI spec there has been no discussion on packaging containers into images. Also, no discussion has started about the concept of layering where one packaged image can rely on another packaged image; OCI will need to tackle identity and distribution to make image layers happen.

[1]: https://github.com/appc/spec/blob/master/spec/aci.md

## Identity Progress

appc defines a cryptographic identity of an ACI as the sha512 of the uncompressed tar file in the [Application Container Image spec][1]. With mechanisms such as tar-split this gives a consistent way to transition from on-disk unpacked copies of an image back to an image and reproduce the same identity.

In the OCI spec there has been some early discussion on giving unpacked on-disk images a [cryptographic identity][2] but nothing has been put into the spec and this work will be insufficient for a trusted distribution identity.

[1] https://github.com/appc/spec/blob/master/spec/aci.md

[2] https://github.com/opencontainers/specs/issues/11

## Distribution Progress

appc defines a DNS federated namespace for distributing images over various transports such as https, bittorrent, or other protocols in the [Application Container Discovery spec][1]. This namespace also allows for delegation of mirrors that were outside of the namespace so example.com could delegate fetching of images from https://objectstore.example.

There have been no discussions on how to mirror on a private or public file store an OCI container image for distribution between machines.

[1] https://github.com/appc/spec/blob/master/spec/discovery.md

## Runtime Progress

appc defines how resource limits, host isolation, environment variables and arguments should be setup for an application in the [Application Container Executor][1]. Also this spec defines in both OS independent and OS dependent terms how a process should be configured along with its lifecycle from start to stop.

Today, the OCI spec has only started to tackle spec'ing the runtime piece. The runtime requirements for Linux containers are well understood and straightforward with many concrete Linux container runtimes that exist today, such as lxc, systemd-nspawn, libvirt-lxc, libcontainer, etc. The OCI runtime spec has a prescriptive configuration file called config.json that specifies things such as the initial state of cgroups, namespaces, application arguments, etc to be consumed by a container runtime implementation. This config.json along with a root filesystem layout for the container is called an "OCI bundle".

The appc spec doesn't define something as prescriptive as the OCI config.json as this is primarily a low-level detail of the container configuration passed from a user or orchestration system to a runtime like lxc/runC/systemd-nspawn. It is useful for being concrete about the runtime in the spec, but isn't critical to the goals of a standard software container; by analogy it is a bit like defining the color of paint inside of the standard shipping container. In appc there is the concept of a "pod manifest" which is similar to the OCI config.json but is designed with a user wanting to launch a complex set of containers in-mind instead of serializing state to internal tools.

Overall, the OCI runtime spec is progressing but is not far enough along for a person to pick it up and independently implement a runtime without looking at the behavior of runC. This is improving and the goal is to have it in a better state before the draft release. The addition of a tool to test a runtime, as discussed during todays meeting, should make that easier[2].

[1] https://github.com/appc/spec/blob/master/spec/ace.md

[2] https://github.com/opencontainers/specs/issues/109

## Conclusion

The OCI is going to succeed once it delivers on the promise of a specification for packaging, identifying, distributing and running a container. We have made some early progress but the hardest work is yet to come to enable the use case of "package and sign once, run under any application container system" that the industry is striving for.

## Next Steps

We discussed on the call, but the goal of the initial draft OCI is to only cover runtime concerns and there are still lose ends to finish before we release this draft including: a test bundle for runtimes, a validation tool, and sorting out all of the remaining open issues around the runtime specification. Also, we are several weeks behind our initial goal of having a draft released in mid-July so I think we need to just focus on this runtime aspect for the initial release and start planning the next things to tackle.

Based on the topics outlined above from the appc + OCI harmonization I think the two places to concentrate after this draft are:

- Distribution: how are OCI images named and how do I find and download an image given a name

- Identity: how to generate a cryptographic identity for a container image

Brandon

[W. Trevor King]

On Thu, Aug 13, 2015 at 03:55:58AM +0000, Brandon Philips wrote:
> Until now the OCI spec has been focused on the runtime spec and I
> think we need to figure out what the next priority is for us after
> that. Hopefully looking at the harmonization work left to do with
> appc will help inspire those priorities.

I think we should *keep* opencontainers/specs focused on the runtime,
independent of all other container tooling. There can be separate
specs and tools for those separate issues [1,2,3]. There's a
lot going on in a container-based ecosystem, and trying to handle all
of that in one place that makes everybody happy is going to be
impossible. We should focus on writing composable tools based on
interoperable specs, so users can swap in the tools that match their
workflow without getting bogged down by complications or design
decisions that they don't need. This composability is §6.a in the
current draft charter [4], and keeping the individual tools minimal is
§6.f.

Of course, we need forums for discussing how these tools will fit
together, and the interface between appc and opencontainers/specs,
Nulecule and opencontainers/specs [5], and other interested
communities will probably be where this happens. Hopefully, the CNCF
will help here as well [5].

> The Application Container Spec ([appc][1]) defines a specification
> for all of these software container concerns: packaging, identity,
> distribution, and runtime. And over time, our goal, is to ensure the
> OCI spec has these same properties.

I'd rather have separate specs for identity and distribution. For
example, IPFS can almost handle this right now, and that's great for
me distributing my bundles. However, I imagine others will have
different tooling that fits into their system for distributing
bundles. We don't want to get into the business of making all the
decisions needed to build a whole container infrastructure, we just
want to make sure our runtime spec is as compatible with the rest of
the ecosystem as possible, and then provide good docs explaining
useful stack combinations (e.g. IPFS + runC, or appc + runC, or …).

> Based on the topics outlined above from the appc + OCI harmonization
> I think the two places to concentrate after this draft are:
>
> - Distribution: how are OCI images named and how do I find and
> download an image given a name
> - Identity: how to generate a cryptographic identity for a container
> image

I'm happy to have new OCI Projects (see my “many projects with
per-project TDCs” proposal [7]) that focus on these issues. But for
me, IPFS already handles distribution and identity well enough
(e.g. see [8]). I think pushing that sort of thing into
opencontainers/specs is just going to be distracting for folks who
only need the runtime container/application launch tooling.

Cheers,
Trevor

[1]: https://en.wikipedia.org/wiki/Unix_philosophy
[2]: https://github.com/opencontainers/specs/pull/76#issuecomment-124913227
[3]: https://groups.google.com/a/opencontainers.org/d/msg/dev/_qzxN228Suw/-GLDjizZDgAJ
Message-ID: <20150808040...@odin.tremily.us>
[4]: https://github.com/opencontainers/web/blob/33bc0b2597cbfd37e4728660d4b74272603b6f97/content/charter.md
[5]: https://github.com/opencontainers/specs/issues/73#issuecomment-126425653
and later comments in that issue
[6]: https://github.com/opencontainers/specs/issues/5#issuecomment-115423504
[7]: https://github.com/opencontainers/web/pull/4
[8]: https://github.com/wking/oci-gentoo-minimal/tree/ipfs-gateway

--
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[W. Trevor King]

On Thu, Aug 13, 2015 at 11:29:08AM -0700, W. Trevor King wrote:
> On Thu, Aug 13, 2015 at 03:55:58AM +0000, Brandon Philips wrote:
> > Until now the OCI spec has been focused on the runtime spec and I
> > think we need to figure out what the next priority is for us after
> > that. Hopefully looking at the harmonization work left to do with
> > appc will help inspire those priorities.
>
> I think we should *keep* opencontainers/specs focused on the
> runtime, independent of all other container tooling. There can be
> separate specs and tools for those separate issues [1,2,3]. There's
> a lot going on in a container-based ecosystem, and trying to handle
> all of that in one place that makes everybody happy is going to be
> impossible. We should focus on writing composable tools based on
> interoperable specs, so users can swap in the tools that match their
> workflow without getting bogged down by complications or design
> decisions that they don't need. This composability is §6.a in the
> current draft charter [4], and keeping the individual tools minimal
> is §6.f.
>
> Of course, we need forums for discussing how these tools will fit
> together, and the interface between appc and opencontainers/specs,
> Nulecule and opencontainers/specs [5], and other interested
> communities will probably be where this happens. Hopefully, the
> CNCF will help here as well [5].

To keep the “how do we organize discussions in the larger container
ecosystem” discussion from fragmenting, I thought I'd drop in a few
cross links.

* Solomon agrees that networking is out of spec for the OCF
(presumably opencontainers/specs?) [1]. But he doesn't explicitly
say whether he sees it as out of scope for the OCI as a whole. I
don't really care about scope limits for the OCI, as long as we have
clear, narrow scope limits for individual OCI projects (which I'd
like to map one-to-one to opencontainers GitHub repositories [2]).
* I've proposed starting a new repository for specifying the runtime's
command-line API, so higher-level tools don't need to look at [3].
That would give us a very narrow example of this sort of workflow if
we wanted to kick its tires before doing something like an
independent network-transport spec / implementation. Of course, we
already have (and I quite like) the independent repository approach
for opencontainers/specs, opencontainers/runc, and hyperhq/runv.
* Vincent and I have been going back and forth in [4] about where the
first config-validator implementation should live. He's currently
pushing for it to live in opencontainers/specs [5], while I've been
pushing to get it in it's own opencontainers repository [6].

Cheers,
Trevor

[1]: https://groups.google.com/a/opencontainers.org/d/msg/dev/3XqoyLqtA6I/p3cHuQiOEQAJ
From: Solomon Hykes
Subject: Re: Do we have any plan for network and storage specs?
Date: Sun, 16 Aug 2015 16:45:10 -0700
Message-ID: <CACfVidfRy1-RFZmyZ+XfDxGDbarbHfi9P6e_khuSEFxmvv=k...@mail.gmail.com>
Cc: dev <d...@opencontainers.org>, …
[2]: https://github.com/opencontainers/web/pull/4#issuecomment-129032943
[3]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/BIxya5eSNLo
From: "W. Trevor King"
Subject: Specifying the runtime's command-line interface
Date: Tue, 18 Aug 2015 10:52:55 -0700
Message-ID: <20150818175...@odin.tremily.us>
To: d...@opencontainers.org
[4]: https://github.com/opencontainers/specs/issues/109
[5]: https://github.com/opencontainers/specs/issues/109#issuecomment-131265085
[6]: https://github.com/opencontainers/specs/issues/109#issuecomment-131598332