Proposal for a new project: OCI Image Format Spec (v2)

[Brandon Philips]

TOB Members-

This is an updated proposal that folds changes in from Diogo's proposal and the original. The primary change is the removal of distribution via http or any other protocol as part of the project. See the FAQ item for more information on that if you haven't been following along.

Diogo's: https://groups.google.com/a/opencontainers.org/forum/#!topic/tob/A3JnmI-D-6Y
Original: https://groups.google.com/a/opencontainers.org/forum/#!topic/tob/WXk1uTgfXrs

We will have a call tomorrow morning where we can discuss. But, my hope is that during the call we can all come to agree on this proposal so we can begin forming the project and maintainers soon. Please take a look and let us know how you feel:

- Agree with this proposal
- Disagree with this proposal
- Lets discuss, I need more information


Thank You,

Brandon

OCI Image Format Spec

New project proposal: The TOB will create a new OCI project tasked with creating a software shipping container image format spec (OCI Image Format) with security and naming as components. In addition the OCI TOB will establish a new set of maintainers for this new project with people who have expertise in image formats and package management.

Initial Recommendation: Over the last 16 months we have seen evolution of container formats towards solid technical underpinnings. When the AppC project was introduced in December 2014 its goals were to create an open specification shared between implementations while addressing concerns in the Docker v1 image format around content addressable images, signing, and a federated/delegatable namespace. Today, the Docker v2.2 image format is close to having all of the desirable traits of AppC while having widespread registry implementations in projects/products from Amazon, Google, CoreOS, Docker, Huawei, and JFrog.

This new OCI project would be recommended to start with the Docker v2.2 specification, improve any remaining technical concerns, and create an OCI project and maintainers to develop and shepard an OCI Image Fromat Spec. By starting from this project we intend to standardize and improve the understood properties of a container image format. This new project will have the objectives of:

• A serialized image format (base layer)

• A process of hashing the image format for integrity and content-addressing (base layer)

• Signatures that are based on signing image content address (optional layer)

• Naming that is federated based on DNS and can be delegated (optional layer)

Initial Maintainers: to be brainstormed on a separate thread.

Cooperation with OCI Runtime Project: The OCI Runtime Specs project is working diligently to create a specification for the lifecycle of a running container. The OCI Image Format Spec project should work with the OCI Runtime Spec project so that the image can support the UX that users have come to expect from container engines like Docker and rkt. Primarliy the ability to run an image with no additional arguments:

  docker run example.com/org/app:v1.0.0

  rkt run example.com/org/app,version=v1.0.0

This implies that the OCI Image Format contain sufficient information to launch the application on the target platform e.g. command, arguments, environment variables, etc.

FAQ

Q: Why doesn't this project mention distribution?

A: Distribution, for example using HTTP as both Docker v2.2 and AppC do today, is not part of this project initially. This is because the OCI scope table, as it reads today, says this is out of scope. But, the TOB discussion has deemed it is necessary to add this as an optional layer in the scope table in the near future. The TOB will need to work to make this addition to the scope table. See this thread https://groups.google.com/a/opencontainers.org/d/msg/tob/A3JnmI-D-6Y/tLuptPDHAgAJ

Q: Why a new project?

A: The first OCI spec centered around defining the run side of a container. This is generally seen to be an orthogonal concern to the shipping container component. As practical examples of this separation you see many organizations separating these concerns into different teams and organizations: the Docker Distribution project and the Docker containerd project; Amazon ECS and Amazon EC2 Container Registry, etc.

Q: Why start this work now?

A: We are seeing many independent implementations of container image handling including build systems, registries, and image analysis tools. As an organization we would like to encourage this growth and bring people together to ensure a technically correct and open specification continues to evolve reflecting the OCI values.

Q: What happens to AppC or Docker Image Formats?

A: Existing formats can continue to be a proving ground for technologies, as needed. For example, in line with the OCI values, we expect mechanisms like the AppC name delegation will help inspire portions of the OCI Software Shipping Container project. The OCI Image Format project should strive to provide a dependable open specification that can be shared between different tools and be evolved for years or decades of compatibility; as the deb and rpm format have.

Proposed Roadmap

• March ?? v0.0.0

• Import Docker v2.2 format

• April 18th v0.1.0

• Spec factored for top to bottom reading with three audiences in-mind:

• Build system creators

• Image registry creators

• Container engine creators

• Test cases for each of the potential users exercising build, push, and pull verbs

• May 16th v0.2.0

• Release version of spec with improvements from two independent experimental implementations from OCI members e.g. Amazon Container Registry and rkt

• June 13th v1.0.0

• Release initial version of spec with two independent non-experimental implementations from OCI members

[Brandon Philips]

Hello TOB-

As discussed on the call. If you are a TOB member please reply with your official vote so we can proceed with the creation of this project: 

- Agree with this proposal
- Disagree with this proposal

Thank You,

Brandon

[Jason Bouzane]

Agree with proposal

--
You received this message because you are subscribed to the Google Groups "Technical Oversight Board" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tob+uns...@opencontainers.org.

[Brandon Philips]

Also, a slight adjustment, the proposed timeline has a leftover part from when distribution was part of this:

"Test cases for each of the potential users exercising build, push, and pull verbs"

Please disregard this bit; it is left over from the first proposal.

Brandon

[Vincent Batts]

On Fri, Mar 18, 2016 at 12:53 PM, Brandon Philips
<brandon...@coreos.com> wrote:
> Also, a slight adjustment, the proposed timeline has a leftover part from
> when distribution was part of this:
>
> "Test cases for each of the potential users exercising build, push, and pull
> verbs"
>
> Please disregard this bit; it is left over from the first proposal.

ah. yeah. perhaps "build" still could apply. Though, these test
use-cases can be handled in the project's own ROADMAP

[Greg KH]

On Fri, Mar 18, 2016 at 04:53:28PM +0000, Brandon Philips wrote:
> Also, a slight adjustment, the proposed timeline has a leftover part from when
> distribution was part of this:
>
> "Test cases for each of the potential users exercising build, push, and pull
> verbs"
>
> Please disregard this bit; it is left over from the first proposal.

So should there be a "corrected" proposal that we can vote on just so
that we all know the correct edits that are made to it? :)

thanks,

greg k-h

[caniszczyk]

I converted what Brandon suggested to GitHub for record keeping purposes (instead of being embedded in email):

https://github.com/opencontainers/tob/pull/4

For voting, I think finishing the vote over the mailing list is fine, you're not really committed to the proposed roadmap, it's just a proposal.

[Brandon Philips]

Agree with this proposal.

[Greg KH]

On Fri, Mar 18, 2016 at 03:00:29PM -0700, caniszczyk wrote:
> On Friday, March 18, 2016 at 4:15:27 PM UTC-5, gregkh wrote:
>
> On Fri, Mar 18, 2016 at 04:53:28PM +0000, Brandon Philips wrote:
> > Also, a slight adjustment, the proposed timeline has a leftover part from
> when
> > distribution was part of this:
> >
> > "Test cases for each of the potential users exercising build, push, and
> pull
> > verbs"
> >
> > Please disregard this bit; it is left over from the first proposal.
>
> So should there be a "corrected" proposal that we can vote on just so
> that we all know the correct edits that are made to it?  :)
>
>
> I converted what Brandon suggested to GitHub for record keeping purposes
> (instead of being embedded in email):
> https://github.com/opencontainers/tob/pull/4

As Brandon points out in the pull request, there are some things that
need to be fixed up in it.

So, I ask again, can someone provide a "this is the real proposal"
somewhere so that we really can vote on it?

thanks,

greg k-h

[Diogo Monica]

Agreed with Greg.

As Brandon points out in the pull request, there are some things that
need to be fixed up in it.

So, I ask again, can someone provide a "this is the real proposal"
somewhere so that we really can vote on it?

thanks,

greg k-h

-- 
You received this message because you are subscribed to the Google Groups "Technical Oversight Board" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tob+uns...@opencontainers.org.

[Chris Aniszczyk]

I advise we use what's on GitHub now:

https://github.com/opencontainers/tob/blob/master/proposals/image-format/README.md

Does anyone have any changes to this to make? If not, lets call it final and vote off that one.

--

Chris Aniszczyk (@cra) | +1-512-961-6719

[Brandon Philips]

Alright, I am closing voting on this v2 proposal thread and opened up v3. Hope this is the final one. https://groups.google.com/a/opencontainers.org/forum/#!topic/tob/KGyPpu8YfNk