Throwing the flag

[Vincent Batts]

(I collaborated with Phil Estes on this. So send him the credit, and me the blame.)

tl;dr;

- The current OCI contention is why the TOB had a call to investigate adding working groups
- We must acknowledge significant industry support for what is in place currently
- We also acknowledge significant effort that has been put in, paving a way forward for new features and capabilities to enable the evolving cloud-native ecosystem (in proposals; discussions; PoC; tangential communities; etc.)
- However, we need a path in the OCI for effective collaboration; the ability to move forward with both near-term and long-term solutions

## The IDEAL state

There is an ideal state that we believe all understand is the long term goal, given 6+ years of learnings and evolution in the cloud native world:

- A content addressable storage API
    - where the named/labeled "manifest" is not strictly bound to a container image
- A generic "index" datastructure for pointing to mime-typed blobs (equivalent to the image-index)
- A generic "manifest" data structure for further association of blobs (references), with adequate metadata/annotations
- various mediaType objects (of which OCI config and rootfs are a type)

## Our current state

- API's and data structures are very _container image_ oriented (image-index, image-manifest, and the registry API that processes them)
- The current API's and data structures are _widely_ implemented by companies, FOSS projects, and third party integrations
- Any significant departure from this state would have a long tail (if ever) of adoption
- The oci/image-spec image-index and image-manifest are _the_ protected and current "common" manifest, despite it being very container image focused
    - The oci/artifacts repo has arrived at the need for a "common" manifest type, to be able to point at arbitrary mediaTypes, but has effectively prohibited from that conversation because it currently is owned by the image-spec
    - [`opencontainers/artifacts`](https://github.com/opencontainers/artifacts) represented  different expectations to different folks since its inception. Without a working group structure in place, it has effectively been unchecked to evolve
    - Howerver, the initial comments and expections for `opencontainers/artifacts` stated it would be a table of media-types with guidance for each on how they should be handled via the distribution API. It now defines a new base manifest, and impacts the distribution API.
    - Even if the discussions/proposals within `opencontainers/artifacts` are working toward an **ideal** state (that we would like to have done since the beginning), it is a departure from what the industry currently supports. This is why it needs to decompose into respective near-term achievable efforts.
    - (vb) i feel personally responsible that this was unchecked, as 2 years ago I did not want to put the energy into it and instead focused on other things

## ✨ Our way forward

The ideal state (e.g. a "v2") will not replace today's "v1" and therefore increases support burden, implementation cost, and so on. Specs are not frozen, but enhancements are limited for this reason. Therefore we believe that:

- Solving today's challenges MUST prioritize extending v1 to enable the near term use-cases
- We need retroactive success criteria on existing de facto working groups:
    - The loose ["OCIv2" efforts](https://hackmd.io/@cyphar/ociv2-brainstorm)
    - New manifest specifications related to oci/artifacts
- If we deem that there is some re-arranging of specs needed (generic manifests; container image specific stuffs; distribution API handling;) then it'll likely require a TOB-level discussion of maintainership

The OCI has very open governance. As such, decision making requires rough consensus. We believe working on these items per the priorities stated above can achieve that end result.

## Conduct

- ASSUME POSITIVE INTENT
    - Humans are the best and worst part of working together
    - Transparency helps
    - Defensiveness often agitates, rather than seeks clarity
- Open conversations are expected
    - DMs are seldom leading to a consensus, and are adding to the confusion
    - Open conversation means more care and consideration is required
    - if folks feel they can not communicate in the open without being picked at, then it will limit the possibilities of this group
- Civil and considerate interactions expected
    - [refer to CoC](https://github.com/opencontainers/.github/blob/master/CODE_OF_CONDUCT.md)

## Notes and Links

- [Original distribution-spec PR for artifacts](https://github.com/opencontainers/distribution-spec/pull/65) (18 June 2019)
    - subsequent [oci/tob request for it's own repo](https://github.com/opencontainers/tob/issues/59) (still not as a spec)
    - [comment confirming this was about tracking media-types, not being a spec](https://github.com/opencontainers/tob/issues/59#issuecomment-512520486)
- [oci/artifacts proposal: reference types (stevelasker)](https://github.com/opencontainers/artifacts/pull/29/files?short_path=b335630#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5)
- [oci/image-spec Proposal: Add references (dlorenc)](https://github.com/opencontainers/image-spec/issues/827) (27 Mar 2021)
- [dlorenc recap of "references" situation](https://docs.google.com/document/d/1HVde05GKbJt4NpcBU7VsGihxdlPhgEwO3abwk3LPNNM/edit#heading=h.mchi1wh9bl9z)
- [SteveLasker proposal to decouple ](https://hackmd.io/iOemxuZaT8O5WciRLm0K9g?view)

[Steve Lasker]

Vincent, Phil,

What are the next steps and timeframes?

--
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@opencontainers.org.

[Vincent Batts]

For sure. I was hoping the wording for the "working group" PR would be further along. We're still having review on the comments and such.

I would think a group to figure out:

- how much "spec" has grown from the oci/artifacts effort and

- the overlap with what's already in distribution-spec and image-spec

- if image-index (and image-manifest?) need to grow to be more generic, then where should they live and

- who should the maintainers be (to include the guarding of breaking changes, and the amount of discussion that has occurred in the oci/artifacts repo)

We can get an outline of the discussion underway now, for what success looks like.

I feel this could actually be fairly quick, just needs clarity and consensus.

vb

[Steve Lasker]

What is the process and location to have this discussion?

[Steve Lasker]

We have the Reference Type working group as an outline, that was provided to facilitate the process, and the Discussion of a new manifest #41 to answer questions for why a new manifest was considered as a non-breaking, opt-in behavioral change that addresses the complexities associated with this type of change.  The premise was to run with the idea with a group of folks that are collaborating on a common goal, continue the experiments and validations to come to a point of recommendation.