Consuming Public Content Article

[Steve Lasker]

The recent Docker Terms of Service (TOS) updates reflect a growing challenge for consuming publicly curated content.

While the TOS updates impose throttles on frequent content pulls, which can impact production workloads, it raises the question on how much impact public content should have on production workloads. It asks the question, who is responsible for bearing the costs of hosting multi gigabyte content and who bears the responsibility of assuring the content is accessible and secure for each environment, 100% of the time. The recent Docker TOS changes forces us to ask a broader set of questions that, as a community, we can address. The problem isn’t limited to production container images, but extends to all package manager content.

 

On Monday, October 12, 2020, representatives from Azure, AWS, Google, IBM, Docker and GitHub met to discuss how, as an industry, we believe we can address the problem. We need a short term solution for the pending November 1^st changes that enable stability through the holiday lockdowns, with a longer term plan that helps customers adapt their workflows.

 

To avoid customer confusion, the cloud vendors and docker have agreed to co-author a paper providing guidance to customers for how they can adapt their workflows in incremental steps. The goal is to provide stability to an ecosystem, with an opportunity to innovate in a common direction. The Open Container Initiative (OCI) was intended to provide a vendor neutral body of governance. We would like to publish the paper under the OCI umbrella in the coming weeks.

 

The content will be something like:

 

• Public content is great to consume, but consider the risks when sourcing public content in mainline/production workflows.
• Consuming public content imposes production risks as it assumes all connections between the production host and the public source are fully available. It assumes the content is in secure and reliable state, compared to the previously tested version. And, it assumes someone is bearing the costs to host that continually pulled, large content.
• As a best practice, import public content to a private registry, do some amount of incremental testing, and utilize from your private registry
• This includes, not building FROM docker hub, rather build FROM your private registry the content you’ve imported and verified
• Automate the importing and verification, to assure you have the latest, but test the latest to assure its safe and reliable for your environment
• As you adapt your import/verify/promote workflows, docker hub users can provide authentication to avoid throttle limits

 

Due to the pending holiday lockdowns, and the November 1^st Docker TOS changes, we’d like to co-author this paper, released under OCI by the end of October.

 

We ask the OCI to support, and where applicable, contribute to the article that we would host under the OCI Blog.

Each cloud vendor could reference this article, with their cloud/ISV specific guidance on how to implement it with their services and products.

 

We’ve added this to this weeks OCI agenda and welcome input here and on the call.

[Chris Aniszczyk]

Do we have a draft of the article yet for feedback, like a gdoc or something easy to comment?

I think opening up an issue on the TOB repo https://github.com/opencontainers/tob/issues with a link to the doc for feedback with a simple vote once you have a final draft is the way to go.

From an OCI/LF perspective, we are happy to promote this and do a press release to further promote this guidance if this is truly something that you want to be more visible across the industry.

--
To unsubscribe from this group and stop receiving emails from it, send an email to tob+uns...@opencontainers.org.

--

Chris Aniszczyk (@cra)

[Steve Lasker]

Thanks Chris,

Here’s the frame of the doc: https://docs.google.com/document/d/1fxayMznIkszBI9Y2S3KGSyi2hFMwUIwDfn3D2wQcye4/edit#heading=h.o8fmuf71a6g5

I’ve intentionally not written it yet as I wanted to leave it open for others to help, possibly take it a direction I hadn’t thought of.

I was going to hold a meeting Friday, for those interested in co-authoring and get next steps down.

I’ll open the TOB issue.

 

Thanks for the great support.

[Steve Lasker]

A quick update:

The Consuming Public Content article is close to final review: https://docs.google.com/document/d/1fxayMznIkszBI9Y2S3KGSyi2hFMwUIwDfn3D2wQcye4/edit?usp=sharing

Our plan is to have it ready for final review EOD today, with Friday, 10/23 as a last min edit day

The OCI TOB would “take ownership” with the hopes to publish sometime next week. The Docker TOS changes go into effect November 1^st.

 

This would be a great time for anyone that has concerns to please raise them so we can quickly adapt.

 

Thanks,

Steve

[Steve Lasker]

The blog post is up and ready for a vote. To provide the Blog team time, and to complete for the pending Nov 1 TOS changes, please vote by: EOD Oct 26, 2020

https://github.com/opencontainers/tob/issues/89#issuecomment-716197921

[Steve Lasker]

For those that haven’t been following the recent Docker Terms of Service changes, it has put a scramble on the cloud providers and ISVs who’s users depend upon docker hub.

Because of the multi-tenant nature of cloud services, the anonymous pulls get aggregated. The cloud providers and ISVs are facing a range of user outages, just as we head into the holidays.

 

While those that look closely may blame Docker Inc., the result is the container industry as a whole will take a hit as upset users are just upset. The blame game doesn’t mitigate their frustration.

 

To mitigate this, we’re being as proactive as possible, and working in unison to not blame Docker, degrading the value of containers. Rather, to question the consumption of public content in critical workloads and focus on what can be done to incorporate public content, but not be dependent for critical workloads.

 

We’ve outlined this with an immediate call to action – provide docker authentication, with a longer term workflow.

 

We see this is an opportunity for OCI to continue being a leading voice to the vendor-neutral container ecosystem.

 

This vote doesn’t require standing in line…

 

Note: I didn’t have the following emails: Michael Crosby, Wei Fu, so apologies for not including directly.