ECR does not expose S3 or Cloudfront

[Hippie Hacker]

Last week ii looked into the tooling and framework available for serving images from AWS ECR Private / Public. As far as I can tell, even with full credentials, we are unable to create signed urls for the underlying s3 buckets backing ECR.

See [ECR Public and Cloudfront Research](https://hackmd.io/@TKToYPauRJ-u_mNRBOh4HQ/Sy8qyJbRY)

Neither S3 or Cloudfront are exposed for configuration and appear to be fully managed behind ECR Private or Public instances. It might be possible to have our proxy make ECR requests to resolve the ECR URI for CloudFront, tracking/caching the auth, tracking the TTL for expiring Cloudfront blobs. However this feels messy and overy complicated.

Ideally AWS would expose or allow access to the CloudFront+s3 underpinning ECR. If we choose to not use ECR we will still need a way of regionalising s3 bucket access and autoredirecting within s3. Multi-Region Access Points or Cloudfront are some options there.

To verify, Caleb pulled an image from k8s.gcr.io, retagged it and pushed it to ECRPublic. Using crane he was able to download a blob using it’s blob digest, however the direct URI is not accessible without auth.

When hitting the exact URI through cURL, it goes through a couple stages: auth, the initial URI, finally a cloudfront signed URI for the blob. The flow does not include any directy s3 urls.

Also there are some minor differences between ECR (Private) and ECR Public.

Both:
  - **very managed**
  - backended via CloudFront and don’t expose S3 buckets
  - neither S3 or CloudFront are exposed or configurable

ECR (Private)
  - provides multi-regioning under the hood via S3+CloudFront
  - provides immutable tags

ECR Public
  - provides a public URL that’s customizable
  - doesn’t provide immutable tags

Caleb and I continue to explore a way to utilise ECR (Public or Private), but I wanted to ensure we had wider conversation on where we currently are.

Cheers,

Hippie Hacker

[Benjamin Elder]

Thanks for looking into this in detail!

> Ideally AWS would expose or allow access to the CloudFront+s3 underpinning ECR. If we choose to not use ECR we will still need a way of regionalising s3 bucket access and autoredirecting within s3. Multi-Region Access Points or Cloudfront are some options there.

It's also possible that the redirector application can handle regionalizing requests, since the IP range data we'd use to detect "this is an AWS client" also appears to publish region specific ranges. We could direct to s3 per-region.

Currently k8s.gcr.io does something similar, directing to {eu,us,asia}.gcr.io actual backing stores.

Am I reading this rundown correctly that ECR public does not regionalize?

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-k8s-infra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-k8s...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-k8s-infra/25a51fd9-2aaa-4364-9668-3faea103d594n%40googlegroups.com.

[Hippie Hacker]

Sorry, that should have been under a feature for both Public and Private.

It's a bit opaque as to how Cloudfront / S3 are handled behind the scenes.

[Tim Bannister]

ECR Public is framed (by AWS) as a global service / global API. There are multiple edge locations (around the world) for CloudFront and I'd assume that this is already handled for you as part of the managed service.

Tim