Flatpak support for OCI image format, and possible additions to spec

[Alexander Larsson]

With the introduction of the index.json in the OCI image format
specification it is now sufficiently advanced that flatpak can use it
as an full-featured alternative to OSTree. I just landed support for
this in flatpak.

I'm doing some interesting stuff with this, which I'd like to discuss.

But first, in case you're not aware of how flatpak works. Here is a
short demo of using it with an OCI registry (over http). These
commands are all run as an unprivileged user, although there is a
policykit helper that allows system-wide installation (per-user
installation is also possible, which is completely unprivileged).

   # Add a remote OCI image repository and its gpg signature key
   $ curl -sO https://sdk.gnome.org/test/oci/gpg.keyq
   $ flatpak remote-add --oci --gpg-import=gpg.key test-oci https://sdk.gnome.org/test/oci/
   # Here I got a policy-kit admin password prompt from the desktop

   # List the contents of the repository
   $ flatpak remote-ls test-oci -d
   app/org.gnome.eog/x86_64/stable               00cd1468627d
   app/org.gnome.gedit/x86_64/stable             f76a5aecf26f
   runtime/org.gnome.Platform.Locale/x86_64/3.22 00f5ef6621da
   runtime/org.gnome.Platform/x86_64/3.22        4fad58cf5b13
   runtime/org.gnome.eog.Locale/x86_64/stable    a296faacea8a
   runtime/org.gnome.gedit.Locale/x86_64/stable  2232f2f1a889

   # Install gedit (and dependencies)
   $ flatpak install test-oci org.gnome.gedit//stable
   Required runtime for org.gnome.gedit/x86_64/stable (org.gnome.Platform/x86_64/3.22) is not installed, searching...
   Found in remote test-oci, do you want to install it? [y/n]: y
   Installing: org.gnome.Platform/x86_64/3.22 from test-oci
   Receiving delta parts: 0/1 254,0 MB/254,0 MB
   Installing: org.gnome.Platform.Locale/x86_64/3.22 from test-oci
   Receiving delta parts: 0/1 109,4 MB/109,4 MB
   Installing: org.gnome.gedit/x86_64/stable from test-oci
   Receiving delta parts: 0/1 2,4 MB/2,4 MB
   Installing: org.gnome.gedit.Locale/x86_64/stable from test-oci
   Receiving delta parts: 0/1 3,2 MB/3,2 MB

   # Try the app:
   $ flatpak run org.gnome.gedit --version
   gedit - Version 3.22.0

   $ flatpak list -d
   org.gnome.gedit/x86_64/stable         test-oci 15fed5bef43d - 10,9 MB  system,alt-id=f76a5aecf26f,current
   org.gnome.Platform.Locale/x86_64/3.22 test-oci 783f410d8aa0 - 458,4 MB system,alt-id=00f5ef6621da,runtime /en
   org.gnome.Platform/x86_64/3.22        test-oci 9da20a393cae - 681,0 MB system,alt-id=4fad58cf5b13,runtime
   org.gnome.gedit.Locale/x86_64/stable  test-oci 1c5e1439ee0b - 9,7 MB   system,alt-id=2232f2f1a889,runtime /en

   # Look for updates
   $ flatpak update
   Looking for updates...
   Updating: org.gnome.Platform/x86_64/3.22 from test-oci
   No updates.
   Updating: org.gnome.Platform.Locale/x86_64/3.22 from test-oci
   No updates.
   Updating: org.gnome.gedit/x86_64/stable from test-oci
   No updates.
   Updating: org.gnome.gedit.Locale/x86_64/stable from test-oci
   No updates.

   # Also, we have human readable metadata in AppStream format, which
   # is used by e.g. gnome-software
   $ flatpak update --appstream test-oci
   Receiving delta parts: 0/1 93,6 kB/93,6 kB
   $ zcat /var/lib/flatpak/appstream/test-oci/x86_64/active/appstream.xml.gz | head
   <?xml version="1.0" encoding="UTF-8"?>
   <components version="0.8" origin="flatpak">
     <component type="desktop">
       <id>org.gnome.eog.desktop</id>
       <translation type="gettext">eog</translation>
       <name>Eye of GNOME</name>
       <name xml:lang="ar">عين جنوم</name>
       <name xml:lang="be@latin">Voka GNOME</name>
       <name xml:lang="bn">জিনোমের চোখ</name>
       <name xml:lang="br">Lagad GNOME</name>

Here is the index.json it is using: https://sdk.gnome.org/test/oci/index.json
For example, here is the part for gedit:

    {
      "mediaType" : "application/vnd.oci.image.manifest.v1+json",
      "digest" : "sha256:f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbb",
      "size" : 1780,
      "annotations" : {
        "org.flatpak.signature-digest" : "sha256:0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c",
        "org.flatpak.installed-size" : "10920960",
        "org.flatpak.metadata" : "[Application]\nname=org.gnome.gedit\nruntime=org.gnome.Platform/x86_64/3.22\nsdk=org.gnome.Sdk/x86_64/3.22\ncommand=gedit\n\n[Context]\nshared=ipc;\nsockets=x11;wayland;\nfilesystems=xdg-run/dconf;host;~/.config/dconf:ro;\n\n[Session Bus Policy]\nca.desrt.dconf=talk\n\n[Environment]\nDCONF_USER_CONFIG_DIR=.config/dconf\n\n[Extension org.gnome.gedit.Locale]\ndirectory=share/runtime/locale\nautodelete=true\n\n[Extension org.gnome.gedit.Debug]\ndirectory=lib/debug\nautodelete=true\nno-autodownload=true\n",
        "org.opencontainers.ref.name" : "app/org.gnome.gedit/x86_64/stable",
        "org.flatpak.download-size" : "2392357"
      }
    },

So, it is a regular image manifest, which you can find at
https://sdk.gnome.org/test/oci/blobs/sha256/f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbb

The org.opencontainers.ref.name annotation is just set to the same
kind of identifier that flatpak uses in ostree (which is a
type/id/arch/branchname tuple), but we also have some org.flatpak
annotations, some (but not all) which are also extracted to the index.

The org.flatpak.installed-size and org.flatpak.download-size keys are
just informational for displaying in an "app-store" like UI. The
org.flatpak.metadata key is the internal flatpak metadata containing
things like dependencies and permissions requested by the app.  I
don't think these are very interesting for non-flatpak use (although I
can see org.opencontainers.download-size possibly being useful).

However, org.flatpak.signature-digest is I think generically
interesting.  Flatpak relies heavily on GPG signatures. Each remote is
configured with a set of private keys, and all the apps from a
repository has to be signed by one of them. It depends on this not
only to protect against network tampering, but also as a point of
trust for the sysadmin. So, the sysadmin can bless a repository+key
and allow users to install and/or update from it without having to
specify a password (as long as the the signs verify). This is very
similar to what linux distributions do with signed packages.

The implementation of this is pretty interesting too. We do all the
complex network stuff as a user, so when you download a remote image
we create a local OCI image repo with just the one image, and when
this is finished we hand over the final installation to a system
policykit helper that does local operations only (and gpg
verification).

Now, OCI doesn't currently have a GPG signature model. So i stole the
docker simple signature model from project atomic, and modified it for
use with OCI. See https://github.com/mtrmac/image/blob/signature-json-schema/docs/atomic-signature.md
for documentation how this looks.

In the OCI case, we have this in the index:
  "org.flatpak.signature-digest" : "sha256:0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c",

And the corresponding blob is a gpg signature of an atomic json signature:

   $ curl -s https://sdk.gnome.org/test/oci/blobs/sha256/0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c | gpg -d
   {
     "critical" : {
       "type" : "flatpak oci image signature",
       "image" : {
         "oci-image-manifest-digest" : "sha256:f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbb"
       },
       "identity" : {
         "oci-image-ref" : "app/org.gnome.gedit/x86_64/stable"
       }
     },
     "optional" : {
       "creator" : "flatpak 0.9.1",
       "timestamp" : 1490788634
     }
   }
   gpg: Signature made ons 29 mar 2017 13:57:14 CEST using RSA key ID 2CAA7666
   gpg: Can't check signature: public key not found

Basically I changed the type, image and identity to say "oci" and "flatpak", instead of "docker" and "atomic".
This format seems like it could be generally useful though. Maybe we could drop the "flatpak" part from the type
and store it as "org.opencontainers.signature-digest"?

[W. Trevor King]

On Thu, Mar 30, 2017 at 03:53:25AM -0700, Alexander Larsson wrote:
> In the OCI case, we have this in the index:
> "org.flatpak.signature-digest" :
> "sha256:0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c",
>
> And the corresponding blob is a gpg signature of an atomic json signature:
>
> $ curl -s
> https://sdk.gnome.org/test/oci/blobs/sha256/0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c
> | gpg -d
> {
> "critical" : {
> "type" : "flatpak oci image signature",
> "image" : {
> "oci-image-manifest-digest" :
> "sha256:f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbb"
> },
> "identity" : {
> "oci-image-ref" : "app/org.gnome.gedit/x86_64/stable"
> }
> },
> "optional" : {
> "creator" : "flatpak 0.9.1",
> "timestamp" : 1490788634
> }
> }
> gpg: Signature made ons 29 mar 2017 13:57:14 CEST using RSA key ID
> 2CAA7666
> gpg: Can't check signature: public key not found

This is fairly close to my signed name-assertion proposal [1]. The
main differences seem to be that your approach generates DAGs like:

referencing descriptor
* {digest, size, media-type} -> named thing
* {org.flatpak.signature-digest} -> signed-name-assertion

signed-name-assertion (name assertion with attached signature)
* {oci-image-manifest-digest} -> named thing

While my signed name-assertion proposal generates DAGs like:

referencing descriptor
* {digest, size, media-type} -> application/vnd.oci.image.signed.blob.v1+json

application/vnd.oci.image.signed.blob.v1+json
* blob {digest, size, media-type} -> application/vnd.oci.image.named.blob.v1+json
* signatures[]
* {digest, size, media-type} -> detached signature

application/vnd.oci.image.named.blob.v1+json (asserting a name)
* blob {digest, size, media-type} -> named thing

The benefit of your annotation-based approach is that you don't need
to teach all consumers about your annotation; consumers which don't
understand your annotation will just ignore the signature.

The benefit of my signed.blob and named.blob objects is that it
follows the SHOULD for references [2], so CAS engines and other
walkers looking for descriptor-like things can generically walk the
DAG (e.g. for garbage-collection [3]).

There's also some related discussion on signatures in [4], although
that's a bit more abstract than our proposals.

Cheers,
Trevor

[1]: https://github.com/opencontainers/image-spec/issues/176
[2]: https://github.com/opencontainers/image-spec/blame/v1.0.0-rc5/descriptor.md#L8
[3]: https://github.com/openSUSE/umoci/blob/v0.1.0/oci/cas/gc.go#L40-L120
[4]: https://github.com/opencontainers/image-spec/issues/400

--
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[Stephen Day]

Awesome stuff!

> Maybe we could drop the "flatpak" part from the type
and store it as "org.opencontainers.signature-digest"?

How can you trust the information in the image if you embed the digest of the signature within? It seems like an attacker could just replace the contents of "org.flatpak.signature-digest" with whatever they want since you haven't verified anything at the top-level.

Why not just lookup the signature based on the digest of the content of the content? Why does it need to be embedded in the format?

--
You received this message because you are subscribed to the Google Groups "dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@opencontainers.org.

[W. Trevor King]

On Thu, Mar 30, 2017 at 08:33:16PM +0000, Stephen Day wrote:
> > Maybe we could drop the "flatpak" part from the type and store it
> > as "org.opencontainers.signature-digest"?
>
> How can you trust the information in the image if you embed the
> digest of the signature within?

The same way folks trust OpenPGP-signed emails (like this one) [1,2]:
you start reading until you find a “this is signed” marker
(Alexander's org.flatpak.signature-digest or OpenPGP's
multipart/signed [3]. Anything outside of that (e.g. Alexander's
index.json and the headers on an OpenPGP-signed email) should not be
trusted. Anything inside can be trusted (assuming the signature
checks out).

> It seems like an attacker could just replace the contents of
> "org.flatpak.signature-digest" with whatever they want since you
> haven't verified anything at the top-level.

If your consumer policy is to require signatures on, say, referenced
image manifests, then this replacement would be a denial of service
attach (the consumer would error out with “could not find a manifest
with a valid signature”). And if an attacker has access to
index.json, they can already DoS you just by emptying the contents
(regardless of how you setup your signing system).

> Why not just lookup the signature based on the digest of the content
> of the content?

Then you have to define an external signature-lookup framework like
Notary [4]. That's fine, but it's another service to implement, and
being able to bundle signatures along with the rest of the image in
CAS is convenient.

Cheers,
Trevor

[1]: https://tools.ietf.org/html/rfc3156
[2]: https://tools.ietf.org/html/rfc4880
[3]: https://tools.ietf.org/html/rfc3156#section-5
[4]: https://github.com/docker/notary

[Stephen Day]

Trevor:

My questions were directed at Alexander. Please allow him to respond.

Stephen.

[Alexander Larsson]

On Thursday, 30 March 2017 17:22:19 UTC+2, W. Trevor King wrote:

On Thu, Mar 30, 2017 at 03:53:25AM -0700, Alexander Larsson wrote:

This is fairly close to my signed name-assertion proposal [1].  The
main differences seem to be that your approach generates DAGs like:

  referencing descriptor
  * {digest, size, media-type} -> named thing
  * {org.flatpak.signature-digest} -> signed-name-assertion

  signed-name-assertion (name assertion with attached signature)
  * {oci-image-manifest-digest} -> named thing

While my signed name-assertion proposal generates DAGs like:

  referencing descriptor
  * {digest, size, media-type} -> application/vnd.oci.image.signed.blob.v1+json

  application/vnd.oci.image.signed.blob.v1+json
  * blob {digest, size, media-type} -> application/vnd.oci.image.named.blob.v1+json
  * signatures[]
    * {digest, size, media-type} -> detached signature

  application/vnd.oci.image.named.blob.v1+json (asserting a name)
  * blob {digest, size, media-type} -> named thing

The benefit of your annotation-based approach is that you don't need
to teach all consumers about your annotation; consumers which don't
understand your annotation will just ignore the signature.

Yeah, understanding signatures is optional, and the entire system can work with signatures coming from other locations than the OCI repository itself too. For instance, project atomic uses some system where the signatures are side-loaded with another mechanism. The json for the signature would be the same, but not loaded from the OCI repo itself.
 

The benefit of my signed.blob and named.blob objects is that it
follows the SHOULD for references [2], so CAS engines and other
walkers looking for descriptor-like things can generically walk the
DAG (e.g. for garbage-collection [3]).

What is the difference here, in your propsal you have to specifically follow the new signed blob to the real manifest, so a generic GC would not find that either.

[Alexander Larsson]

On Thursday, 30 March 2017 22:33:29 UTC+2, Stephen Day wrote:

Awesome stuff!

> Maybe we could drop the "flatpak" part from the type
and store it as "org.opencontainers.signature-digest"?

How can you trust the information in the image if you embed the digest of the signature within? It seems like an attacker could just replace the contents of "org.flatpak.signature-digest" with whatever they want since you haven't verified anything at the top-level.

If you replace the digest then the image is no longer signed (or its signed with the wrong key or garbage or whatever), and this will cause flatpak to report a GPG error.

The trust is based on the image manifest reference name and digest being signed with a pre-declared gpg key, not that someone is unable to modify the OCI repo.
 

Why not just lookup the signature based on the digest of the content of the content? Why does it need to be embedded in the format?

That is not what is being asserted though. We don't just want to sign the contents. We want to sign that this content has this name. So, we need to sign a statement containing both the ref and the content.

If this is not done then anyone could take say a repo with images all signed with one key, and then replace one image with another (possibly vulnerable) one.

[Alexander Larsson]

I'd like to point out further, that even if you *do* sign the ref and the manifest, this still allows a MITM attacker to enforce a downgrade to an earlier (possibly vulnerable) versions, so implementations of this should additionally verify some kind of timestamp inside the signed contents, and refuse to update if time goes backwards. We have such a timestamp in the image configuration. Flatpak has an annotation for the timestamp so that this can be checked directly in the image manifest itself, so maybe a similar timestamp should be recommended if other people want to use this setup too.
 
That leaves the option for a MITM attacker to do a "prevent updates" attack. It is possible to design systems that will detect this case and warn the user (see for instance https://theupdateframework.github.io/), but this vastly complicates things for an imho small gain.

[W. Trevor King]

Why not? It finds a reference that looks like a descriptor (digest,
size, and mediaType [1]) and walks that link. But that only works as
long as all Merkle links are descriptor based (which mine are, and
your org.flatpak.signature-digest annotation is not). The best of
both worlds (an optional descriptor), might be to move your property
outside of annotations:


{

"mediaType" : "application/vnd.oci.image.manifest.v1+json",
"digest" : "sha256:f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbb",
"size" : 1780,

"org.flatpak.signature": {
"mediaType": "application/pgp-signature",
"digest" : "sha256:0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c",
"size": 588,
}
"annotations" : {

"org.flatpak.installed-size" : "10920960",
"org.flatpak.metadata" : "[Application]\nname=org.gnome.gedit\nruntime=org.gnome.Platform/x86_64/3.22\nsdk=org.gnome.Sdk/x86_64/3.22\ncommand=gedit\n\n[Context]\nshared=ipc;\nsockets=x11;wayland;\nfilesystems=xdg-run/dconf;host;~/.config/dconf:ro;\n\n[Session Bus Policy]\nca.desrt.dconf=talk\n\n[Environment]\nDCONF_USER_CONFIG_DIR=.config/dconf\n\n[Extension org.gnome.gedit.Locale]\ndirectory=share/runtime/locale\nautodelete=true\n\n[Extension org.gnome.gedit.Debug]\ndirectory=lib/debug\nautodelete=true\nno-autodownload=true\n",
"org.opencontainers.ref.name" : "app/org.gnome.gedit/x86_64/stable",
"org.flatpak.download-size" : "2392357"
}
},

On Fri, Mar 31, 2017 at 04:50:52AM -0700, Alexander Larsson wrote:
> I'd like to point out further, that even if you *do* sign the ref and the
> manifest, this still allows a MITM attacker to enforce a downgrade to an

> earlier (possibly vulnerable) versions…

More prior-art sketches along these lines in [2].

Cheers,
Trevor

[1]: https://github.com/openSUSE/umoci/blob/v0.1.0/oci/cas/gc.go#L35-L38
[2]: https://github.com/ipfs/specs/tree/143319a99a9f191c1dfd50814623b9b7a4752254/iprs#example-record-types

[Stephen Day]

Alexander:

> That is not what is being asserted though. We don't just want to sign the contents. We want to sign that this content has this name. So, we need to sign a statement containing both the ref and the content.

I am not sure if I was clear above, but this assertion could be indexed via the digest of the target object. That statement could contain anything. Both ensuring that the data within the target is verifiable, as well as the assertion of the statement. 

> I'd like to point out further, that even if you *do* sign the ref and the manifest, this still allows a MITM attacker to enforce a downgrade to an earlier (possibly vulnerable) versions, so implementations of this should additionally verify some kind of timestamp inside the signed contents, and refuse to update if time goes backwards.

Like you said, this would be a property of the signing store. Notary or TUF can handle this kind of thing. While I suggested an "indexed" or "lookup" model above, the actual use includes traversing through a signed name assertion, which is more user friendly. Mostly, I am advocating for a decentralized approach. In general, we can get more flexible security models that can remain fully decoupled.

Perhaps this discussion is moot if "org.flatpak.signature-digest" points to a signature of the flatpak internals. There is no reason not be able to work in multiple signing contexts and have them work independently, as long as the unverified "header" statements can be validated independently.

Great to see this validating the format!

[W. Trevor King]

On Fri, Mar 31, 2017 at 08:53:14PM +0000, Stephen Day wrote:
> There is no reason not be able to work in multiple signing contexts
> and have them work independently, as long as the unverified "header"
> statements can be validated independently.

Although the image spec doesn't have much to say about it at the
moment, “Specifying way to attach signatures” is currently in-scope
for the OCI as an optional layer [1] with two open issues [2,3]. So
while having the ability to plug in your favorite signing approach is
great, I think it would be beneficial for interop if the image-spec
made some SHOULD-level recommendations on this front. That would
avoid needless community fragmentation while still allowing folks with
strong opinions to do their own thing. Both of the open issues are
labeled post-v1.0.0, so there's no need to rush into anything, but I
don't think “do whatever you want” is the best long-term policy.

Cheers,
Trevor

[1]: https://www.opencontainers.org/about/oci-scope-table
[2]: https://github.com/opencontainers/image-spec/issues/22
[3]: https://github.com/opencontainers/image-spec/issues/400

[Alexander Larsson]

On Friday, 31 March 2017 22:53:27 UTC+2, Stephen Day wrote:

Alexander:

> That is not what is being asserted though. We don't just want to sign the contents. We want to sign that this content has this name. So, we need to sign a statement containing both the ref and the content.

I am not sure if I was clear above, but this assertion could be indexed via the digest of the target object. That statement could contain anything. Both ensuring that the data within the target is verifiable, as well as the assertion of the statement. 

> I'd like to point out further, that even if you *do* sign the ref and the manifest, this still allows a MITM attacker to enforce a downgrade to an earlier (possibly vulnerable) versions, so implementations of this should additionally verify some kind of timestamp inside the signed contents, and refuse to update if time goes backwards.

Like you said, this would be a property of the signing store. Notary or TUF can handle this kind of thing. While I suggested an "indexed" or "lookup" model above, the actual use includes traversing through a signed name assertion, which is more user friendly. Mostly, I am advocating for a decentralized approach. In general, we can get more flexible security models that can remain fully decoupled.

Perhaps this discussion is moot if "org.flatpak.signature-digest" points to a signature of the flatpak internals. There is no reason not be able to work in multiple signing contexts and have them work independently, as long as the unverified "header" statements can be validated independently.

The atomic simple signature stuff *also* uses an external lookup for the signature, and I don't think any recommendation we make should disallow this.

However, in my mind serving the signatures on the side seems like an overly complicated setup. I mean, all we have to do is deliver signed statements, indexed by a name and an image manifest, and we already have such an index in the index.json. Having to have an entire new service for this seems unnecessary.

For example, consider the case for an upstream free software project (say the Gimp) that want to deliver releases of its software using flatpak and OCI. Hosting an OCI repositoiry just means a static HTTP server. But if they want to also supply signatures for it, then they have to run an instance of notary, and they also have to tell all the users to configure trust in that instance. In comparison, distributing the signatures inside the OCI repo just means a few extra files on the webserver.

Am I missing something important that makes the side-channel model better?