With the introduction of the index.json in the OCI image format
specification it is now sufficiently advanced that flatpak can use it
as an full-featured alternative to OSTree. I just landed support for
this in flatpak.
I'm doing some interesting stuff with this, which I'd like to discuss.
But first, in case you're not aware of how flatpak works. Here is a
short demo of using it with an OCI registry (over http). These
commands are all run as an unprivileged user, although there is a
policykit helper that allows system-wide installation (per-user
installation is also possible, which is completely unprivileged).
# Add a remote OCI image repository and its gpg signature key
$ curl -sO https://sdk.gnome.org/test/oci/gpg.keyq
$ flatpak remote-add --oci --gpg-import=gpg.key test-oci https://sdk.gnome.org/test/oci/
# Here I got a policy-kit admin password prompt from the desktop
# List the contents of the repository
$ flatpak remote-ls test-oci -d
app/org.gnome.eog/x86_64/stable 00cd1468627d
app/org.gnome.gedit/x86_64/stable f76a5aecf26f
runtime/org.gnome.Platform.Locale/x86_64/3.22 00f5ef6621da
runtime/org.gnome.Platform/x86_64/3.22 4fad58cf5b13
runtime/org.gnome.eog.Locale/x86_64/stable a296faacea8a
runtime/org.gnome.gedit.Locale/x86_64/stable 2232f2f1a889
# Install gedit (and dependencies)
$ flatpak install test-oci org.gnome.gedit//stable
Required runtime for org.gnome.gedit/x86_64/stable (org.gnome.Platform/x86_64/3.22) is not installed, searching...
Found in remote test-oci, do you want to install it? [y/n]: y
Installing: org.gnome.Platform/x86_64/3.22 from test-oci
Receiving delta parts: 0/1 254,0 MB/254,0 MB
Installing: org.gnome.Platform.Locale/x86_64/3.22 from test-oci
Receiving delta parts: 0/1 109,4 MB/109,4 MB
Installing: org.gnome.gedit/x86_64/stable from test-oci
Receiving delta parts: 0/1 2,4 MB/2,4 MB
Installing: org.gnome.gedit.Locale/x86_64/stable from test-oci
Receiving delta parts: 0/1 3,2 MB/3,2 MB
# Try the app:
$ flatpak run org.gnome.gedit --version
gedit - Version 3.22.0
$ flatpak list -d
org.gnome.gedit/x86_64/stable test-oci 15fed5bef43d - 10,9 MB system,alt-id=f76a5aecf26f,current
org.gnome.Platform.Locale/x86_64/3.22 test-oci 783f410d8aa0 - 458,4 MB system,alt-id=00f5ef6621da,runtime /en
org.gnome.Platform/x86_64/3.22 test-oci 9da20a393cae - 681,0 MB system,alt-id=4fad58cf5b13,runtime
org.gnome.gedit.Locale/x86_64/stable test-oci 1c5e1439ee0b - 9,7 MB system,alt-id=2232f2f1a889,runtime /en
# Look for updates
$ flatpak update
Looking for updates...
Updating: org.gnome.Platform/x86_64/3.22 from test-oci
No updates.
Updating: org.gnome.Platform.Locale/x86_64/3.22 from test-oci
No updates.
Updating: org.gnome.gedit/x86_64/stable from test-oci
No updates.
Updating: org.gnome.gedit.Locale/x86_64/stable from test-oci
No updates.
# Also, we have human readable metadata in AppStream format, which
# is used by e.g. gnome-software
$ flatpak update --appstream test-oci
Receiving delta parts: 0/1 93,6 kB/93,6 kB
$ zcat /var/lib/flatpak/appstream/test-oci/x86_64/active/appstream.xml.gz | head
<?xml version="1.0" encoding="UTF-8"?>
<components version="0.8" origin="flatpak">
<component type="desktop">
<id>org.gnome.eog.desktop</id>
<translation type="gettext">eog</translation>
<name>Eye of GNOME</name>
<name xml:lang="ar">عين جنوم</name>
<name xml:lang="be@latin">Voka GNOME</name>
<name xml:lang="bn">জিনোমের চোখ</name>
<name xml:lang="br">Lagad GNOME</name>Here is the index.json it is using:
https://sdk.gnome.org/test/oci/index.jsonFor example, here is the part for gedit:
{
"mediaType" : "application/vnd.oci.image.manifest.v1+json",
"digest" : "sha256:f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbb",
"size" : 1780,
"annotations" : {
"org.flatpak.signature-digest" : "sha256:0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c",
"org.flatpak.installed-size" : "10920960",
"org.flatpak.metadata" : "[Application]\nname=org.gnome.gedit\nruntime=org.gnome.Platform/x86_64/3.22\nsdk=org.gnome.Sdk/x86_64/3.22\ncommand=gedit\n\n[Context]\nshared=ipc;\nsockets=x11;wayland;\nfilesystems=xdg-run/dconf;host;~/.config/dconf:ro;\n\n[Session Bus Policy]\nca.desrt.dconf=talk\n\n[Environment]\nDCONF_USER_CONFIG_DIR=.config/dconf\n\n[Extension org.gnome.gedit.Locale]\ndirectory=share/runtime/locale\nautodelete=true\n\n[Extension org.gnome.gedit.Debug]\ndirectory=lib/debug\nautodelete=true\nno-autodownload=true\n",
"org.opencontainers.ref.name" : "app/org.gnome.gedit/x86_64/stable",
"org.flatpak.download-size" : "2392357"
}
},
So, it is a regular image manifest, which you can find at
https://sdk.gnome.org/test/oci/blobs/sha256/f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbbThe
org.opencontainers.ref.name annotation is just set to the same
kind of identifier that flatpak uses in ostree (which is a
type/id/arch/branchname tuple), but we also have some org.flatpak
annotations, some (but not all) which are also extracted to the index.
The org.flatpak.installed-size and org.flatpak.download-size keys are
just informational for displaying in an "app-store" like UI. The
org.flatpak.metadata key is the internal flatpak metadata containing
things like dependencies and permissions requested by the app. I
don't think these are very interesting for non-flatpak use (although I
can see org.opencontainers.download-size possibly being useful).
However, org.flatpak.signature-digest is I think generically
interesting. Flatpak relies heavily on GPG signatures. Each remote is
configured with a set of private keys, and all the apps from a
repository has to be signed by one of them. It depends on this not
only to protect against network tampering, but also as a point of
trust for the sysadmin. So, the sysadmin can bless a repository+key
and allow users to install and/or update from it without having to
specify a password (as long as the the signs verify). This is very
similar to what linux distributions do with signed packages.
The implementation of this is pretty interesting too. We do all the
complex network stuff as a user, so when you download a remote image
we create a local OCI image repo with just the one image, and when
this is finished we hand over the final installation to a system
policykit helper that does local operations only (and gpg
verification).
Now, OCI doesn't currently have a GPG signature model. So i stole the
docker simple signature model from project atomic, and modified it for
use with OCI. See
https://github.com/mtrmac/image/blob/signature-json-schema/docs/atomic-signature.mdfor documentation how this looks.
In the OCI case, we have this in the index:
"org.flatpak.signature-digest" : "sha256:0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c",
And the corresponding blob is a gpg signature of an atomic json signature:
$ curl -s
https://sdk.gnome.org/test/oci/blobs/sha256/0cad4045e12cfaa2be81e48cd18e2cdee6a71e9882340a71f126dcb384549b8c | gpg -d
{
"critical" : {
"type" : "flatpak oci image signature",
"image" : {
"oci-image-manifest-digest" : "sha256:f76a5aecf26f7feb22429e1a10fd125cef639b2c32fcc76badeb2b75fda29cbb"
},
"identity" : {
"oci-image-ref" : "app/org.gnome.gedit/x86_64/stable"
}
},
"optional" : {
"creator" : "flatpak 0.9.1",
"timestamp" : 1490788634
}
}
gpg: Signature made ons 29 mar 2017 13:57:14 CEST using RSA key ID 2CAA7666
gpg: Can't check signature: public key not found
Basically I changed the type, image and identity to say "oci" and "flatpak", instead of "docker" and "atomic".
This format seems like it could be generally useful though. Maybe we could drop the "flatpak" part from the type
and store it as "org.opencontainers.signature-digest"?