Default location for artifact signing key

[Knut Ørland]

Hello!

I am in the progress of integrating Mender in our product, and it is going very well so far!

I just wanted to mention a discrepancy I found in the documentation regarding the artifact signing public key on the following page:

https://docs.mender.io/1.6/artifacts/building-for-production#artifact-signing-and-verification-keys

It states that the public artifact signing key is stored in the data-partition by default, but when I went looking for it on my data partition, it was nowhere to be found!

It seems like the default (for the meta-mender layer) is to put it in /etc/mender/artifact-verify-key.pem:

https://github.com/mendersoftware/meta-mender/blob/sumo-v2018.10/meta-mender-core/recipes-mender/mender/mender.inc#L189

I'm not sure what the correct "fix" here is, I just though that I should mention it to avoid some possible confusion.

- Knut

[Mirza Krak]

On Wed, Oct 31, 2018 at 2:35 PM Knut Ørland <k...@zaptec.com> wrote:

Hello!

Hello Knut!

Thanks for reporting this.

I do not actually see a problem with it being in /etc/mender/artifact-verify-key.pem by default, which means that I would update the docs as "the fix".

But lets see if anyone else has opinions about this.

--

Mirza Krak | Embedded Solutions Architect | https://mender.io

 Northern.tech AS | @northerntechHQ

[Don Cross]

Hi everyone.

On Wed, Oct 31, 2018 at 11:25 AM Mirza Krak <mirza...@northern.tech> wrote:

On Wed, Oct 31, 2018 at 2:35 PM Knut Ørland <k...@zaptec.com> wrote:

I just wanted to mention a discrepancy I found in the documentation regarding the artifact signing public key on the following page:

https://docs.mender.io/1.6/artifacts/building-for-production#artifact-signing-and-verification-keys

It states that the public artifact signing key is stored in the data-partition by default, but when I went looking for it on my data partition, it was nowhere to be found!

In my own non-Yocto Mender environment, I ended up putting my artifact verification public key on the data partition, based on the wording in the above link.  And this makes sense to me because I wanted to use a consistent key pair across all deployments. Once I know it is working, I know it will keep working, and it will be secure so long as I keep the private key a secret.

The documentation above mentions that you can choose to put the artifact public key in the artifact itself, if you want to rotate keys. But I think changing the key is dangerous. Here is an example. Imagine you have three Mender deployments called A, B, C.  You have many customer devices all on A.  Suppose also that you have 3 different verification keys KA, KB, KC in the respective deployments.  In order to update from A to B, the key KA has to verify deployment B.  That means B has to be signed with the private counterpart of the public key KA.  Now if you do an update to B, everything works fine. 

But what if some machines are offline for a week or two (maybe just turned off for some reason) and never see the update to B.  Then you release an update C that requires verification with KB.  Then one of those machines that is still on update A may get powered on. This machine tries to update to C, using KA to verify. It will think update C is fake because KA does not verify it. It does not have KB, which is the key that would verify it.  At this point, that machine will never be able to update again; it is permanently stuck at update A. The chain has been broken!

In general, I don't see how key rotation is safe unless you can verify that *all* customer devices have been updated and are all on the same deployment with the same, modified, public key.

It seems so much simpler and safer to always use the same key pair. Then you might as well keep it on the data partition.  It's not really "bad" to put the key in the deployment itself, on the root partition, so long as the key never changes.

So my recommendation would be to change the documentation to say that the key should remain the same across all deployments, whether it is stored in the data partition or the root partition.

Don

[Dell Green]

maybe in that scenario, the update that contains the new key also needs to manipulate MENDER_DEVICE_TYPE and/or  MENDER_DEVICE_TYPES_COMPATIBLE to in future pull from a new set of deployments?

[Drew Moseley]

There was some discussion of implementing update ordering to address this and other similar situations; in some cases it is not ok to go directly from rev A to rev C without forcing the device to go to rev B.  There are situations other than simple key rotation that I can envision would require something like this. I'm pretty sure it's not planned to implemented in the immediate future but it is certainly something being considered.

It feels to me (without any real evidence) that changing the DEVICE_TYPE would introduce other issues. I agree that changing the key is problematic and it should probably not be done but I wonder if leaving it in /etc/ is a better approach as it will give users the ability to change if/when update ordering is available.

Drew