The encodings bikeshed

[Filippo Valsorda]

It's time to pick the bikeshed's encoding. Here are two things I'd like to fix.

1. Using _ in keys is not markup safe, both in some Markdown parsers and in simpler markups.
   
2. It's very easy to copy-paste the armor wrong, I forgot the first line myself a couple times.
   

For (1) I think I like switching the keys to Bech32. It selects as a single word, it doesn't have symbols, and it's case insensitive so it can be read out loud. (I might read an age key on stage at RWC in a lighting talk as a tribute to Ian Goldberg.)

# created: 2019-12-19T00:02:55-05:00

# public key: age14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa

AGE-SECRET-KEY-1FWY45RKECMS2XN3R9W43K9QX07NJ5QKN4KPKQXZ0053HEHALY4SQT8020E

I initially used pubkey: instead of age: because "age age:81gSRoyO7KMDK00bYKGsML1Yi7nqv61_k9GBnQ-VOzc" stuttered, but now that we have -r it's not as bad: "age -r age14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa".

At that point we might as well make github: recipients just "@FiloSottile"?

I never liked alias: recipients so let's just drop them for now.

Since it's not used in user-facing encoding anymore, we can also switch the Base64 alphabet in the header to the standard one, just to reduce the weird.

For (2) it's with a heavy heart that I admit the idea of leaving the header as-is in the armor was clever. And you know what it means when I call something clever: it's going away. Now we should decide if we just sell our soul to PEM (maybe with a Tool: header for the URL?), if we pick the saltpack thing (ugh, 3 different encoding bases in a spec?), or if we just make something up.

The part I don't like about making the armor better is that I don't want people to actually use this for email, but there are non 8-bit safe environments, and if we're going to have an armor making it annoying just feels user-hostile. So PEM I guess? Or can v1 get away without an armored format?

[Tony Arcieri]

I've been working on generalized bech32 formats for cryptographic keys / signatures / digests etc here:

https://github.com/cryptouri

The main content is in:

https://github.com/cryptouri/cryptouri-spec/issues

https://github.com/cryptouri/cryptouri.rs

(I've actually had a version of this deployed in production for over a year)

The main drawback of bech32 for private key storage is I'm unaware of any constant time encoders/decoders.

It seems possible with a combination of a constant time base32 implementation and bitslicing for the bech32 lookup table, but I haven't seen it implemented yet.

--
You received this message because you are subscribed to the Google Groups "age-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to age-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/age-dev/59580227-ba8b-4125-a928-79c843c6c47c%40www.fastmail.com.

--

Tony Arcieri

[Jack Grigg]

On Thu, Dec 19, 2019 at 12:29 AM Filippo Valsorda <fil...@ml.filippo.io> wrote:

It's time to pick the bikeshed's encoding. Here are two things I'd like to fix.

1. Using _ in keys is not markup safe, both in some Markdown parsers and in simpler markups.
   
2. It's very easy to copy-paste the armor wrong, I forgot the first line myself a couple times.
   

For (1) I think I like switching the keys to Bech32. It selects as a single word, it doesn't have symbols, and it's case insensitive so it can be read out loud. (I might read an age key on stage at RWC in a lighting talk as a tribute to Ian Goldberg.)

# created: 2019-12-19T00:02:55-05:00

# public key: age14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa

AGE-SECRET-KEY-1FWY45RKECMS2XN3R9W43K9QX07NJ5QKN4KPKQXZ0053HEHALY4SQT8020E

I initially used pubkey: instead of age: because "age age:81gSRoyO7KMDK00bYKGsML1Yi7nqv61_k9GBnQ-VOzc" stuttered, but now that we have -r it's not as bad: "age -r age14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa".

I like this, particularly the single-word selection (the lack of this in the initial format has annoyed me during development).

For the native YubiKey / PIV support, we could use "age1piv" as the HRP, which would give visually-subclassed pubkeys like:

age1piv14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa

My main concern is whether we need to address the recent Bech32 malleability issue. I think not, as long as all secret and public keys are fixed-length; is this likely to always be the case?

At that point we might as well make github: recipients just "@FiloSottile"?

This decouples them from GitHub, which might be nice (implementations could query several different username-based services to find keys) at the cost of potential fragmentation and non-obviousness.

 

I never liked alias: recipients so let's just drop them for now.

I do like the idea of a local address book of sorts, but for now I'll put my alias: implementation behind an "unstable" feature flag, and we can discuss it more in another thread.

Since it's not used in user-facing encoding anymore, we can also switch the Base64 alphabet in the header to the standard one, just to reduce the weird.

👍

For (2) it's with a heavy heart that I admit the idea of leaving the header as-is in the armor was clever. And you know what it means when I call something clever: it's going away. Now we should decide if we just sell our soul to PEM (maybe with a Tool: header for the URL?), if we pick the saltpack thing (ugh, 3 different encoding bases in a spec?), or if we just make something up.

If we think about what we want to get out of an armored format, we need four things:

1. A beginning marker.
2. Some way to represent the header data: recipient lines and MAC.
3. An encoding and layout for the data.
4. An ending marker.

Comparing the options:

• The current age armored format:
1. A marker that is slightly modified from the normal marker.
2. Reuses the normal header and its MAC.
3. 56-columned Base64-encoded data.
4. An ending marker that is distinct from 1.
• PEM:
1. Hyphen-enclosed text marker.
   
2. Has an email-like text header.
3. Columned Base64-encoded data, that is internally DER-encoded.
4. Symmetric with 1.
• We could use the text header for recipient lines, but:
• We'd need to carefully examine the specs to know how we can compatibly use it.
• We'd need to define how the header is MACed.
• Alternatively, we take a native age message, and treat the entire thing as a binary payload for PEM.
• Con: The email-like PEM headers are unauthenticated and untrusted
• Pro: We don't have to think about how we would authenticate the headers.
• Con: Recipients aren't as visible as in the native format.
• saltpack:
1. Period-suffixed text marker.
2. No unencoded header, and specifies its own internal packet system for the data, which would have to be repurposed or replaced.
3. 15-character-chunked base64-encoded data.
   
4. Symmetric with 1.

I don't think the saltpack format suits our use-case. We'd need to modify it enough that we'd basically just have our own format that looks like saltpack on the outside.

Is there maybe a well-defined subset of PEM we could use? My guess is no, given how loosely PEM is followed. I also really don't want to use DER any more that we already have to. But maybe a single OctetString wrapping the age data is "fine"?

If we defined our own encoding, and the main complaint is that we re-used the normal header, then what about simply making the markers symmetric (like PEM, and more visually distinct from the normal age format), and leaving the rest as-is? I do like the "This is a file" aspect of the current marker, which nudges users away from using it in-line.

 

The part I don't like about making the armor better is that I don't want people to actually use this for email, but there are non 8-bit safe environments, and if we're going to have an armor making it annoying just feels user-hostile. So PEM I guess? Or can v1 get away without an armored format?

How prevalent and problematic are non 8-bit safe environments? Is this just older Windows CLIs, or are there newer shells that cause issues as well? I know I was having issues with pipes in PowerShell, but that was more of a wchar issue.

Cheers,

str4d

[Filippo Valsorda]

2019-12-19 07:46 GMT+01:00 Tony Arcieri <bas...@gmail.com>:

I've been working on generalized bech32 formats for cryptographic keys / signatures / digests etc here:

https://github.com/cryptouri

CryptoURI itself is more parametrized than we want to allow in age, but it's still a confirmation that Bech32 is a good format to pick.

The main drawback of bech32 for private key storage is I'm unaware of any constant time encoders/decoders.

It seems possible with a combination of a constant time base32 implementation and bitslicing for the bech32 lookup table, but I haven't seen it implemented yet.

Oh, that's a good point. I guess we can probably do without it because even an age API is probably only loading the identity from the string format once, which should not be enough for a side channel(?).

2019-12-22 19:19 GMT+01:00 Jack Grigg <m...@jackgrigg.com>:

On Thu, Dec 19, 2019 at 12:29 AM Filippo Valsorda <fil...@ml.filippo.io> wrote:

It's time to pick the bikeshed's encoding. Here are two things I'd like to fix.

1. Using _ in keys is not markup safe, both in some Markdown parsers and in simpler markups.
   
2. It's very easy to copy-paste the armor wrong, I forgot the first line myself a couple times.
   

For (1) I think I like switching the keys to Bech32. It selects as a single word, it doesn't have symbols, and it's case insensitive so it can be read out loud. (I might read an age key on stage at RWC in a lighting talk as a tribute to Ian Goldberg.)

# created: 2019-12-19T00:02:55-05:00

# public key: age14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa

AGE-SECRET-KEY-1FWY45RKECMS2XN3R9W43K9QX07NJ5QKN4KPKQXZ0053HEHALY4SQT8020E

I initially used pubkey: instead of age: because "age age:81gSRoyO7KMDK00bYKGsML1Yi7nqv61_k9GBnQ-VOzc" stuttered, but now that we have -r it's not as bad: "age -r age14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa".

I like this, particularly the single-word selection (the lack of this in the initial format has annoyed me during development).

For the native YubiKey / PIV support, we could use "age1piv" as the HRP, which would give visually-subclassed pubkeys like:

age1piv14dmj9at47tgfv8l9n499z6jreca3rurndypp582wqlwv40e8cxaqevqcsa

👍

My main concern is whether we need to address the recent Bech32 malleability issue. I think not, as long as all secret and public keys are fixed-length; is this likely to always be the case?

Yeah, I think for each HRP we'll want the payload to be fixed. We can use the HRP as the one joint in key formats.

AFAIK PEM encodes arbitrary binary data, which just happens to be DER most of the times. Or at least that's how we implemented it in Go.

We are 100% not using DER.

┏┓ 

┃┃╱╲ in this

┃╱╱╲╲  house

╱╱╭╮╲╲  we

▔▏┗┛▕▔

╱▔▔▔▔▔▔▔▔▔▔╲ 

      don't use ASN.1

╱╱┏┳┓╭╮┏┳┓ ╲╲

▔▏┗┻┛┃┃┗┻┛▕▔

For both PEM and saltpack I would not want to mess with mapping our headers onto theirs or anything like that: it would be a wrapper to deliver a single binary blob, which is the entire age file, header included. Double encoding the header is inelegant but it keeps parsers simple. I don't want humans to look at this too much, and anyway the header doesn't really tell you much, since X25519 recipients are untagged.

saltpack is really tempting because PEM is underspecified by design, but it's not base64, it's base62, which frankly might be a little much as a third base-something encoding.

If we defined our own encoding, and the main complaint is that we re-used the normal header, then what about simply making the markers symmetric (like PEM, and more visually distinct from the normal age format), and leaving the rest as-is? I do like the "This is a file" aspect of the current marker, which nudges users away from using it in-line.

Good point. So maybe we just go with PEM without headers and -----BEGIN AGE-ENCRYPTION.ORG FILE-----? That probably turns into a URL and breaks in Markdown. Ugh.

The part I don't like about making the armor better is that I don't want people to actually use this for email, but there are non 8-bit safe environments, and if we're going to have an armor making it annoying just feels user-hostile. So PEM I guess? Or can v1 get away without an armored format?

How prevalent and problematic are non 8-bit safe environments? Is this just older Windows CLIs, or are there newer shells that cause issues as well? I know I was having issues with pipes in PowerShell, but that was more of a wchar issue.

Oh, hey, wait, do I need to do something in filippo.io/age to make PowerShell work? (See https://github.com/FiloSottile/age/issues/2.)

[ciec...@gmail.com]

W dniu niedziela, 22 grudnia 2019 19:20:05 UTC+1 użytkownik Jack Grigg napisał:

On Thu, Dec 19, 2019 at 12:29 AM Filippo Valsorda <fil...@ml.filippo.io> wrote:

This decouples them from GitHub, which might be nice (implementations could query several different username-based services to find keys) at the cost of potential fragmentation and non-obviousness.

 

If services A, B, C are checked for username match, it would be easy for an attacker to impersonate B/Alice by setting up A/Alice (assuming Alice created an account only on B). It is not reasonable to require the sender, who might have good confidence in identitity of B/Alice, to check A, C and possibly more services for a name conflict and do so securely. In my opinion it is much better to be explicit about recipient namespace. Alternatively, one could always check all services rather than first matched, and use only susbet of keys which is found at all of them (possibly none, in which case refuse encryption).

(I understand this is hardly on topic for this thread, will gladly post elsewhere if the recipient specification is still subject to ongoing discussion)