ASCII armored format

[Filippo Valsorda]

The ASCII armor format is now in the spec and implemented in
github.com/FiloSottile/age (command line flag -a).

https://docs.google.com/document/d/11yHom20CrsuX8KQJXBBw04s80Unjv8zCg_A7sPAX_9Y/view#heading=h.qjfi2qewlqec

Here is an example message encrypted to the key

AGE_SECRET_KEY_l6sUOU2sZgLkyIkDHT_Q7wNqUjeeyUe4pJCsyt_-qII

This is an armored file encrypted with age-tool.com, version 1
-> X25519 DeasSsF-PM4SYVmy0qhVEU3ocF3MBQCmDQaDPeensF0
sX7zpwhdOf2jdJcuoe6ErYHlwi0Pf2dustQ0Jl9Fisc
--- 4zCOMvM4Bjs77obKwWzaUj2f-hM3IxrbZaA6Ayml8wY
4yDhLT-fldYWzinLGYK13Tqr35lUhGCOTKgHwTdioRW4E3UTPJ5kgMSF
mZtqWUUHTMHSFOok5TH1vRrlpxPbOU-A
--- end of file ---

Note that this uses a different end of file marker than the initial
draft implemented by rage (sorry Str4d!).

[Jack Grigg]

On Mon, Nov 25, 2019 at 3:30 AM Filippo Valsorda <fil...@ml.filippo.io> wrote:

The ASCII armor format is now in the spec and implemented in
github.com/FiloSottile/age (command line flag -a).

Same flat as I'm using for rage 👍

 

https://docs.google.com/document/d/11yHom20CrsuX8KQJXBBw04s80Unjv8zCg_A7sPAX_9Y/view#heading=h.qjfi2qewlqec

Here is an example message encrypted to the key

AGE_SECRET_KEY_l6sUOU2sZgLkyIkDHT_Q7wNqUjeeyUe4pJCsyt_-qII

This is an armored file encrypted with age-tool.com, version 1
-> X25519 DeasSsF-PM4SYVmy0qhVEU3ocF3MBQCmDQaDPeensF0
sX7zpwhdOf2jdJcuoe6ErYHlwi0Pf2dustQ0Jl9Fisc
--- 4zCOMvM4Bjs77obKwWzaUj2f-hM3IxrbZaA6Ayml8wY
4yDhLT-fldYWzinLGYK13Tqr35lUhGCOTKgHwTdioRW4E3UTPJ5kgMSF
mZtqWUUHTMHSFOok5TH1vRrlpxPbOU-A
--- end of file ---

Note that this uses a different end of file marker than the initial
draft implemented by rage (sorry Str4d!).

Heh, no worries. Gives me a chance to show off my interoperability tests! Which incidentally show that my CRLF handling is completely broken 😅

https://github.com/str4d/rage/runs/319854543

 

--
You received this message because you are subscribed to the Google Groups "age-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to age-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/age-dev/ab7a1151-14fb-4e05-85db-25f4f5d849e8%40www.fastmail.com.

[aaron...@gmail.com]

On Monday, November 25, 2019 at 3:30:06 AM UTC, Filippo Valsorda wrote:

The ASCII armor format is now in the spec and implemented in
github.com/FiloSottile/age (command line flag -a).

(copied from github)

Hey, maybe this is off-topic or too late, but is there a reason the spec hard-wraps at 56 columns instead of allowing the file/data viewer to soft-wrap?

The salt pack format simply uses spaces every 15 characters, which wraps much more nicely on various screen sizes, as shown on their website: https://saltpack.org/

Thanks for your work, and I look forward to using age!

[Filippo Valsorda]

Thank you for following up here. I don't really have a strong opinion on this.

age files already had a textual header, so it felt natural to encode the payload and leave the header alone, instead of double-encoding the header. Using the saltpack armor for just the payload would look kind of awkward.

This is an armored file encrypted with age-tool.com, version 1

-> X25519 jxogGETLAZtc70T35gEa7-TyTsqRKjoh4w-URI2tHC4

FflFC6gUrydahfQj9KmVI8eznYR7XNm5Ra_19CvbvyY

--- i7xr4pMS8q4zCzK9-qO4vk6ENoCw-kwcSMYwnJCD6NY

kYM5h1pg6qz9UMn j6G7KB2OUX3itkn C30ZZBPzjc8STxf i06TIQhJ4wRFSMU hFa9gmcvl2AW8Kk qmTdLkkppPieOWq o9aWouAaMpQ9kWt eMlv17NOUUr9Gp3 fClo7khRnJ12T7j 6ZVkfDXUpznTp57 0btBywDV848jyp2 EceloYGiuOolWim 8HCx77p22iulWja ixShPFcOi1mkG2i 4Iur3QfGYeKpflx a1GXmvQLi1G99mH 625dH5HGcQ63pOb K1i7g3lXIQ9Kcfy NRDfdBIDMHJaJf1 uTKB4GJ9l4M7glS 07h9QsU4gPueyNC hzm6LmA9CFllzxy 8ZA0Ys5qDnSuwaN obowMNXpbm1nlsx fXFtMolx6ghLuEw 2s8f1jBxBQjQPwa GG90h5BbpoWGPk6 dRsou5kdNxcLaFJ KKXWTUR2h9P0P7p 9UYRsQ6QqGNiwmG wXC7YFh1xCUdAib gjZbUYUKN6KVLem hZI6XYtX2l1w8d5 jL8KJ5ZZpKhJ4JC faVWCU2VRtUFgQO ejKm6wjs6NcekTd KK4bOh5kr87cyRu 0aDjEtfMSyZZTG5 hIrEWcMq1Iotzrx iRdmY5GYf2Kx0Br 4K0rqrj8ZGa.

I do like the Markdown resistance of the saltpack armor, but then to get its benefits we'd have to change the encoding of everything that currently uses Base64. Also, every language has a Base64 implementation, while this is a custom encoding. (Maybe we should use a more innocuous pair of special characters than "_" and "-"? Maybe "!" and "?" or the standard "+" and "/"?)

Finally, keep in mind that the age armor is more meant for non 8-bit clean environments (like PowerShell, and yeah maybe online forums and stuff) than for messages, because age is a file encryption tool, not an email or message encryption tool.

Ben, Str4d, opinions?

[Aaron Janse]

On Friday, November 29, 2019 at 4:27:33 AM UTC, Filippo Valsorda wrote:

Using the saltpack armor for just the payload would look kind of awkward.

I agree. However, I don't know if this is a problem with the encoding or with the header. One potential issue with the "This is an armored file..." header is that it's not obvious that the textual header is part of the file. Would age throw an error if that textual header would be left out?

I do like the Markdown resistance of the saltpack armor

In my opinion, part of the greatest strengths of the saltpack armor is its opaqueness to humans and markdown processors.

I assume that humans are really good at copy-pasting full blobs of base62 text that begins and ends with a simple message ("BEGIN/END SALTPACK MESSAGE.").

Also, I'd like to note that the three dashes in the age armor are converted to an em-dash in some text editors.

 

but then to get its benefits we'd have to change the encoding of everything that currently uses Base64. Also, every language has a Base64 implementation, while this is a custom encoding. (Maybe we should use a more innocuous pair of special characters than "_" and "-"? Maybe "!" and "?" or the standard "+" and "/"?)

That is a very good point. However, once the base64 is decoded, wouldn't specialized software be needed in order to do the rest of the necessary processing?

If base62 (or something similar) ends up being better than base64 (I don't know; that's up to you), maybe a simple base62 decoder script could be included in the "disaster recovery" documentation.

Once again, thank you all for the hard work on this tool.

I'm sure you'll all make the right decision, and even if we disagree on the armor, age's cryptographic quality would definitely make up for any stylistic grievances :-)

[@moritzdietz]

Am Freitag, 29. November 2019 07:03:21 UTC+1 schrieb Aaron Janse:

On Friday, November 29, 2019 at 4:27:33 AM UTC, Filippo Valsorda wrote:

Using the saltpack armor for just the payload would look kind of awkward.

However, I don't know if this is a problem with the encoding or with the header. One potential issue with the "This is an armored file..." header is that it's not obvious that the textual header is part of the file. Would age throw an error if that textual header would be left out?

 

I agree with their observation. I was following the live stream on Twitch and thought to myself that the armor header looks a bit problematic in exactly that regard. Maybe I am just used to seeing the pgp armor header and it's obvious visual indicators that it is indeed part of the entire file and if removed will cause problems. Maybe there isn't a real problem as I am not a cryptographer, but since someone else noticed it too I thought of speaking up about it.

[Jack Grigg]

On Fri, 29 Nov 2019, 8:39 am '@moritzdietz' via age-dev, <age...@googlegroups.com> wrote:

Am Freitag, 29. November 2019 07:03:21 UTC+1 schrieb Aaron Janse:

On Friday, November 29, 2019 at 4:27:33 AM UTC, Filippo Valsorda wrote:

Using the saltpack armor for just the payload would look kind of awkward.

However, I don't know if this is a problem with the encoding or with the header. One potential issue with the "This is an armored file..." header is that it's not obvious that the textual header is part of the file. Would age throw an error if that textual header would be left out?

Yes; it's as much a magic string as the PEM-style "BEGIN FOO BAR FORMAT" strings are. If it's instead altered to a different age-compatible magic string (like the binary one), the header MAC verification will fail.

 

I agree with their observation. I was following the live stream on Twitch and thought to myself that the armor header looks a bit problematic in exactly that regard. Maybe I am just used to seeing the pgp armor header and it's obvious visual indicators that it is indeed part of the entire file and if removed will cause problems. Maybe there isn't a real problem as I am not a cryptographer, but since someone else noticed it too I thought of speaking up about it.

This is a UX question rather than a cryptographic one. I'll go poke some UX friends and see if any of them have thoughts on file formats 😝

One of the sub-questions I'd ask here is "What is the cost to the user of getting this wrong?". The format and cryptographic design makes it trivial to discover and learn from a mistake (per above, though we should make sure we have good error messages on parser failures), but the costs will depend on how the user is actually using age. Two contrasting examples:

- If the user encrypts an existing file without overwriting it, and sends the file without the first line magic, the recipient can alert the sender of the problem, and/or try re-adding the magic to repair the file.

- If the user encrypts an existing file overwriting it, or encrypts some ephemeral stdin, and then deletes the entire header (or sends the file without it and then deletes the original), then the plaintext is lost forever.

Ideally our user feedback (both interactively within the CLI and passively within the file format) should indicate / teach that the encrypted output is a standalone file and shouldn't be modified directly or copied around. Given that age will likely end up being a library component of other e.g. GUI applications, it would be good to have notes on the intended UX as part of the spec and library documentation as well.

[Ben Cox]

Base64 encoding is already a bit of a minefield as far a charter
choice is concerned.

My 2c for armor, would be to either:

A) Keep base64, everything stays normal, and is the least effort

B) Keep base64, but replace special characters to be more markdown and
markdown aligned proof, should be easy to fix in other languages with
a string replace or something similar.

[Leigh McCulloch]

Hi Filippo,

Here are a couple things that stood out to me when reading the spec. These might not be important, and if that's the case, please disregard! I'm very new to this and I've tried to catch up to make sure I'm not asking an existing question, but if I have, sorry!

I love that a secret key is unmistakable. It is so clear that the secret key is a secret key.

I think there's huge value in using a standard base64 implementation, and I love that age is erring on the side of making implementations easier in many languages.

The spec says the headers for the binary and armored files are:

>This is a file encrypted with age-tool.com, version 1

>This is an armored file encrypted with age-tool.com, version 1

It's unclear to me if the version number in both is the version of the "age-tool" implementation, or version 1 of the specification that can be implemented by anyone. If I was writing an implementation would I change the text to read "This is a file encrypted with <My Age Implementation>, version ?". If it's the tool version 👍, if it is the spec version is it worth making that clear?

The second header implies the file has been armored, then encrypted, because 'armored' is used as an adjective of the file being encrypted. My understanding is that it is a file that's been encrypted and the resulting payload is encoded in ASCII armor. Given that the only thing that is different in an armored file is the payload portion, maybe the indicator for the encoding of the payload should be lower down in the file just before the payload? This would also mean the header would be consistent for all age files, regardless of the payload encoding. This could also allow the payload encoding to evolve (if needed) without the header of the file changing. This would allow tools to still parse the header irrespective of the payload and without needing to handle different text headers, if there are tools that need to do this.

The prefix -> on lines indicating the beginning of a new recipient is two characters. Other lines are three dashes. Reading the message format was confusing to me initially, partly because the delimiters are so different from section to section. Is this inconsistency in delimiters valuable?

In several places of the spec a 56 line length wrap is mentioned, but the text headers break that limit getting up to 62 characters wide. Is the 56 limit important if it is broken by other components of the file? What purpose does the limit serve?

Is supporting non-armored encrypted files valuable? One annoying things I've encountered with other tools is the many ways keys and messages can be encoded. It'd be great if there was one, and only one format for an encrypted message and if age messages were always easily passed around as attachments or blobs. It might also increase portability of age files into other tools if other tools only need to consider one format.

If we're considering saltpack message format, should we also consider PGP's format? It is very easy to read and understand by eye. We could label recipients and payloads since age messages have multiple components separated in the message.

Example:

-----BEGIN AGE MESSAGE-----

Format: age-tool.com version 1

Tool: github.com/FiloSottile/age v1.0.1

Recipient:

X25519 CJM36AHmTbdHSuOQL-NESqyVQE75f2e610iRdLPEN20
C3ZAeY64NXS4QFrksLm3EGz-uPRyI0eQsWw7LWbbYig

Recipient:

X25519 ytazqsbmUnPwVWMVx0c1X9iUtGdY4yAB08UQTY2hNCI
N3pgrXkbIn_RrVt0T0G3sQr1wGWuclqKxTSWHSqGdkc

Payload:
yDgBO22WxBHv7O8X7O/jygAEzol56iUKiXmV+XmpCtmpqQUKiQrFqclFqUDBovzS
vBSFjNSiVHsuAA===njUN
-----END AGE MESSAGE-----

Leigh