Compact encoding for headers?

[John Schmidt]

Hi!

I have a question about the Age header format. Age files are considered binary [1], but the format is a mix of:

Text (version)
Text + Base64 (algorithm specs)
Base64 (MAC)
Binary (encrypted data)

Encoding the header in this way adds a lot of overhead to small messages: encrypting 'hello' with X25519 results in a 206 byte output, which could be reduced if the header used some sort of binary encoding.

Now, 206 bytes is not the end of the world, and indeed I am mostly asking out of curiosity. That being said:

1. What led to the choice of the current header format, as opposed to a fully binary one?
2. Is it worth discussing ways to make the header more compact, or do you have bigger fish to fry?

Best,
John

[1] (https://groups.google.com/g/age-dev/c/Gk9qHPqKtyE/m/0wpTeq6-AwAJ)

[Ben Cox]

> 1. What led to the choice of the current header format, as opposed to a fully binary one?

Being able to look at a file and easily see what it is (the header is
self describing) is critically useful for something you may have
encrypted your files with N years ago

> 2. Is it worth discussing ways to make the header more compact, or do you have bigger fish to fry?

That depends, is there a direct application for you having small
headers? If you are encrypting a lot of small files then you might get
better mileage just tar/zip-ing them.

> --
> You received this message because you are subscribed to the Google Groups "age-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to age-dev+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/age-dev/CAFhOaoWjkS8tCB%3DuYz14toNj1XsQPDYWHzuuepQ%3Dn%3DKVpar9iQ%40mail.gmail.com.

[Filippo Valsorda]

I picked text because I find it easier to work with in modern languages, and existing binary formats come with a lot of baggage.

Encryption overhead with X25519 is not going to get much lower than 32 (X25519 share) + 32 (16 of file key, 16 of Poly1305 tag) + 32 (header HMAC) + 16 (STREAM nonce) + 16 (Poly1305 for one block) = 128, plus encoding (which TLV binary formats can have higher than text), plus identifiers and headers. Sounds like we could save 40 bytes, maybe 60.

Doesn't feel worth the extra complexity.

To view this discussion on the web visit https://groups.google.com/d/msgid/age-dev/CAL%3D9YSW-vL3%3DW4wENqmVXa%3D3Dexqccqyb1qMEBk0hVTgOqfPKw%40mail.gmail.com.

[John Schmidt]

Makes sense, given that age is for files, and files are most of the time "big" compared to the header.

Thank you both for your replies and your work on age!