Uneccesary features in the specification

[Richard Ulmer]

Hi,
I love the simplicity of age and it's UNIX-Style composability. As I
took a closer look at the specification [1], however, I found some
features that should not be implemented in age in my opinion, because
they can be resolved by applying UNIX-Style composition. Leaving
these features out will simplify the code and user interface, avoid
ambiguity (what if a recipient file was named 'age1recipients.txt'?)
and in consequence make age more secure (less features means less
exploitability).

1. Encryption to a list of recipients in a file
I think reading files with the '-r' flag should not be supported,
because this can be solved with xargs(1):
$ awk '{print "-r \"" $0 "\""}' recipients.txt | xargs -J % age % -o data.age data
If one wanted to stream data into age instead of using the 'data' file,
fifos could be utilized:
$ mkfifo tmp
$ tar cv ~/xxx > tmp &
$ awk '{print "-r \"" $0 "\""}' recipients.txt | xargs -J % age % -o xxx.tar.age tmp
$ rm tmp

2. HTTP recipient files
$ age -r https://github.com/FiloSottile.keys -o data.age data
could be replaced with (doesn't really work, because ecdsa-sha2-nistp256
keys are not yet supported):
$ wget https://github.com/FiloSottile.keys
$ awk '{print "-r \"" $0 "\""}' FiloSottile.keys | xargs -J % age % -o data.age data

3. GitHub recipients
Instead of supporting
$ age -r github:FiloSottile
I would prefer to not support anything special here at all. If users
want this, it would be easy to implement a little script for downloading
key files from GitHub. I will provide an example script at [3].

Of course the awk/xarg pipelines I suggested are tiresome to type, so I
would advice users to write scripts for that. Maybe such scripts could
be advertised in the README. I'll append two rudimentary example scripts
in [2] and [3].

I'm aware, that I'm a bit of a UNIX-philosophy extremist and I know
that not everyone likes their tools to be as minimal as I like them.
After all, most people use the feature rich GNU userland tools, instead
of the more barebones ones like OpenBSD's. I could thus understand if
my suggestions were disregarded. I just want to communicate my opinion
before these features are implemented (and thus more or less set in
stone).

Best Regards,
Richard Ulmer (aka codesoap)

[1] https://docs.google.com/document/d/11yHom20CrsuX8KQJXBBw04s80Unjv8zCg_A7sPAX_9Y/preview#
[2] Example script: ~/bin/age-multienc

[code...@mailbox.org]

My mail client messed up notes [2] and [3]. Here they are again:

[2] Example script: ~/bin/age-multienc
if [ $# -ne 3 ]
then
printf "Usage: $0 <recipients-file> <infile> <outfile>\n" \
exit 1
fi
awk '{print "-r \"" $0 "\""}' "$1" | xargs -J % age % -o "$3" "$2"

[3] Example script: ~/bin/age-githubenc
if [ $# -ne 3 ]
then
printf "Usage: $0 <github-username> <infile> <outfile>\n" \
exit 1
fi
curl -s "https://github.com/$1.keys" \
| awk '{print "-r \"" $0 "\""}' \
| xargs -J % age % -o "$3" "$2"

[Ben Cox]

While age may be designed to be as close to UNIX philosophy, it does
mimic it rather than implement it.

The rough design of age was roughly "what if we had a command like
gzip that encrypted stuff". The thinking being that people are used to
commands like gzip and how they work, but also have a semi
indistinctive way on how they think (assuming the gzip idea sticks) it
will work next.

I'm adamant that stripping away usages or options is not helping
anyone here, other than supporting a UNIX philosophy war. That's not
the design of age, it's designed to be picked up by users easily. The
lack of commands/functions does not help users pick stuff up faster,
In fact in your original post you point out that some of our commands
should go away in place of people hand writing xargs and awk! While
this is "very unix" it's also terrible for non-UNIX tamed humans in
the real world (we want everyone to use this, not just unix users).

Overall this is to say "I see your point, but that is not what we are
looking for in terms of user experience"

---

To go point by point though:

> what if a recipient file was named 'age1recipients.txt'?)
> and in consequence make age more secure (less features means less
> exploitability).

This can likely be detected and warned at, rather than dropping recipient files.

> 2. HTTP recipient files

These are still useful, There are likely some tweaks we can do to warn
people against pulling insecure key files, but having the tool itself
fetch this is useful.

> 3. GitHub recipients

This is just a convenience feature, Often people find themselves in
the use case of "I know their github, but not their public file
encryption keys", the github shortcut serves to solve this easily,
while to some degree admitting that the only git code host that really
has critical mass is GitHub, of course that may change one day, but
ideally we are focusing on making lives better now, we can always drop
or add other macros when it's deemed worth it, but giving this feature
is slightly obscure and really a edge case handler, I think it's okay
for now.

Cheers
Ben Cartwright-Cox

> --
> You received this message because you are subscribed to the Google Groups "age-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to age-dev+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/age-dev/2ZAYNE5RAEWAA.1YHUBIUSHCMNN%40t450.fritz.box.

[Richard Ulmer]

Thanks for your thorough response! I can see that you thought about this
before and I understand your decision. It is not my intention to engage
in a UNIX philosophy war, but my concerns were about maintainability and
the ability to reason about the program's behavior.

> I'm adamant that stripping away usages or options is not helping
> anyone here, other than supporting a UNIX philosophy war. That's not
> the design of age, it's designed to be picked up by users easily.

My main concerns about those convenience features are, that they
complicate the code and thus make it harder to understand or audit and
that they introduce new threats (e.g. downloading and parsing files from
the internet). I can, however, see that those features make age more
user friendly. I guess if the code is carefully written and reviewed,
they will not cause great harm.

> Overall this is to say "I see your point, but that is not what we are
> looking for in terms of user experience"

I understand and respect your decision.

> > what if a recipient file was named 'age1recipients.txt'?)
> > and in consequence make age more secure (less features means less
> > exploitability).
>
> This can likely be detected and warned at, rather than dropping recipient files.

I think such "conditional interactivity" would be inconvenient for
using age in scripts, but I see that this is a rare edge case.

Slightly off topic: I just discovered that pull request #115 [1]
addresses this issue by requiring the 'file:' prefix for files, which
I find to be good idea. What do you think about including this in the
specification?

> > 2. HTTP recipient files
>
> These are still useful, There are likely some tweaks we can do to warn
> people against pulling insecure key files, but having the tool itself
> fetch this is useful.

Sounds like a good idea to warn the user about pulling files from the
internet, but I think it should be no more than a note in the usage text
or a (non-interactive) warning, to avoid the potential need for a new
command line flag which deactivates interactivity.

> > 3. GitHub recipients
>
> This is just a convenience feature, Often people find themselves in
> the use case of "I know their github, but not their public file
> encryption keys", the github shortcut serves to solve this easily,
> while to some degree admitting that the only git code host that really
> has critical mass is GitHub, of course that may change one day, but
> ideally we are focusing on making lives better now, we can always drop
> or add other macros when it's deemed worth it, but giving this feature
> is slightly obscure and really a edge case handler, I think it's okay
> for now.

Point taken. If HTTP interactivity gets implemented, I guess
implementing this little convenience function is no big change, so it
wouldn't bother me too much.

Best Regards,
Richard Ulmer

[1] https://github.com/FiloSottile/age/pull/115

[Filippo Valsorda]

This was a very well-timed discussion, str4d and I also just finished talking about how to and whether to implement these features, because none were yet implemented in age, and most were behind unstable flag in rage. There are definitely valid concerns on both sides of the tradeoff: limiting complexity and increasing usability are in tension. Here's a suggestion we discussed with Ben and str4d which I think strikes a good balance.

Recipient files are not that easy to replace with UNIX tooling. I like awk and I think I am more familiar with it than the average engineer, but I could not confidently write the incantation to support recipient files without testing it each time. We need to think about the complexity of using age as a whole, not just of the tool itself, and if the awk scripts that people have to write around it end up buggy and unpredictable, we did not reduce complex or unexpected behavior, we just shifted it elsewhere. Recipient files are also relatively straightforward in how they can be expected to work: opening a file is a standard UNIX operation with no choices to make or parameters, and "ignore empty lines and lines starting with #" is approximately lingua franca.

However, the confusion between filenames and recipient strings is a problem. It's annoying enough to predict if age will consider opening files that also parse as recipients, and whether it depends on the file actually existing or not, and what wins in that case, but it becomes a real headache of regrets once we start thinking about what happens if we introduce new recipient formats in a new release. Do some files stop parsing as files? Nightmare fuel. The confusion can even be a security issue: if an application uses ["-r", recipient] it might not expect a user setting recipient to /etc/passwd, and there's a risk an error message might leak file contents (although we should try to avoid that).

All of those issues go away if we dedicate a separate CLI flag to recipient files, like -R/--recipients-file.

This also avoids the file:// prefix which breaks shell autocompletion, and makes it unambiguous that filenames don't nest inside recipient files, because filenames are not recipients themselves. Finally, there's plenty of precedent in UNIX tools to take a flag for a filename that contains a list of things otherwise passed on the CLI (e.g. rsync --exclude-from).

For HTTPS recipient sources (including GitHub), the balance is different though. First, fetching a file from the Internet is not straightforward, and don't want to add options for timeouts, retries, trusted roots, redirects, caching, and proxies, which users might reasonably require. Moreover, the security issue of user-provided URLs can be severe, as it can lead to SSRF (which for some reason we made into a critical attack in cloud scenarios). In terms of predictability, age making a network connection is just weird, and as Ben pointed out, messes with sandboxing.

At the same time, once we have -R, it's easy to compose other UNIX tools to achieve the same goal. For example, curl is pretty much ubiquitous, people are familiar with it, and it already comes with all of the options you might need to customize the network request.

age -R <(curl https://github.com/FiloSottile.keys)

or even

curl https://filippo.io/keys.txt | age -R - input.txt

is easy to remember and predictable.

I suggest we implement recipient files as -R/--recipients-file, and drop native support for HTTPS and GitHub recipients. I implemented this yesterday, will merge it today, and will ship it in the next release if there are no overriding objections. str4d landed a similar PR in rage. In general, I think this follows the philosophy of "make the common thing easy, and the rest easy to build on top" which worked well for Go.

Thanks everyone for joining the discussion!

-- 

You received this message because you are subscribed to the Google Groups "age-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to age-dev+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/age-dev/3HMZ8H6B8YOWH.2BCFUNKZRMUGW%40t450.fritz.box.

[Richard Ulmer]

Hi Filippo,
I really like your proposal, since using a different command line flag
for the files removes any ambiguity. I'm even more happy/relieved about
you deciding against HTTP integration.

> *Recipient files* are not that easy to replace with UNIX tooling. I like

> awk and I think I am more familiar with it than the average engineer, but I
> could not confidently write the incantation to support recipient files
> without testing it each time.

Fair point. To be honest, I don't think I could confidently write the
awk/xargs pipe without testing either.

Best Regards,
Richard Ulmer