Widening golang.org/x/crypto/openpgp maintainership

[Filippo Valsorda]

Hi all,

One of the observations of proposal 30141 was that the openpgp package is responsible for more than half of all golang.org/x/crypto issues and CLs, and there aren't currently enough resources dedicated to properly maintaining it.

Given that, we'd like to solicit help from the community to maintain it. In particular, it would be great if folks who made contributions to the package in the past and folks who maintain forks of it could work together to figure out a way forward. Ideally, golang.org/x/crypto/openpgp would centralize the efforts currently distributed over the various forks.

Note that the intention is not to indiscriminately add all features. The package should stay as minimal as possible while being useful (where an approximate bar for usefulness is "most heavy users don't end up with their own fork"), and where appropriate hooks should be preferred to implementing all options (like tls.Config.GetConfigForClient replaced the need for a number of other callbacks).

This mailing list is available for coordination, and we can provide tooling (like forwarding code reviews here) and access as necessary. API changes are still expected to go through the proposal process (possibly after iterating here or on the issue tracker).

Thank you to everyone for contributing to the package.

Cheers,

Filippo for the Go team

[Axel Wagner]

Hi,

I am interested in contributing and helping [sic] to maintain - though I don't think I am necessarily the most reliable person for the job, given that it'd just be a hobby and my disorder makes it hard for me to maintain interests consistently. There's also the less specific issue that I don't really have a lot of formal education in cryptography and am probably not up to date on all the best practices.

That being said, I see a little bit of a hen-and-egg problem here with anyone willing and able to put in the work: I assume potential new maintainers need to show a little bit of a track record in their contributions to the package - and without consistent current maintainership, it's difficult to build that track record. What are your plans/ideas in that regard?

Do we know if the owners of existing forks are already on this list? I think it might be a good start to talk about what's needed to re-merge those forks with upstream and whether they would be able to go on from there. I perused the search results for openpgp on godoc.org to take stock. Many of them are unmaintained forks that have diverged a significant time ago. My completely subjective list of candidates for "serious" forks would be

* Fluidkeys has a fork that isn't diverged very much (it forked off in September of last year).

* Ben Burkert has a fork that isn't maintained, but it at least contains a pretty clearly defined set of changes - at least some of which have active feature requests in the upstream repo, as far as I can tell. Probably not easily merged back, but maybe he would be interested in contributing them upstream.

* Keybase maintains a fork that has diverged significantly, but is very actively maintained. It probably can't be easily re-integrated, but it seems that Keybase has at least a vested interest in maintaining a healthy package.

* Similar with ScriptRock (UpGuard?)

There are also two forks that are seeing recent-ish work, but it's hard to evaluate on a glance how much they diverged, because they aren't clear-cut forks (but vendorings, pretty much): sesh and Psiphon.

Again, the list is purely subjective, apologies if I missed anything :)

Best,

Axel

--
You received this message because you are subscribed to the Google Groups "golang-openpgp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-openpg...@googlegroups.com.
To post to this group, send email to golang-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-openpgp/CA%2B2K_KoNvR1XL4gJ%3DigJyFeNKwkRDPzJTyZzB%3DRBBnza-quZOQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Filippo Valsorda]

On Saturday, March 16, 2019 at 6:34:44 PM UTC-4, Axel Wagner wrote:

That being said, I see a little bit of a hen-and-egg problem here with anyone willing and able to put in the work: I assume potential new maintainers need to show a little bit of a track record in their contributions to the package - and without consistent current maintainership, it's difficult to build that track record. What are your plans/ideas in that regard?

Yeah, we definitely don't want to cause a deadlock. As long as someone is willing to take up maintainership, there are many things we can look at besides past work on golang.org/x/crypto/openpgp: work done on other openpgp forks, work done elsewhere in the Go project, work done elsewhere in the Go community. If someone already has +2 access, for example, they totally can start code reviewing openpgp patches, ideally starting from ones without API implications and coordinating here.

Formal cryptography education is not at all a requirement, nor is being up to date with the latest papers—OpenPGP is from the 90ies after all!

 

Do we know if the owners of existing forks are already on this list?

I have not systematically reached out. If you or anyone wants to loop them in before I find time to, it would definitely be useful.

Thank you,

Filippo 

[alex]

Being a gpg user in general and the openpgp package, I wouldn't mind contributing with reviews and code to x/crypto/openpgp or whatever the import path it'll end up being.

I often contribute to x/crypto/acme packages as dd...@google.com and have +2 access.

--

You received this message because you are subscribed to the Google Groups "golang-openpgp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-openpg...@googlegroups.com.
To post to this group, send email to golang-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-openpgp/6fb7146b-6daa-4c95-9e5a-573e2deafd8a%40googlegroups.com.

[Filippo Valsorda]

On Tuesday, April 2, 2019 at 4:48:26 PM UTC-4, Alex Vaghin wrote:

Being a gpg user in general and the openpgp package, I wouldn't mind contributing with reviews and code to x/crypto/openpgp or whatever the import path it'll end up being.

I often contribute to x/crypto/acme packages as dd...@google.com and have +2 access.

Please feel free to start reviewing and submitting any changes without API impact. For changes to the API, please work with the requester and/or this list to figure out a proposal with a focus on sustainability and misuse resistance, and then ping me on its GitHub issue, for now.

If you wish, you can also add yourself to the owners of the package so you'll get notified on new issues and CLs. https://github.com/golang/build/blob/master/devapp/owners/table.go

I would still like for someone to try and reach out to the maintainers of the forks (cc'ing this list). x/crypto/openpgp would be most useful if it becomes the place where all the previously uncoordinated efforts converge.

Thank you,

Filippo

 

On Mon, 1 Apr 2019 at 20:34, Filippo Valsorda <fil...@golang.org> wrote:

On Saturday, March 16, 2019 at 6:34:44 PM UTC-4, Axel Wagner wrote:

That being said, I see a little bit of a hen-and-egg problem here with anyone willing and able to put in the work: I assume potential new maintainers need to show a little bit of a track record in their contributions to the package - and without consistent current maintainership, it's difficult to build that track record. What are your plans/ideas in that regard?

Yeah, we definitely don't want to cause a deadlock. As long as someone is willing to take up maintainership, there are many things we can look at besides past work on golang.org/x/crypto/openpgp: work done on other openpgp forks, work done elsewhere in the Go project, work done elsewhere in the Go community. If someone already has +2 access, for example, they totally can start code reviewing openpgp patches, ideally starting from ones without API implications and coordinating here.

Formal cryptography education is not at all a requirement, nor is being up to date with the latest papers—OpenPGP is from the 90ies after all!

 

Do we know if the owners of existing forks are already on this list?

I have not systematically reached out. If you or anyone wants to loop them in before I find time to, it would definitely be useful.

Thank you,

Filippo 

--
You received this message because you are subscribed to the Google Groups "golang-openpgp" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-openpgp+unsubscribe@googlegroups.com.
To post to this group, send email to golang-openpgp@googlegroups.com.

[Simon Ser]

On Wednesday, April 17, 2019 1:25 AM, Filippo Valsorda <fil...@golang.org> wrote:

On Tuesday, April 2, 2019 at 4:48:26 PM UTC-4, Alex Vaghin wrote:

Being a gpg user in general and the openpgp package, I wouldn't mind contributing with reviews and code to x/crypto/openpgp or whatever the import path it'll end up being.

I often contribute to x/crypto/acme packages as dd...@google.com and have +2 access.

Please feel free to start reviewing and submitting any changes without API impact. For changes to the API, please work with the requester and/or this list to figure out a proposal with a focus on sustainability and misuse resistance, and then ping me on its GitHub issue, for now.

If you wish, you can also add yourself to the owners of the package so you'll get notified on new issues and CLs. https://github.com/golang/build/blob/master/devapp/owners/table.go

I would still like for someone to try and reach out to the maintainers of the forks (cc'ing this list). x/crypto/openpgp would be most useful if it becomes the place where all the previously uncoordinated efforts converge.

Hi,

Just like Axel and Alex, I'm no cryptanalyst, but I use the openpgp package and I'd be interested in helping out maintaining it. I have experience maintaining Go libraries (e.g. the go-imap package).

I'll try reaching out the forks' owners (if nobody beats me to it). To the list of forks above, I'd also add the ProtonMail fork which shouldn't have diverged too much.

Thanks,

---

Simon Ser

https://emersion.fr

Thank you,

Filippo

 

On Mon, 1 Apr 2019 at 20:34, Filippo Valsorda <fil...@golang.org> wrote:

On Saturday, March 16, 2019 at 6:34:44 PM UTC-4, Axel Wagner wrote:

That being said, I see a little bit of a hen-and-egg problem here with anyone willing and able to put in the work: I assume potential new maintainers need to show a little bit of a track record in their contributions to the package - and without consistent current maintainership, it's difficult to build that track record. What are your plans/ideas in that regard?

Yeah, we definitely don't want to cause a deadlock. As long as someone is willing to take up maintainership, there are many things we can look at besides past work on golang.org/x/crypto/openpgp: work done on other openpgp forks, work done elsewhere in the Go project, work done elsewhere in the Go community. If someone already has +2 access, for example, they totally can start code reviewing openpgp patches, ideally starting from ones without API implications and coordinating here.

Formal cryptography education is not at all a requirement, nor is being up to date with the latest papers—OpenPGP is from the 90ies after all!

 

Do we know if the owners of existing forks are already on this list?

I have not systematically reached out. If you or anyone wants to loop them in before I find time to, it would definitely be useful.

Thank you,

Filippo 

--

You received this message because you are subscribed to the Google Groups "golang-openpgp" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-openpgp+unsubscribe@googlegroups.com.

To post to this group, send email to golang-openpgp@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-openpgp/6fb7146b-6daa-4c95-9e5a-573e2deafd8a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "golang-openpgp" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-openpg...@googlegroups.com.

To post to this group, send email to golang-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-openpgp/d64f10fd-eea6-49d1-b445-d865242e9f15%40googlegroups.com.

[Simon Ser]

Hi ProtonMail/crypto maintainers,

I'm writing on behalf of the golang-openpgp mailing list. golang/org/x/crypto/openpgp is currently searching for new maintainers (see below).

Since you maintain a fork of the library, would you be interested in helping out with maintenance and upstreaming your changes? It seems your fork has not diverged a lot, mainly adding ECC keys support if I understand correctly? There are a handful of forks, it would be nice if efforts to improve the library could be centralized.

Thanks,

--

Simon Ser

https://emersion.fr

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

--

You received this message because you are subscribed to the Google Groups "golang-openpgp" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-openpg...@googlegroups.com.

To post to this group, send email to golang-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-openpgp/CA%2B2K_KoNvR1XL4gJ%3DigJyFeNKwkRDPzJTyZzB%3DRBBnza-quZOQ%40mail.gmail.com.

[Daniel Huigens]

Hi Simon, Filippo and everyone else in golang-crypto,

Thanks for reaching out. We (ProtonMail) are potentially interested in helping maintain x/crypto/openpgp – we agree it'd be good to centralize efforts, and we're obviously committed to keeping a maintained and secure package, in fact we've recently had a security audit done on it by SEC Consult (as I believe you've seen, Filippo).

We have some questions about the process to get there, though. We don't have the bandwidth to maintain both mainline and our fork simultaneously. We'd be happy to make PRs for our changes so far, molding any API changes, code style, and commit messages to fit mainline's standards (since our fork has so far mostly been for our own use, we haven't really made an effort for that so far). However, to be very frank, the effort of doing so would not be worth it for us if, after discussion on each of them, 75% get merged and we still end up having to maintain a fork. I'd also understand though if you don't want to hand out blanket access right away – in that case perhaps we could start with having a discussion here about both the specific changes we've made so far and the general direction of changes in the future, to make sure we're on the same page before we dive into a merge? If you'd like I could summarize and talk a bit more about those here to kick that off. Alternatively or additionally, if you have any other ideas about what the process would look like, we'd also love to hear about those.

Best wishes,

Daniel Huigens

ProtonMail

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

Op zondag 28 april 2019 12:36, Simon Ser <con...@emersion.fr> schreef:

[Simon Ser]

Hi,

Thanks for reaching out. We (ProtonMail) are potentially interested in helping maintain x/crypto/openpgp – we agree it'd be good to centralize efforts, and we're obviously committed to keeping a maintained and secure package, in fact we've recently had a security audit done on it by SEC Consult (as I believe you've seen, Filippo).

Excellent!

We have some questions about the process to get there, though. We don't have the bandwidth to maintain both mainline and our fork simultaneously. We'd be happy to make PRs for our changes so far, molding any API changes, code style, and commit messages to fit mainline's standards (since our fork has so far mostly been for our own use, we haven't really made an effort for that so far). However, to be very frank, the effort of doing so would not be worth it for us if, after discussion on each of them, 75% get merged and we still end up having to maintain a fork. I'd also understand though if you don't want to hand out blanket access right away – in that case perhaps we could start with having a discussion here about both the specific changes we've made so far and the general direction of changes in the future, to make sure we're on the same page before we dive into a merge? If you'd like I could summarize and talk a bit more about those here to kick that off. Alternatively or additionally, if you have any other ideas about what the process would look like, we'd also love to hear about those.

This is perfectly understandable. It'd be very helpful if you could summarize the changes you've made and want to make.

(For the record, I think maintaining a fork with 50 patches is way trickier than maintaining a fork with just a few of them -- merging changes from upstream is necessary to fix potential security issues and becomes complicated pretty fast. So I'd still argue upstreaming most of the changes would be a win. However (1) I'd still understand you wouldn't want to upstream your changes and (2) let's discuss about the changes in the fork before we go down this road!)

Br,

[Daniel Huigens]

Hi!

It'd be very helpful if you could summarize the changes you've made and want to make.

Alrighty :) Here's a list of things we've done so far:

- Expand ECC support, as you mentioned

- Bump default and minimum S2K parameters

- Default to AES128 and SHA256 instead of CAST5 and RIPEMD160 for keys without preferred algorithms, as required by rfc4880bis-06 (rfc4880 required 3DES and SHA1)

- Disallow generating RSA keys of less than 1024 bits (recommended by the audit)

- Fix and move Signature.KeyExpired to PublicKey.KeyExpired; add Signature.SigExpired (https://github.com/golang/go/issues/22312)

- Detect expired signatures; though leaving it up to the caller whether they want to reject them

- Use latest-created valid self-signature instead of the last-appearing one; don't drop other self-signatures when re-serializing the key

- Add higher-level function to verify clearsigned messages, in order to be able to verify that the algorithm mentioned in the header matches the algorithm used in the signature (one of the findings of the audit)

Let me know if you have any questions or concerns about these :)

The last one allows me to segue into future changes/more generally maintainership philosophy. Filippo, you said that

The package should stay as minimal as possible while being useful

We agree that minimizing API surface = minimizing bug count = good, but on the other hand, having too low-level APIs, and forcing applications to fill in the details, can itself lead to security vulnerabilities. The clearsigned messages thing is not a perfect example of this, but an example nonetheless. Where does your opinion fall on adding APIs like that?

Best regards, and have a good weekend,

Daniel Huigens

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

Op zondag 5 mei 2019 19:04, Simon Ser <con...@emersion.fr> schreef:

[Filippo Valsorda]

From: 'Daniel Huigens' via golang-openpgp <golang-...@googlegroups.com>

Date: Fri, May 10, 2019 at 9:26 AM
To: Simon Ser
Cc: sanjan...@gmail.com, golang-...@googlegroups.com

- Add higher-level function to verify clearsigned messages, in order to be able to verify that the algorithm mentioned in the header matches the algorithm used in the signature (one of the findings of the audit)

Let me know if you have any questions or concerns about these :)

The last one allows me to segue into future changes/more generally maintainership philosophy. Filippo, you said that

The package should stay as minimal as possible while being useful

We agree that minimizing API surface = minimizing bug count = good, but on the other hand, having too low-level APIs, and forcing applications to fill in the details, can itself lead to security vulnerabilities. The clearsigned messages thing is not a perfect example of this, but an example nonetheless. Where does your opinion fall on adding APIs like that?

Agreed that there is a tradeoff there. In general, the most common the operation is, the more high-level the API for it should be.

Specifically around clearsigned messages, I disagree that the Hash header serves any security purpose (or any purpose at all in our implementation, really), so I don't see validating it something worth influencing API design over.

I elaborated at https://github.com/golang/crypto/commit/c05e17bb3b2dca130fc919668a96b4bec9eb9442

[Simon Ser]

On Friday, May 10, 2019 4:26 PM, Daniel Huigens <d.hu...@protonmail.com> wrote:

Hi!

It'd be very helpful if you could summarize the changes you've made and want to make.

Alrighty :) Here's a list of things we've done so far:

- Expand ECC support, as you mentioned

Yeah

- Bump default and minimum S2K parameters

- Default to AES128 and SHA256 instead of CAST5 and RIPEMD160 for keys without preferred algorithms, as required by rfc4880bis-06 (rfc4880 required 3DES and SHA1)

- Disallow generating RSA keys of less than 1024 bits (recommended by the audit)

I think all three should be good additions.

- Fix and move Signature.KeyExpired to PublicKey.KeyExpired; add Signature.SigExpired (https://github.com/golang/go/issues/22312)

Did you add anything about revocation signatures?

- Detect expired signatures; though leaving it up to the caller whether they want to reject them

What is the motivation behind "leaving it up to the caller whether they want to reject them"? Do you collect expired signatures under a different list?

- Use latest-created valid self-signature instead of the last-appearing one; don't drop other self-signatures when re-serializing the key

Makes sense.

- Add higher-level function to verify clearsigned messages, in order to be able to verify that the algorithm mentioned in the header matches the algorithm used in the signature (one of the findings of the audit)

Can you elaborate on this one?

Let me know if you have any questions or concerns about these :)

Thanks!

[Daniel Huigens]

Hi Filippo and Simon!

Thanks for your responses.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Monday May 13 2019 19:14, Filippo Valsorda <fil...@golang.org> wrote:

Agreed that there is a tradeoff there. In general, the most common the operation is, the more high-level the API for it should be.

Specifically around clearsigned messages, I disagree that the Hash header serves any security purpose (or any purpose at all in our implementation, really), so I don't see validating it something worth influencing API design over.

I elaborated at https://github.com/golang/crypto/commit/c05e17bb3b2dca130fc919668a96b4bec9eb9442

I agree, it's not a very good example of the general point.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Wednesday May 15 2019 18:03, Simon Ser <con...@emersion.fr> wrote:

Did you add anything about revocation signatures?

No, not to my knowledge. Are you referring to something specific?

What is the motivation behind "leaving it up to the caller whether they want to reject them"?

I believe the motivation was that signatures are also considered "expired" if they are ostensibly created in the future, which can happen if the local clock is skewed. But since this can also be addressed using a `config` parameter (with `config.Time()`), which we've already added to `CheckDetachedSignature` and `signatureCheckReader.Read` (`ReadMessage` already has it), I'd also personally be fine with always rejecting them outright. In the future, I imagine we'd also like to add a `config` param to `ReadEntity` and `pk.Verify*Signature`, and add support for short-lived subkeys and signatures that automatically roll over (i.e. not use subkeys and sigs deliberately "created" in the future). There's some argument to be had, though, whether the signatures and keys that are used should be selected at key-read-time or key-usage-time. Pedantically speaking, the current time could change between the two, but the latter would be a rather large reorganization.

- Add higher-level function to verify clearsigned messages, in order to be able to verify that the algorithm mentioned in the header matches the algorithm used in the signature (one of the findings of the audit)

Can you elaborate on this one?

See the link from Filippo quoted at the top of this email, it explains the issue well. We then added `clearsign.Block.VerifySignature`:

https://github.com/ProtonMail/crypto/blob/efb430e751f2f00d8d9aedb254fc14ef76954880/openpgp/clearsign/clearsign.go#L401-L418

Best wishes,

Daniel Huigens

[Shashank Yadav]

Hi Daniel and Simon,

I have a long time pending review which adds new APIs but does not break existing ones (github tracking issue link). The review contains changes around subkeys and revocation signatures with tests. To be specific:

1. AddSubkey adds support for generating new subkeys associated with an Entity. It also adds support for adding signing subkeys.

2. RevokeKey generates a 0x20 key revocation signature against the entity.

3. RevokeSubKey generates a 0x28 subkey revocation signature for a subkey.

4. Add Revocation reason subpacket and EmbeddedSignature subpacket to output subpackets with corresponding tests.

5. Re-sign the embedded signatures for subkeys in entity.SerializePrivate()

I was hoping to merge these changes upstream instead of maintaining a separate fork as well. It would be great to get opinions of other developers working with crypto/openpgp and also understand what can do to get these changes merged.

Regards,

Shashank Yadav

[Daniel Huigens]

Hi Filippo and others,

Sorry for the delay in starting to send CLs to upstream. I have, in an effort to start helping with maintaining this library, reviewed https://go-review.googlesource.com/c/crypto/+/191658, and Shashank has addressed all comments to the point where I'd be happy to merge it in our fork. Filippo, would you like to look it over and merge it here, first? The API should be fully backwards-compatible, now.

After that, Shashank, I'll review the other CL you linked - I've held off on that for now since it's a bit bigger.

And then finally I hope to start sending CLs with our own changes. In particular, I wanted to discuss one large feature we've been working on, which is support for the new AEAD Encrypted Data Packet defined in rfc4880bis. The main advantage of this packet is that for large messages, you can securely stream-decrypt them while maintaining integrity protection (thus protecting against stuff like EFAIL).

This is not only a rather large feature, but also depends on support for two new AES modes - EAX and OCB. In our fork, we've opted to put these outside the `openpgp/` namespace, since they might be more generally useful - but that makes them perhaps out of scope for this mailing list? :)

These have been code reviewed here:

https://github.com/ProtonMail/crypto/pull/26 (EAX and OCB)

https://github.com/ProtonMail/crypto/pull/36 (AEAD packet - not merged yet, but pretty close to done)

So I suppose my questions are:

- Would there be interest in merging this here?

- If so, do you agree that EAX and OCB should be outside `openpgp/`, or do you think it would be easier to merge this here if they are inside it?

- If the former, who/where should I ask about interest for merging those?

Thanks in advance!

Best regards,

Daniel

Op dinsdag 6 augustus 2019 12:03:22 UTC+2 schreef Shashank Yadav:

[Shashank Yadav]

Hi Filippo,

Any updates for Daniel's post?

https://go-review.googlesource.com/c/crypto/+/191658 is one of the recent changes that is being pushed upstream and was reviewed and +1'ed by Daniel who's a maintainer for protonmail's crypto package. However there is no clear path on how to actually get the changes into the package once the changes are reviewed. We would need someone with +2 permissions to move forward. It would be great to be able to get some clarity regarding the same e.g. who to contact once changes are +1'ed.

Regards,

Shashank Yadav