State of the Puck

[Casey Marshall]

Several exciting developments recently with Hockeypuck. Release 1.0 is getting closer, but there is still much to be done!

SKS reconciliation is now working!

That's right, folks. With latest Hockeypuck & conflux trunk I've successfully synchronized keys with SKS 1.1.3 (and other Hockeypucks) in the small. The conflux implementation of the protocol and reconciliation algorithm works: I've gotten SKS to discover Hockeypuck keys and vice-versa. But, I haven't scaled up testing yet, because...

Conflux needs a better backend database!

The backend stores for conflux are all broken in their own special way. I need a rock-solid prefix-tree database. MongoDB failed pretty bad at a performant ptree. I could probably make it work, but it's a poor fit. I tried leveldb, it's fast, but it's also a poor fit (everything is a binary blob). Plus, I don't like depending on a C library that isn't readily available on some distros. I also don't like two different types of data store -- Hockeypuck and Conflux should be able to share a database.

If you want to try Hockeypuck's recon capabilities, you'll hit a problem because I think my leveldb ptree store doesn't shutdown properly -- sample points get out of sync, and pretty soon you get strange, broken behavior -- re-requesting the same keys over and over, etc.

I've decided that Hockeypuck 1.0 and on will be PostgreSQL-only. I would rather be opinionated on a reliable platform, and spend my time building out services. A PostgreSQL backend for Hockeypuck and prefix tree store for Conflux are now my top priorities. I'm trying to really capture the structure of RFC 4880 in my schema, balancing with database performance and application design. I'm also leaving room for tackling some unsolved OpenPGP issues, such as keyserver clutter.

Beyond that, I'll need ways to deal with the gaps between Hockeypuck and SKS. For example, there are some keys Hockeypuck can't load -- go.crypto doesn't support older packet versions (and some newer ones like ECC). Until it does, I'll need to keep track of these in such a way that I can still reconcile them with SKS. I may also need to support some sort of throttling when reconciling with SKS, so that Hockeypuck doesn't flood these peers.

Finally, if I can fit it in 1.0, I'm going to restructure Hockeypuck to leave room for serving and synchronizing keys from other PKIs beyond 1.0 - SSH, X509, custom apps, who knows?

Here we go,

Casey

[Casey Marshall]

Hockeypuck 1.0 is now in beta testing, latest build in Launchpad is 1.0~b1~bzr187 (https://launchpad.net/~hockeypuck/+archive/unstable/+packages). Reconciliation works in the small, but there are still some issues to work out before Hockeypuck will be suitable for peering.

#1208290

Need to track unsupported keys for SKS reconciliation

The SKS reconciliation protocol is working, but there is still a problem with Hockeypuck not supporting all of the same packets/keys that SKS does. Until this is handled properly, gossip will re-transmit a lot of keys over and over.

#1195901

UID ordering and preference is suboptimal

Still need to investigate this one, make sure the primary_uid reference is set properly. The new key validation post-refactoring is much better suited to handle rules like this.

#1208295

Support PostgreSQL bulk loading techniques

This would be a nice-to-have. Loading a full SKS dump slows to a crawl with all the indexes turned on.

-Casey