Re: incorrect salmon signatures

[John Panzer]

Hi all,

Sorry for the rough edges around the specification.  I want to provide some background here for those who are interested in it; James pretty much nailed the high level summary of "real close now".

The signature specification has been going through various changes, and we've deliberately not been committing them to specific version numbers or anything like that because we don't want anyone to be stuck long term with having to support stuff that was based on a draft spec (possibly, one with security bugs -- though none have been found so far, knock wood).

We have some experience with the process of getting people to rev their libraries in the OAuth world, in which pretty much everyone was able to rev their services and library code within a week or two when a session fixation attack was discovered and a solution found.  That was in a spec that had already been declared "done" by an informal working group and implemented widely, so my hope is that upgrading experimental Salmon signatures to 1.0 will be much easier than that.

That said, it is a pain to experiment with at the moment because, well, everybody is experimenting.  The signature specification bleeding edge has moved to the Magic Signatures Experimental Draft (http://salmon-protocol.googlecode.com/svn/trunk/draft-panzer-magicsig-experimental-00.html).  That draft has gone through a few rounds of feedback from implementors and we think it's ready for running code -- in fact many services have upgraded already.  I'm personally nearly ready to declare it a 1.0 candidate (and deal only with final reviews, wording fixes, etc. that shouldn't affect existing code). 

There's one complication.  One of the motivations for the changes in Experimental was to see if we could also use it as a basis for signed JSON OAuth Tokens, so that people using both (probably a lot of people) would have fewer arbitrarily different signing mechanisms to have to deal with.  I worked on this with Dirk Balfanz and we came up with a spec proposal for the OAuth Working Group.  It turned out that two other people also had proposals, and Mike Jones of Microsoft is working on "normalizing" them into one coherent proposal, which is being thrashed out right now and will be discussed face to face at the Internet Identity Workshop in Mountain View next week.  If the consensus that comes out of that is one that wants to leverage Magic Signatures, AND there are only one or two small changes to MagicSig needed to make that happen, I'd like to propose making those changes.  If it looks like they're diverging anyway, then I'd like to just label MagicSig 1.0 and be done.  Either way, we should have something that we believe will be stable for a while in a couple of weeks.  The Experimental draft is much improved from the feedback it's received by considering additional environments and use cases, including supporting more general data types and a compact serialization for space savings, so the effort has been worth while in any case.

The changes being asked for so far amount mainly to things like not including the == padding in the base64 encoded data.  If that were the only thing keeping us from converging I'd change it in a second.  

As an aside:  Does anyone else have an opinion about including the padding or not including it just based on the merits, not on compatibility with OAuth Tokens?  

Pros: Shorter, does not require URL escaping.

Cons: Nonstandard, requires library switch or manual truncation of content & re-padding if library doesn't support.  Also, you can detect and fix padding over-escaping fairly easily.

 

--
John Panzer / Google
jpa...@google.com / abstractioneer.org / @jpanzer

On Thu, Oct 28, 2010 at 9:58 AM, James Walker <wal...@walkah.net> wrote:

Hey Mike (et al),

On Tue, Oct 26, 2010 at 6:46 PM, Mike Macgirvin <mi...@macgirvin.com> wrote:
> During some of my compatibility testing w/mistpark, I've discovered
> that several projects are using incorrect implementations of salmon
> magic signatures. In fact, I have not yet found one that is
> compliant.

It's true, the signing message format has changed (I apparently missed
it getting promoted from 'experimental').

> This creates a huge mess because one must send two slaps to each of
> these providers - the first using the correct implementation, and then
> if that fails - repeat using the incorrect format. These have to be
> verified in the same way, falling back to the incorrect format if the
> correct one fails.
>
> Unfortunately, this means that each of you who wishes to provide
> federated services must also implement this duplication of effort
> until such time as everybody has upgraded and the broken
> implementations begin to fade from existence (this could take a few
> years -  which is why everybody needs to start fixing it today(!)).

Agreed it's a hassle, but it's the nature of the beast when
implementing draft specs that are still under heavy revision. I
believe Salmon is getting "real close now"(tm) - but I'll let John
provide official word there.

That said, I can speak for StatusNet and say - thanks for bringing
this to our attention, I'll be rolling out a fix ASAP.

> The crux of the problem is in section 8 of draft-panzer-magicsig-00
> (section 9.1 if you're working off the trunk).
>
> The implementations I've encountered are only signing the
> base64url_encoded data itself. The correct method is to append the
> string ".YXBwbGljYXRpb24vYXRvbSt4bWw=.YmFzZTY0dXJs.UlNBLVNIQTI1Ng=="
> to the armoured data and use that both for signing and verification.
> (This is a simplification, you should calculate this string yourself
> to allow for future revisions).
>

Good catch. Thanks again!
--
James Walker :: http://walkah.net/ :: http://james.status.net/

[John Panzer]

We're using bog-standard encryption libraries for  RSA-SHA256.  I'm curious as to what encryption problems PHP brings to the table?

(Note that the reason for not requiring certificates, and using a simple RSA keypair format, is that you end up needing to install compiled C libraries to parse ASN.1 stuff for a lot (all?) non-Java languages, which in some environments is a nonstarter.)

--
John Panzer / Google
jpa...@google.com / abstractioneer.org / @jpanzer

On Fri, Oct 29, 2010 at 2:27 PM, Mike Macgirvin <mi...@macgirvin.com> wrote:

Cons: Nonstandard

That pretty well sums it up for me. With all due respects John - salmon magic envelopes already require the addition of a lot of bloat to support its unique way of doing things - at least on the php platform.

I understand the reasoning and appreciate the mechanisms. But couldn't some of this stuff be done with standard libraries and not require special salmon flavours of everything (e.g. encryption libs, key formats, base64 encoders, etc.)?

[James Walker]

On Fri, Oct 29, 2010 at 6:00 PM, John Panzer <jpa...@google.com> wrote:
> We're using bog-standard encryption libraries for  RSA-SHA256.  I'm curious
> as to what encryption problems PHP brings to the table?

Plenty - but that's PHP's problem ... lack of quality libraries. (imnsho).

That said - it was an issue when we first started implementing salmon,
but the phpseclib library that StatusNet is using has proven to be
reliable since. Thought, the PHP gmp extension makes a *world* of
difference for performance (but that's true of *any* crypto -
diffe-hellman for openid, et al has the same issue).

I think the only "non standard" bit is the base64url encoding but it's
a single line to implement in PHP on top of the standard
base64_encode/decode.

> (Note that the reason for not requiring certificates, and using a simple RSA
> keypair format, is that you end up needing to install compiled C libraries
> to parse ASN.1 stuff for a lot (all?) non-Java languages, which in some
> environments is a nonstarter.)
>
> --
> John Panzer / Google
> jpa...@google.com / abstractioneer.org / @jpanzer
>
>
> On Fri, Oct 29, 2010 at 2:27 PM, Mike Macgirvin <mi...@macgirvin.com> wrote:
>>>
>>> Cons: Nonstandard
>>
>> That pretty well sums it up for me. With all due respects John - salmon
>> magic envelopes already require the addition of a lot of bloat to support
>> its unique way of doing things - at least on the php platform.
>>
>> I understand the reasoning and appreciate the mechanisms. But couldn't
>> some of this stuff be done with standard libraries and not require special
>> salmon flavours of everything (e.g. encryption libs, key formats, base64
>> encoders, etc.)?
>>
>
>

--

[John Panzer]

On Fri, Oct 29, 2010 at 3:10 PM, James Walker <wal...@walkah.net> wrote:

On Fri, Oct 29, 2010 at 6:00 PM, John Panzer <jpa...@google.com> wrote:
> We're using bog-standard encryption libraries for  RSA-SHA256.  I'm curious
> as to what encryption problems PHP brings to the table?

Plenty - but that's PHP's problem ... lack of quality libraries. (imnsho).

That said - it was an issue when we first started implementing salmon,
but the phpseclib library that StatusNet is using has proven to be
reliable since. Thought, the PHP gmp extension makes a *world* of
difference for performance (but that's true of *any* crypto -
diffe-hellman for openid, et al has the same issue).

I think the only "non standard" bit is the base64url encoding but it's
a single line to implement in PHP on top of the standard
base64_encode/decode.

OK, that's good to know.  (Note: I'd call base64url "not as well supported as we'd like" rather than non standard, since it's in the original base64 encoding standard; this is actually somewhat important since if a library DOES implement it, I think it's likely to be interoperable with other libraries.)

[Mike Macgirvin]