What is meant by 'folding whitespace...'

[john napiorkowski]

In the docs at:

http://code.google.com/apis/storage/docs/developer-guide.html#authentication

There's a section of rules for how to sign the extended headers. One
rule as follows:

"Replace any folding whitespace or newlines (CRLF or LF) with a single
space."

I'm not 100% clear what 'folding' whitespace means... I assume you
mean all whitespace inside the value, although calling it 'folding'
leads me to think of something else.

I see in the boto code on:

http://code.google.com/p/gsutil/source/browse/trunk/src/boto/utils.py#73

there's a .strip which removes preceding and trailing white-space, so
I guess I'll add that as well, although the docs don't mention that.
However I don't see any code that collapses white-space in any
manner. Anyone know what this means? Right now I'm doing this using
String::Utils::crunch (http://search.cpan.org/~miko/String-Util-0-12/
Util.pm#crunch%28string%29)

It would be really handy if there was a test suite I could check out.
It wouldn't really matter what language, just seeing the assertions
would give me a lot more confidence I'm writing the correct stuff.

[Andrew Chilton]

Hi John,

On 15 June 2010 13:41, john napiorkowski <jjn...@gmail.com> wrote:
> "Replace any folding whitespace or newlines (CRLF or LF) with a single
> space."

Essentially it means that for whatever length of whitespace, they
should be 'folder' down to only one space.

e.g. CR LF SP SP SP CR LF SP TAB SP -> SP

So whenever you get a length of whitespace, you should replace it all
with a single space.

Hope that helps,
Andy

--
contact: Andrew Chilton
website: http://www.chilts.org/blog/

[john napiorkowski]

It does help thanks!

I have code that is working (ie getting a not error response) but I'm
still a bit confused about the folding rules. It seems like after I
'folder down' whitespace in a header value, I have set the request
header to the 'foldered down' value as well. Does that sound right?
Because what if for some reason your HTTP header value does require
extra spaces and so forth? Seems like an edge case to me but I want
to be sure before I require this in my code.

Additionally, it does seem I need to trim whitespace from the start
and end of values, otherwise I get errors. Is that intentional or a
bug (or me doing something wrong?)

Thank you!
John

On Jun 14, 11:14 pm, Andrew Chilton <andychil...@gmail.com> wrote:
> Hi John,
>

[Andrew Chilton]

Hi John,

On 15 June 2010 15:34, john napiorkowski <jjn...@gmail.com> wrote:
> I have code that is working (ie getting a not error response) but I'm
> still a bit confused about the folding rules.  It seems like after I
> 'folder down' whitespace in a header value, I have set the request
> header to the 'foldered down' value as well.  Does that sound right?
> Because what if for some reason your HTTP header value does require
> extra spaces and so forth?  Seems like an edge case to me but I want
> to be sure before I require this in my code.

I think you want to post examples with what you have, what you think
you want and what you ended up with. Also, remember that this is only
for signing the request (ie. when making the
CanonicalExtensionHeaders, so it keeps your real headers intact).

> Additionally, it does seem I need to trim whitespace from the start
> and end of values, otherwise I get errors.  Is that intentional or a
> bug (or me doing something wrong?)

That's completely correct, in fact it says so in points 3, 4 and 5 of
that section you linked to. It's probably you doing something wrong.
Take a look at the example on that same page to compare.

Cheers,

[john napiorkowski]

Okay so I think I've narrowed this down a bit. Here's what I am
doing. First I create a request with a bunch of headers that need
extensive normalization before I can sign them:

my $headers = HTTP::Headers->new(
Date => time2str(time),
a => 1,
b => 2,
'x-goog-meta-test-a' => "a",
'x-goog-meta-TEST-b' => "b",
'x-goOG-meta-test-A' => "aa ",
'x-Goog-MEta-test-c' => " c",
'x-gOOg-meta-test-d' => " d ",
'x-gOOg-meTa-tESt-e' => "a bc",
);

I realize you may not be so familiar with Perl but hopefully the above
is straightforward. I just create a bunch of headers where I mess up
the case and toss in extra whitespace. Now, the code I am using to
normalize works fine EXCEPT for the last header:

'x-gOOg-meTa-tESt-e' => "a bc",

That value is "a" + two spaces + "bc"

That header causes a failure for me. If I change it to one space
between the "a" and "bc" the code runs fine. Here's the error message
I get:

<?xml version='1.0' encoding='UTF-8'?
><Error><Code>SignatureDoesNotMatch</Code><Message>The request
signature we calculated does not match the signature you provided.
Check your Google secret key and signing method.</
Message><StringToSign>GET


Tue, 15 Jun 2010 14:08:14 GMT
x-goog-meta-test-a:a,aa
x-goog-meta-test-b:b
x-goog-meta-test-c:c
x-goog-meta-test-d:d
x-goog-meta-test-e:a bc
/</StringToSign></Error> at t/basic.t line 77.

I think its odd that the StringToSign has two spaces in the "x-goog-
meta-test-e:a bc" bit. Based on my understanding of the docs I'd
think it would be one space. At first I though I was accidentally
inserting some evil hidden code instead of a space but I've verified
this is definitely two spaces.

Here's a dump of what I have normalized my header string before I sign
and base encode it:
(
"GET",
"",
"",
"Tue, 15 Jun 2010 14:08:14 GMT",
"x-goog-meta-test-a:a,aa",
"x-goog-meta-test-b:b",
"x-goog-meta-test-c:c",
"x-goog-meta-test-d:d",
"x-goog-meta-test-e:a bc",
)

I know the above might be hard to read but there's only a single space
between the "a" and "bc" in that last header.

Any ideas? I don't want to continue writing my wrapper API until I
have a solid set of test cases for the encoding rules. After I get
passed this issue I still need a bunch of unicode tests, after which I
think the bulk of the rest of the API work will be easy. Thanks!

Sincerely,
John Napiorkowski

On Jun 15, 12:30 am, Andrew Chilton <andychil...@gmail.com> wrote:
> Hi John,
>

[Google Storage Team]

Hi,

FYI, the concept of folding white space is defined in http://tools.ietf.org/html/rfc2822#page-11

 - Mike

--
You received this message because you are subscribed to the Google Groups "Google Storage for Developers" group.
To post to this group, send email to gs-dis...@googlegroups.com.
To unsubscribe from this group, send email to gs-discussio...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gs-discussion?hl=en.

[john napiorkowski]

Wow, folding is not simply collapsing whitespace, there's a pretty
thick set of rules here.

The thing is... I don't see an example of this level of folding
compliance in the gsutil example. gsutil appears to just simply trim
the string and move on. So I have to ask if this is something that's
actually implemented on the receiving side of Google Storage? For
example is there a unit test you have out there for this? If so would
be awesome if that could be shared so that I could write a real test
suite and not just guess my code works.

Thanks!

On Jun 15, 1:50 pm, Google Storage Team <gs-t...@google.com> wrote:
> Hi,
>
> FYI, the concept of folding white space is defined inhttp://tools.ietf.org/html/rfc2822#page-11
> <http://tools.ietf.org/html/rfc2822#page-11> - Mike
>

> On Mon, Jun 14, 2010 at 8:14 PM, Andrew Chilton <andychil...@gmail.com>wrote:
>
> > Hi John,
>

> > On 15 June 2010 13:41, john napiorkowski <jjn1...@gmail.com> wrote:
> > > "Replace any folding whitespace or newlines (CRLF or LF) with a single
> > > space."
>
> > Essentially it means that for whatever length of whitespace, they
> > should be 'folder' down to only one space.
>
> > e.g. CR LF SP SP SP CR LF SP TAB SP -> SP
>
> > So whenever you get a length of whitespace, you should replace it all
> > with a single space.
>
> > Hope that helps,
> > Andy
>
> > --
> > contact: Andrew Chilton
> > website:http://www.chilts.org/blog/
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google Storage for Developers" group.
> > To post to this group, send email to gs-dis...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > gs-discussio...@googlegroups.com<gs-discussion%2Bunsu...@googlegroups.com>
> > .