Bug in the way the API checks the signature since at least v9.7

[si...@ateb.co.uk]

Hi,

Since upgrading from 9.3  to 9.7 and then 9.8 I am having problems working with the API from some clients.

The API expects all parameters to be UrlEncoded before the request is signed.

The code in ./api/index.php uses parse_str (line 35) to extract the parameters into variables and then later rebuilds the query string using http_build_query (line 40) before checking it against the signature (line 57).

The problem arises here because the UrlEncoding should be case INsensitve according to the spec, but the re-encoding using http_build_query forces all percent encoded hex characters to uppercase. This then only passes the signature check if the original URL parameters were also in uppercase. Here a / character can be encoded to %2F or %2f legally - but the API only allows the former.

Thus a url like this works:

user=myuser&function=do_search&param1=up%2Fdown&sign=xxxxx

but this doesn't:

user=myuser&function=do_search&param1=up%2fdown&sign=yyyyy

and we get Invalid Signature.

The problem is that some implementations of UrlEncode (e.g. System.Web.HttpUtility.UrlEncode in Microsoft .NET) encode hex digits in lower case!

This means that these clients can no longer work with ResourceSpace.

Version 9.3 did not do this decoding and then re-encoding before the signature check and so worked ok with all UrlEncoding implementations in all languages.

Is there a plan to fix this?

[Adam]

OMG.  I've been spending hours trying to figure out why my signatures are invalid when using create_resource api.  Thank you so much for documenting this!

The tool I'm using is encoding spaces in the json metadata as "%20", but when I log the value on the server the spaces are encoded as a "+".

I confirmed this is the issue by constructing a special query string the same way the server does and signing that, but sending the real query string.  When I do that, the create_resource api call succeeds.

[si...@ateb.co.uk]

Yes, the way Resourcespace is doing the signature check now seems to be fundamentally flawed.

As you have found, it not just that some UrlEncoded strings can contain upper and lowercase hex values e.g. %2F (works in RS)  or %2f (doesn't work in RS) but also that the earlier RFC of UrlEncoding converts spaces to + and the most recent disallows this and converts to %20. (IIS server on Windows will not accept any urls containing + these days)

So the possible UrlEncoded character that are going to fail the signature check are probably:

%20 %2a %2b  %2c %2f %3a %3b %3d %3f %5b %5d

I hope someone from Montala picks up this thread and can investigate!

thanks
Simon

[Adam]

To give more context on this issue, since it's still causing issues for me.

The file api/index.php uses the function http_build_query() to rebuild the query string on the server before verifying the signature.

From the http_build_query PHP docs

encoding_type

By default, PHP_QUERY_RFC1738.

If encoding_type is PHP_QUERY_RFC1738, then encoding is performed per » RFC 1738 and the application/x-www-form-urlencoded media type, which implies that spaces are encoded as plus (+) signs.

If encoding_type is PHP_QUERY_RFC3986, then encoding is performed according to » RFC 3986, and spaces will be percent encoded (%20).

Why doesn't resource space check the signature using query string provided by the user?