How does M/DB calculate the signature?
40 views
Skip to first unread message
Dennis
Dec 29, 2010, 1:28:42 AM12/29/10
to M/DB Community Forum
I calculate the signature in the same way as AWS-SDK(Signature
Version: 2, Method:HmacSHA256). It works in most scenarios, but the
error happens when '%' or ' ' involved in the value.
Here is an example about '%':
Insert an item with the attribute whose key is "field1" and value is "!
~ZO9uI[^-QpkMcr4kZ8DA~e(j&Wcx2a?+ni+&Opf$YGF2l>Z><$:ay5o0K(6m|9@M>O/
%56Vrt\>UVnXp*Zf9EC4L/{4M#L4|Q:"
However, when I removed the '%', it works.
I tested by using a lot of data. the conclusion is: if the values
don't involve '%' and ' ', inserting and updating an item work
without errors.
Is there any explanation?
Thanks,
Dennis
Version: 2, Method:HmacSHA256). It works in most scenarios, but the
error happens when '%' or ' ' involved in the value.
Here is an example about '%':
Insert an item with the attribute whose key is "field1" and value is "!
~ZO9uI[^-QpkMcr4kZ8DA~e(j&Wcx2a?+ni+&Opf$YGF2l>Z><$:ay5o0K(6m|9@M>O/
%56Vrt\>UVnXp*Zf9EC4L/{4M#L4|Q:"
However, when I removed the '%', it works.
I tested by using a lot of data. the conclusion is: if the values
don't involve '%' and ' ', inserting and updating an item work
without errors.
Is there any explanation?
Thanks,
Dennis
rtweed
Jan 4, 2011, 5:39:36 AM1/4/11
to M/DB Community Forum
I believe the % problem is now fixed. I've put the new build of MDB.m
into Github at https://github.com/robtweed/mdb
Simply copy the file MDB.m from the Github repo and overwrite the
version that you'll find in /usr/local/gtm/ewd in your M/DB
Appliance. Then relink the M/DB routine:
cd /usr/local/gtm/ewd
mumps MDB.m
and then restart Apache:
sudo /etc/init.d/apache2 restart
Let me know if you still have problems
Rob
into Github at https://github.com/robtweed/mdb
Simply copy the file MDB.m from the Github repo and overwrite the
version that you'll find in /usr/local/gtm/ewd in your M/DB
Appliance. Then relink the M/DB routine:
cd /usr/local/gtm/ewd
mumps MDB.m
and then restart Apache:
sudo /etc/init.d/apache2 restart
Let me know if you still have problems
Rob
Dennis
Jan 6, 2011, 3:32:56 AM1/6/11
to M/DB Community Forum
Hi Rob,
Contrats! I have tested the new MDB.m, which indicates that you have
successfully fixed the % problem.
However, the ' '(a BLANK char, which is got by pressing the SPACE key)
problem still exists.
When values include ' ', the signature doesn't match too. Could you
solve this problem too?
Dennis
On Jan 4, 6:39 pm, rtweed <rob.tw...@gmail.com> wrote:
> I believe the % problem is now fixed. I've put the new build of MDB.m
> into Github athttps://github.com/robtweed/mdb
Contrats! I have tested the new MDB.m, which indicates that you have
successfully fixed the % problem.
However, the ' '(a BLANK char, which is got by pressing the SPACE key)
problem still exists.
When values include ' ', the signature doesn't match too. Could you
solve this problem too?
Dennis
On Jan 4, 6:39 pm, rtweed <rob.tw...@gmail.com> wrote:
> I believe the % problem is now fixed. I've put the new build of MDB.m
rtweed
Jan 6, 2011, 6:14:03 AM1/6/11
to M/DB Community Forum
Dennis
I'm really surprised that spaces cause problems - I don't remember
having had problems myself but I'll test it out and see what the
problem might be.
Just to be sure - we're talking about ASCII 32 - the normal space
character? On its own as a value? Or within a character string
value?...or both?
Unfortunately I'm not sure when I'll be able to do this as my network
is currently down due to a BT phone line problem...once they fix it
I'll check it out
Rob
I'm really surprised that spaces cause problems - I don't remember
having had problems myself but I'll test it out and see what the
problem might be.
Just to be sure - we're talking about ASCII 32 - the normal space
character? On its own as a value? Or within a character string
value?...or both?
Unfortunately I'm not sure when I'll be able to do this as my network
is currently down due to a BT phone line problem...once they fix it
I'll check it out
Rob
Dennis
Jan 6, 2011, 8:32:53 PM1/6/11
to M/DB Community Forum
Rob,
Yes, the character I mentioned is the SPACE with the value of 32 in
the ASCII table.
The problem happens not only when ' '(SPACE) is as a value, but also
when it's in a string.
As examples, " " and "abc def" can both cause this problem.
Dennis
Yes, the character I mentioned is the SPACE with the value of 32 in
the ASCII table.
The problem happens not only when ' '(SPACE) is as a value, but also
when it's in a string.
As examples, " " and "abc def" can both cause this problem.
Dennis
rtweed
Jan 7, 2011, 3:25:54 AM1/7/11
to M/DB Community Forum
I've tested the latest version of M/DB (build 38) using the latest
version of the Bolso client. Adding/updating fields that contain
spaces appears to work fine for me (which is what I expected since I'd
have expected to be inundated with complaints if M/DB couldn't handle
spaces in data!). Here's the trace for when I use PutAttributes to
set an attribute named spacetest with a value of "abc def ghi":
^%zewdTrace(2)="62098,29276: string to sign:POST"_
$C(10)_"192.168.1.110"_$C(10)_
"/mdb/request.mgwsi"_
$C(10)_"AWSAccessKeyId=xxxxxx&Action=PutAttributes&A
ttribute.0.Name=spaces&Attribute.0.Replace=false&Attribute.
0.Value=abc
%20def
%20ghi&DomainName=testing&ItemName=spacetest&SignatureMethod=Hma
cSHA1&SignatureVersion=2&Timestamp=2011-01-07T08%3A06%3A49Z&Version=20
07-11-07"
^%zewdTrace(3)="62098,29276: hash=cfYACV1ZQhD6AnBvrMf67xYPNKc=;
signature rcvd=c
fYACV1ZQhD6AnBvrMf67xYPNKc="
(Note: I've replaced the AWSAccessKeyId so you'll get a different hash
if you out it through the signing logic!)
Bolso is using HmacSHA1 rather than the now recommended HmacSHA256
encryption, but that shouldn't make any difference.
Perhaps it's the client you are using. The AWS "string to sign"
escape rules specifically say that space characters must be converted
to %20 rather than the usual URL escaped + character (you can see this
in the trace above). Perhaps that's what your client is (wrongly)
sending?
Can you run a trace for a similar example that is failing for you? For
trace mode:
In a terminal session:
cd /usr/local/gtm/ewd
$gtm
At the GTM> prompt:
s ^zewd("trace")=1
k ^%zewdTrace
k ^mdbKey
k ^mdbdata
Now run your test (just once) through M/DB
At the GTM> prompt:
zwr ^%zewdTrace (This will show a trace of M/DB's actions)
zwr ^mdbKey (This will provide a breakdown of the unescaped
name/value pairs received by M/DB)
zwr ^mdbdata (This will provide a breakdown of the raw escaped
name/value pairs received by M/DB)
To reset trace mode:
s ^zewd("trace")=0
k ^%zewdTrace
k ^mdbKey
k ^mdbdata
To get out of the GT.M shell, at the GTM> prompt:
h (followed by the Enter key)
Can you post a copy of the trace information?
Regards
Rob
version of the Bolso client. Adding/updating fields that contain
spaces appears to work fine for me (which is what I expected since I'd
have expected to be inundated with complaints if M/DB couldn't handle
spaces in data!). Here's the trace for when I use PutAttributes to
set an attribute named spacetest with a value of "abc def ghi":
^%zewdTrace(2)="62098,29276: string to sign:POST"_
$C(10)_"192.168.1.110"_$C(10)_
"/mdb/request.mgwsi"_
$C(10)_"AWSAccessKeyId=xxxxxx&Action=PutAttributes&A
ttribute.0.Name=spaces&Attribute.0.Replace=false&Attribute.
0.Value=abc
%20def
%20ghi&DomainName=testing&ItemName=spacetest&SignatureMethod=Hma
cSHA1&SignatureVersion=2&Timestamp=2011-01-07T08%3A06%3A49Z&Version=20
07-11-07"
^%zewdTrace(3)="62098,29276: hash=cfYACV1ZQhD6AnBvrMf67xYPNKc=;
signature rcvd=c
fYACV1ZQhD6AnBvrMf67xYPNKc="
(Note: I've replaced the AWSAccessKeyId so you'll get a different hash
if you out it through the signing logic!)
Bolso is using HmacSHA1 rather than the now recommended HmacSHA256
encryption, but that shouldn't make any difference.
Perhaps it's the client you are using. The AWS "string to sign"
escape rules specifically say that space characters must be converted
to %20 rather than the usual URL escaped + character (you can see this
in the trace above). Perhaps that's what your client is (wrongly)
sending?
Can you run a trace for a similar example that is failing for you? For
trace mode:
In a terminal session:
cd /usr/local/gtm/ewd
$gtm
At the GTM> prompt:
s ^zewd("trace")=1
k ^%zewdTrace
k ^mdbKey
k ^mdbdata
Now run your test (just once) through M/DB
At the GTM> prompt:
zwr ^%zewdTrace (This will show a trace of M/DB's actions)
zwr ^mdbKey (This will provide a breakdown of the unescaped
name/value pairs received by M/DB)
zwr ^mdbdata (This will provide a breakdown of the raw escaped
name/value pairs received by M/DB)
To reset trace mode:
s ^zewd("trace")=0
k ^%zewdTrace
k ^mdbKey
k ^mdbdata
To get out of the GT.M shell, at the GTM> prompt:
h (followed by the Enter key)
Can you post a copy of the trace information?
Regards
Rob
Dennis
Jan 10, 2011, 2:42:57 AM1/10/11
to M/DB Community Forum
Rob,
Sorry for my late reply, because I am not on work at weekends.
Yes, you are right. The problem lies in escaping code.
I have solved the problem by converting the space character to '+'
instead of '%20' when calculating signature at client side.
In my observation, AWS JAVA SDK calculates signature by converting
space to '%20', BUT it sends '+' to the server!
For example, "abc def" is converted to "abc%20def" at client side, and
is sent as "abc+def" to the server(I found this by using your trace
tool, in which i saw "abc%2Bdef"). At server, M/DB calculates
signature with "abc%2Bdef", while the AWS, I guess, calculates
signature after converting "abc%2Bdef" to "abc%20def".
That might be why it failed in my case.
I am not sure this can be an explanation, because I have few
experience in detailed HTTPstuff. Rob, if you can't locate this
problem, just reply. I will offer detail description of this problem.
Thanks.
Dennis
Sorry for my late reply, because I am not on work at weekends.
Yes, you are right. The problem lies in escaping code.
I have solved the problem by converting the space character to '+'
instead of '%20' when calculating signature at client side.
In my observation, AWS JAVA SDK calculates signature by converting
space to '%20', BUT it sends '+' to the server!
For example, "abc def" is converted to "abc%20def" at client side, and
is sent as "abc+def" to the server(I found this by using your trace
tool, in which i saw "abc%2Bdef"). At server, M/DB calculates
signature with "abc%2Bdef", while the AWS, I guess, calculates
signature after converting "abc%2Bdef" to "abc%20def".
That might be why it failed in my case.
I am not sure this can be an explanation, because I have few
experience in detailed HTTPstuff. Rob, if you can't locate this
problem, just reply. I will offer detail description of this problem.
Thanks.
Dennis
Dennis
Jan 10, 2011, 2:58:32 AM1/10/11
to M/DB Community Forum
Oops, that can't be a solution, because the space character is stored
as '+' in the M/DB.
Dennis
as '+' in the M/DB.
Dennis
Dennis
Jan 10, 2011, 3:50:32 AM1/10/11
to M/DB Community Forum
Rob,
To simplify this problem.
I access M/DB via JAVA AWS-SDK. The M/DB server calculates the
signature considering the space character as '+'(which I found via
your trace tool). However, the same request is OK when sent to the
SimpleDB server.
Thanks.
Dennis
To simplify this problem.
I access M/DB via JAVA AWS-SDK. The M/DB server calculates the
signature considering the space character as '+'(which I found via
your trace tool). However, the same request is OK when sent to the
SimpleDB server.
Thanks.
Dennis
Rob Tweed
Jan 10, 2011, 4:42:36 AM1/10/11
to mdb-commu...@googlegroups.com
So just to completely clarify, can you post the relevant values that were in ^%zewdTrace, ^mdbKey and ^mdbdata so I can correctly trace the issue through the M/DB code
--
Rob Tweed
Director, M/Gateway Developments Ltd
http://www.mgateway.com
------------------
iWD: build iPhone Apps faster :
http://www.mgateway.com/iwd.html
Basically it sounds like the Java AWS-SDK is behaving differently from most other SimpleDB clients, but then again nothing ever surprises me about Java....
Cheers
Rob
--
You received this message because you are subscribed to the Google Groups "M/DB Community Forum" group.
To post to this group, send an email to mdb-commu...@googlegroups.com.
To unsubscribe from this group, send email to mdb-community-f...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mdb-community-forum?hl=en-GB.
--
Rob Tweed
Director, M/Gateway Developments Ltd
http://www.mgateway.com
------------------
iWD: build iPhone Apps faster :
http://www.mgateway.com/iwd.html
Dennis
Jan 11, 2011, 3:10:59 AM1/11/11
to M/DB Community Forum
Info by AWS-SDK:
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient execute
INFO: Sending Request: POST http://localhost/mdb/request.mgwsi /
Parameters: (Action: PutAttributes, SignatureMethod: HmacSHA256,
ItemName: user1268118805, DomainName: usertable, Attribute.1.Value:
abc def, AWSAccessKeyId: test, Version: 2009-04-15, SignatureVersion:
2, Attribute.1.Name: field4, Timestamp: 2011-01-11T07:54:32.164Z,
Signature: ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M=, Attribute.
1.Replace: true, )
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient
handleErrorResponse
INFO: Received error response: Status Code: 403, AWS Request ID:
d830030b-4312-7746-38b3-ac61b41db1eb, AWS Error Code:
SignatureDoesNotMatch, AWS Error Message: The request signature we
calculated does not match the signature you provided. Check your
Secret Access Key and signing method. Consult the service
documentation for details
Info by the trace tool:
^%zewdTrace(8)="62102,57272: string to sign:POST"_$C(10)_"localhost"_
$C(10)_"/md
b/request.mgwsi"_
$C(10)_"AWSAccessKeyId=test&Action=PutAttributes&Attr
ibute.1.Name=field4&Attribute.1.Replace=true&Attribute.
1.Value=abc%2Bd
ef&DomainName=usertable&ItemName=user1268118805&SignatureMethod=HmacSH
A256&SignatureVersion=2&Timestamp=2011-01-11T07%3A54%3A32.164Z&Version
=2009-04-15"
^%zewdTrace(9)="62102,57272: hash=1bPDemoMe43JZ9s5H8Dg18OG5aH
+5RW6Xf7E9cf6va0=;
signature rcvd=ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M="
^%zewdTrace(10)="MDB request processing ended at 62102,57272"
^mdbKey(2,"Action")="PutAttributes"
^mdbKey(2,"Attribute.1.Name")="field4"
^mdbKey(2,"Attribute.1.Replace")="true"
^mdbKey(2,"Attribute.1.Value")="abc+def" (I don't know why the
space character is considered as '+' here)
^mdbKey(2,"DomainName")="usertable"
^mdbKey(2,"ItemName")="user1268118805"
^mdbKey(2,"Signature")="ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M="
^mdbKey(2,"SignatureMethod")="HmacSHA256"
^mdbKey(2,"SignatureVersion")=2
^mdbKey(2,"Timestamp")="2011-01-11T07:54:32.164Z"
^mdbKey(2,"Version")="2009-04-15"
^mdbdata(2,"Action",1)="PutAttributes"
^mdbdata(2,"Attribute.1.Name",1)="field4"
^mdbdata(2,"Attribute.1.Replace",1)="true"
^mdbdata(2,"Attribute.1.Value",1)="abc+def"
^mdbdata(2,"DomainName",1)="usertable"
^mdbdata(2,"ItemName",1)="user1268118805"
^mdbdata(2,"Signature",1)="ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M
%3D"
^mdbdata(2,"SignatureMethod",1)="HmacSHA256"
^mdbdata(2,"SignatureVersion",1)=2
^mdbdata(2,"Timestamp",1)="2011-01-11T07%3A54%3A32.164Z"
^mdbdata(2,"Version",1)="2009-04-15"
On Jan 10, 5:42 pm, Rob Tweed <rob.tw...@gmail.com> wrote:
> So just to completely clarify, can you post the relevant values that were
> in ^%zewdTrace, ^mdbKey and ^mdbdata so I can correctly trace the issue
> through the M/DB code
>
> Basically it sounds like the Java AWS-SDK is behaving differently from most
> other SimpleDB clients, but then again nothing ever surprises me about
> Java....
>
> Cheers
>
> Rob
>
> On 10 January 2011 08:50, Dennis <dennis.sun...@gmail.com> wrote:
>
>
>
>
>
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient execute
INFO: Sending Request: POST http://localhost/mdb/request.mgwsi /
Parameters: (Action: PutAttributes, SignatureMethod: HmacSHA256,
ItemName: user1268118805, DomainName: usertable, Attribute.1.Value:
abc def, AWSAccessKeyId: test, Version: 2009-04-15, SignatureVersion:
2, Attribute.1.Name: field4, Timestamp: 2011-01-11T07:54:32.164Z,
Signature: ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M=, Attribute.
1.Replace: true, )
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient
handleErrorResponse
INFO: Received error response: Status Code: 403, AWS Request ID:
d830030b-4312-7746-38b3-ac61b41db1eb, AWS Error Code:
SignatureDoesNotMatch, AWS Error Message: The request signature we
calculated does not match the signature you provided. Check your
Secret Access Key and signing method. Consult the service
documentation for details
Info by the trace tool:
^%zewdTrace(8)="62102,57272: string to sign:POST"_$C(10)_"localhost"_
$C(10)_"/md
b/request.mgwsi"_
$C(10)_"AWSAccessKeyId=test&Action=PutAttributes&Attr
ibute.1.Name=field4&Attribute.1.Replace=true&Attribute.
1.Value=abc%2Bd
ef&DomainName=usertable&ItemName=user1268118805&SignatureMethod=HmacSH
A256&SignatureVersion=2&Timestamp=2011-01-11T07%3A54%3A32.164Z&Version
=2009-04-15"
^%zewdTrace(9)="62102,57272: hash=1bPDemoMe43JZ9s5H8Dg18OG5aH
+5RW6Xf7E9cf6va0=;
signature rcvd=ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M="
^%zewdTrace(10)="MDB request processing ended at 62102,57272"
^mdbKey(2,"Action")="PutAttributes"
^mdbKey(2,"Attribute.1.Name")="field4"
^mdbKey(2,"Attribute.1.Replace")="true"
^mdbKey(2,"Attribute.1.Value")="abc+def" (I don't know why the
space character is considered as '+' here)
^mdbKey(2,"DomainName")="usertable"
^mdbKey(2,"ItemName")="user1268118805"
^mdbKey(2,"Signature")="ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M="
^mdbKey(2,"SignatureMethod")="HmacSHA256"
^mdbKey(2,"SignatureVersion")=2
^mdbKey(2,"Timestamp")="2011-01-11T07:54:32.164Z"
^mdbKey(2,"Version")="2009-04-15"
^mdbdata(2,"Action",1)="PutAttributes"
^mdbdata(2,"Attribute.1.Name",1)="field4"
^mdbdata(2,"Attribute.1.Replace",1)="true"
^mdbdata(2,"Attribute.1.Value",1)="abc+def"
^mdbdata(2,"DomainName",1)="usertable"
^mdbdata(2,"ItemName",1)="user1268118805"
^mdbdata(2,"Signature",1)="ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M
%3D"
^mdbdata(2,"SignatureMethod",1)="HmacSHA256"
^mdbdata(2,"SignatureVersion",1)=2
^mdbdata(2,"Timestamp",1)="2011-01-11T07%3A54%3A32.164Z"
^mdbdata(2,"Version",1)="2009-04-15"
On Jan 10, 5:42 pm, Rob Tweed <rob.tw...@gmail.com> wrote:
> So just to completely clarify, can you post the relevant values that were
> in ^%zewdTrace, ^mdbKey and ^mdbdata so I can correctly trace the issue
> through the M/DB code
>
> Basically it sounds like the Java AWS-SDK is behaving differently from most
> other SimpleDB clients, but then again nothing ever surprises me about
> Java....
>
> Cheers
>
> Rob
>
>
>
>
>
>
Dennis
Jan 11, 2011, 3:00:22 AM1/11/11
to M/DB Community Forum
The info by Java AWS-SDK:
> > mdb-community-f...@googlegroups.com<mdb-community-forum%2Bunsu bsc...@googlegroups.com>
> > .
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient execute
INFO: Sending Request: POST http://localhost/mdb/request.mgwsi /
Parameters: (Action: PutAttributes, SignatureMethod: HmacSHA256,
ItemName: user1268118805, DomainName: usertable, Attribute.1.Value:
abc def, AWSAccessKeyId: test, Version: 2009-04-15, SignatureVersion:
2, Attribute.1.Name: field4, Timestamp: 2011-01-11T07:54:32.164Z,
Signature: ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M=, Attribute.
1.Replace: true, )
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient
handleErrorResponse
INFO: Received error response: Status Code: 403, AWS Request ID:
d830030b-4312-7746-38b3-ac61b41db1eb, AWS Error Code:
SignatureDoesNotMatch, AWS Error Message: The request signature we
calculated does not match the signature you provided. Check your
Secret Access Key and signing method. Consult the service
documentation for details
Status Code: 403, AWS Request ID: d830030b-4312-7746-38b3-
ac61b41db1eb, AWS Error Code: SignatureDoesNotMatch, AWS Error
Message: The request signature we calculated does not match the
signature you provided. Check your Secret Access Key and signing
method. Consult the service documentation for details
The info by your trace tool:
INFO: Sending Request: POST http://localhost/mdb/request.mgwsi /
Parameters: (Action: PutAttributes, SignatureMethod: HmacSHA256,
ItemName: user1268118805, DomainName: usertable, Attribute.1.Value:
abc def, AWSAccessKeyId: test, Version: 2009-04-15, SignatureVersion:
2, Attribute.1.Name: field4, Timestamp: 2011-01-11T07:54:32.164Z,
Signature: ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M=, Attribute.
1.Replace: true, )
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient
handleErrorResponse
INFO: Received error response: Status Code: 403, AWS Request ID:
d830030b-4312-7746-38b3-ac61b41db1eb, AWS Error Code:
SignatureDoesNotMatch, AWS Error Message: The request signature we
calculated does not match the signature you provided. Check your
Secret Access Key and signing method. Consult the service
documentation for details
Status Code: 403, AWS Request ID: d830030b-4312-7746-38b3-
ac61b41db1eb, AWS Error Code: SignatureDoesNotMatch, AWS Error
Message: The request signature we calculated does not match the
signature you provided. Check your Secret Access Key and signing
method. Consult the service documentation for details
> So just to completely clarify, can you post the relevant values that were
> in ^%zewdTrace, ^mdbKey and ^mdbdata so I can correctly trace the issue
> through the M/DB code
>
> Basically it sounds like the Java AWS-SDK is behaving differently from most
> other SimpleDB clients, but then again nothing ever surprises me about
> Java....
>
> Cheers
>
> Rob
>
> in ^%zewdTrace, ^mdbKey and ^mdbdata so I can correctly trace the issue
> through the M/DB code
>
> Basically it sounds like the Java AWS-SDK is behaving differently from most
> other SimpleDB clients, but then again nothing ever surprises me about
> Java....
>
> Cheers
>
> Rob
>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/mdb-community-forum?hl=en-GB.
>
> --
> Rob Tweed
> Director, M/Gateway Developments Ltdhttp://www.mgateway.com
> >http://groups.google.com/group/mdb-community-forum?hl=en-GB.
>
> --
> Rob Tweed
Dennis
Jan 11, 2011, 3:04:23 AM1/11/11
to M/DB Community Forum
Info by AWS-SDK:
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient execute
INFO: Sending Request: POST http://localhost/mdb/request.mgwsi /
Parameters: (Action: PutAttributes, SignatureMethod: HmacSHA256,
ItemName: user1268118805, DomainName: usertable, Attribute.1.Value:
abc def, AWSAccessKeyId: test, Version: 2009-04-15, SignatureVersion:
2, Attribute.1.Name: field4, Timestamp: 2011-01-11T07:54:32.164Z,
Signature: ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M=, Attribute.
1.Replace: true, )
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient
handleErrorResponse
INFO: Received error response: Status Code: 403, AWS Request ID:
d830030b-4312-7746-38b3-ac61b41db1eb, AWS Error Code:
SignatureDoesNotMatch, AWS Error Message: The request signature we
calculated does not match the signature you provided. Check your
Secret Access Key and signing method. Consult the service
documentation for details
Info by the trace tool:
> > mdb-community-f...@googlegroups.com<mdb-community-forum%2Bunsu bsc...@googlegroups.com>
> > .
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient execute
INFO: Sending Request: POST http://localhost/mdb/request.mgwsi /
Parameters: (Action: PutAttributes, SignatureMethod: HmacSHA256,
ItemName: user1268118805, DomainName: usertable, Attribute.1.Value:
abc def, AWSAccessKeyId: test, Version: 2009-04-15, SignatureVersion:
2, Attribute.1.Name: field4, Timestamp: 2011-01-11T07:54:32.164Z,
Signature: ol8ch8O1IOrUSYei5v1dbxcQKo4OTxTYUTp4sq6ry9M=, Attribute.
1.Replace: true, )
Jan 11, 2011 3:54:32 PM com.amazonaws.http.HttpClient
handleErrorResponse
INFO: Received error response: Status Code: 403, AWS Request ID:
d830030b-4312-7746-38b3-ac61b41db1eb, AWS Error Code:
SignatureDoesNotMatch, AWS Error Message: The request signature we
calculated does not match the signature you provided. Check your
Secret Access Key and signing method. Consult the service
documentation for details
> So just to completely clarify, can you post the relevant values that were
> in ^%zewdTrace, ^mdbKey and ^mdbdata so I can correctly trace the issue
> through the M/DB code
>
> Basically it sounds like the Java AWS-SDK is behaving differently from most
> other SimpleDB clients, but then again nothing ever surprises me about
> Java....
>
> Cheers
>
> Rob
>
> in ^%zewdTrace, ^mdbKey and ^mdbdata so I can correctly trace the issue
> through the M/DB code
>
> Basically it sounds like the Java AWS-SDK is behaving differently from most
> other SimpleDB clients, but then again nothing ever surprises me about
> Java....
>
> Cheers
>
> Rob
>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/mdb-community-forum?hl=en-GB.
>
> --
> Rob Tweed
> Director, M/Gateway Developments Ltdhttp://www.mgateway.com
> >http://groups.google.com/group/mdb-community-forum?hl=en-GB.
>
> --
> Rob Tweed
rtweed
Mar 7, 2011, 4:02:36 AM3/7/11
to M/DB Community Forum
Apologies for the delay in getting an answer on this issue - pressure
of work I'm afraid.
OK we have a potential ambiguity problem with the Java interface
you're using. I can easily change M/DB's logic so that it changes all
+ characters to space characters before subsequent processing, but
your trace above clearly shows that by the time M/DB is given the name/
value pairs, the values already have a + in them that could represent
a space, but, I assume, could equally represent a genuine +
character. So, simply replacing all + characters with a space will
potentially corrupt values that are supposed to have a + in them, and
I can't afford to do that.
(The ^mdbdata global is a record of the initial raw data that is
passed to M/DB by Apache and m_apache, so that's my starting point).
There's one hope - that the Java interface escapes + characters
themselves, so they'll come through as an equivalent % escaped string,
in which case there's no problem.
Given that none of the SDB interfaces I use for testing exhibit this
barmy behaviour (only the Java community could dream up such a daft
mechanism), I can't replicate what happens, so I'm back in your
hands. Could you put some requests through M/DB where some name/value
pairs contain both spaces and genuine + characters and see what you
get in the ^mdbdata and ^mdbKey globals for me please.
If both come through as + characters, then I'm afraid there's really
nothing I can do and you'll have to get onto the people who wrote the
Java interface and get them to make it work the way everyone else
does.
If, however, the + characters are safely escaped so they don't look
the same as escaped space characters, post the trace details as in the
example above and I can fix the problem
Regards
Rob
> ...
>
> read more »
of work I'm afraid.
OK we have a potential ambiguity problem with the Java interface
you're using. I can easily change M/DB's logic so that it changes all
+ characters to space characters before subsequent processing, but
your trace above clearly shows that by the time M/DB is given the name/
value pairs, the values already have a + in them that could represent
a space, but, I assume, could equally represent a genuine +
character. So, simply replacing all + characters with a space will
potentially corrupt values that are supposed to have a + in them, and
I can't afford to do that.
(The ^mdbdata global is a record of the initial raw data that is
passed to M/DB by Apache and m_apache, so that's my starting point).
There's one hope - that the Java interface escapes + characters
themselves, so they'll come through as an equivalent % escaped string,
in which case there's no problem.
Given that none of the SDB interfaces I use for testing exhibit this
barmy behaviour (only the Java community could dream up such a daft
mechanism), I can't replicate what happens, so I'm back in your
hands. Could you put some requests through M/DB where some name/value
pairs contain both spaces and genuine + characters and see what you
get in the ^mdbdata and ^mdbKey globals for me please.
If both come through as + characters, then I'm afraid there's really
nothing I can do and you'll have to get onto the people who wrote the
Java interface and get them to make it work the way everyone else
does.
If, however, the + characters are safely escaped so they don't look
the same as escaped space characters, post the trace details as in the
example above and I can fix the problem
Regards
Rob
>
> read more »
Matt Gilg
Apr 5, 2011, 6:32:34 PM4/5/11
to M/DB Community Forum
Hello, sorry to jump in here, but I'm trying to accomplish the same
thing as Dennis.
The SDK I'm using (and presumably the one Dennis is using) is the AWS
java sdk distributed by amazon.
I'm getting the following response back after redirecting to M/DB:
<?xml version='1.0'?>
<Response><Errors><Error><Code>SignatureDoesNotMatch</
Code><Message>The request signature we calculated does not match the
Error></Errors><RequestID>9872e33a-95ec-1c49-760d-2f2938873112</
RequestID></Response>
Here is the request that I am sending:
Action=Select&SignatureMethod=HmacSHA256&AWSAccessKeyId=IISXXXXXXXXXXXXXXXXXSSD&SignatureVersion=2&Version=2009-04-15&Signature=LBdKTPV8boiMOu5n0ljF9ZoDEE1TJvwiUDF2E
%2BDZf%2Fk
%3D&Timestamp=2011-04-05T22%3A08%3A47.692Z&SelectExpression=select...
I've stripped all funky characters out of my access key and secret
key, but still no luck.
Not trying to hijack this thread, but I'm assuming we are both dealing
with the same issue.
-Matt
thing as Dennis.
The SDK I'm using (and presumably the one Dennis is using) is the AWS
java sdk distributed by amazon.
I'm getting the following response back after redirecting to M/DB:
<?xml version='1.0'?>
<Response><Errors><Error><Code>SignatureDoesNotMatch</
Code><Message>The request signature we calculated does not match the
signature you provided. Check your Secret Access Key and signing
method. Consult the service documentation for details</Message></
Error></Errors><RequestID>9872e33a-95ec-1c49-760d-2f2938873112</
RequestID></Response>
Here is the request that I am sending:
Action=Select&SignatureMethod=HmacSHA256&AWSAccessKeyId=IISXXXXXXXXXXXXXXXXXSSD&SignatureVersion=2&Version=2009-04-15&Signature=LBdKTPV8boiMOu5n0ljF9ZoDEE1TJvwiUDF2E
%2BDZf%2Fk
%3D&Timestamp=2011-04-05T22%3A08%3A47.692Z&SelectExpression=select...
I've stripped all funky characters out of my access key and secret
key, but still no luck.
Not trying to hijack this thread, but I'm assuming we are both dealing
with the same issue.
-Matt
Rob Tweed
Apr 7, 2011, 8:13:35 AM4/7/11
to mdb-commu...@googlegroups.com
I'm still awaiting feedback on the + escape issue: till someone tells me how the Java interface handles it, there's little more I can do at this point.
Not sure your issue is the same. I'd need more diagnostic info to figure out what the problem is. See my earlier postings which describe how to get M/DB to trap the low-level information I need to determine what's going wrong:
Rob
--
You received this message because you are subscribed to the Google Groups "M/DB Community Forum" group.
To post to this group, send an email to mdb-commu...@googlegroups.com.
To unsubscribe from this group, send email to mdb-community-f...@googlegroups.com.
Matt Gilg
Apr 7, 2011, 11:01:54 AM4/7/11
to M/DB Community Forum
Hi Rob,
The AWS sdk uses the apache commons-codec, version 1.3. I've been in
communication with Dennis, and he sent me a hacked version of the
Apache library that has the escape issue 'corrected', this fixed my
authentication problem.
I've also posted about executing multi-condition 'select' statements -
this seems to be problematic, at least with the AWS sdk. I've snooped
the traffic and posted the exact URL; hopefully we can find some
differences between the way AWS encodes a request, and the way other
working toolkits encode a request.
The AWS sdk is open source, and ports exist for Java, Python, PHP, and
Ruby. I'm digging in this stuff now, and would be happy to work with
you to troubleshoot any incompatibilities. I'm happy point my Java
requests at one of your M/DB instances, or do whatever else would be
useful.
This is a great project, BTW - really cool stuff.
-Matt
The AWS sdk uses the apache commons-codec, version 1.3. I've been in
communication with Dennis, and he sent me a hacked version of the
Apache library that has the escape issue 'corrected', this fixed my
authentication problem.
I've also posted about executing multi-condition 'select' statements -
this seems to be problematic, at least with the AWS sdk. I've snooped
the traffic and posted the exact URL; hopefully we can find some
differences between the way AWS encodes a request, and the way other
working toolkits encode a request.
The AWS sdk is open source, and ports exist for Java, Python, PHP, and
Ruby. I'm digging in this stuff now, and would be happy to work with
you to troubleshoot any incompatibilities. I'm happy point my Java
requests at one of your M/DB instances, or do whatever else would be
useful.
This is a great project, BTW - really cool stuff.
-Matt
rtweed
Apr 22, 2011, 4:01:15 AM4/22/11
to M/DB Community Forum
With Matt's assistance I've now fixed this +/space bug that has
affected the AWS Java client. Build 39 of M/DB is available on
Github. See separate posting on how to upgrade.
Rob
affected the AWS Java client. Build 39 of M/DB is available on
Github. See separate posting on how to upgrade.
Rob
Reply all
Reply to author
Forward
0 new messages