Groups keyboard shortcuts have been updated
Dismiss
See shortcuts
M/DB list values
15 views
Skip to first unread message
project2501
Apr 22, 2011, 7:39:48 AM4/22/11
to M/DB Community Forum
Hi,
I use boto for simpledb and M/DB access. In simpledb you can have
multiple values for the same attribute.
boto will translate to and from a list of values in this case. It
works on simpledb, but on M/DB it returns only 1 item from the list.
It makes me wonder if M/DB supports multiple values for the SAME
attribute or not.
Example (again, using boto)
SimpleDB result for item:
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1'}
key3 has multiple values.
M/DB result for item:
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1'}
key3 has only one value when it should have 3.
It should work the same I believe.
Darren
I use boto for simpledb and M/DB access. In simpledb you can have
multiple values for the same attribute.
boto will translate to and from a list of values in this case. It
works on simpledb, but on M/DB it returns only 1 item from the list.
It makes me wonder if M/DB supports multiple values for the SAME
attribute or not.
Example (again, using boto)
SimpleDB result for item:
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1'}
key3 has multiple values.
M/DB result for item:
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1'}
key3 has only one value when it should have 3.
It should work the same I believe.
Darren
rtweed
Apr 22, 2011, 7:43:58 AM4/22/11
to M/DB Community Forum
Interesting. M/DB *should* behave identically and it's been designed
to support multiple values per attribute. Now I understand the issue
I'll check it out and see what the problem is
Rob
to support multiple values per attribute. Now I understand the issue
I'll check it out and see what the problem is
Rob
rtweed
Apr 22, 2011, 11:21:16 AM4/22/11
to M/DB Community Forum
OK I've tested my M/DB reference system and can't find a problem.
I first used the Bolso client tool to create a domain named "test",
then created an item named "item1", with two attributes named "attr1"
and "attr2". "attr1" was given 3 values: "value1", "value2" and
"value3", while "attr2" was given a single value of "value3"
This data showed up just fine in Bolso. I then tried the boto Python
client as follows:
import boto
import boto.sdb
import boto.sdb.regioninfo
mdbRegion =
boto.sdb.regioninfo.SDBRegionInfo(name='mdb',endpoint='127.0.0.1')
mdb=boto.connect_sdb('myusername','myprivatekey',isSecure=False,region=mdbRegion,path='/
mdb/request.mgwsi')
query="SELECT * FROM test"
results=mdb.select("test",query)
print results
[{u'attr2': u'value3', u'attr1': [u'value1', u'value2', u'value3']}]
...which is what you'd expect in SimpleDB also, right?
Can you provide some more details on exactly what you're doing which
just returns a single value? Are you using something other than
Select, or perhaps a different type of query?
Thanks
Rob
I first used the Bolso client tool to create a domain named "test",
then created an item named "item1", with two attributes named "attr1"
and "attr2". "attr1" was given 3 values: "value1", "value2" and
"value3", while "attr2" was given a single value of "value3"
This data showed up just fine in Bolso. I then tried the boto Python
client as follows:
import boto
import boto.sdb
import boto.sdb.regioninfo
mdbRegion =
boto.sdb.regioninfo.SDBRegionInfo(name='mdb',endpoint='127.0.0.1')
mdb=boto.connect_sdb('myusername','myprivatekey',isSecure=False,region=mdbRegion,path='/
mdb/request.mgwsi')
query="SELECT * FROM test"
results=mdb.select("test",query)
print results
[{u'attr2': u'value3', u'attr1': [u'value1', u'value2', u'value3']}]
...which is what you'd expect in SimpleDB also, right?
Can you provide some more details on exactly what you're doing which
just returns a single value? Are you using something other than
Select, or perhaps a different type of query?
Thanks
Rob
project2501
Apr 22, 2011, 9:11:49 PM4/22/11
to M/DB Community Forum
Rob,
Thank you for looking into this. Here is my boto program.
----
import boto
from boto.sdb import *
from boto.sdb.regioninfo import *
from boto.sdb.connection import *
r = RegionInfo(name='mdb',endpoint='192.168.0.110')
conn =
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
for item in domain:
print item.name
#item = domain.new_item('item1')
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item)
#item.save()
retrievedItem = domain.get_item('item1')
keys = retrievedItem['key3']
for k in keys:
print ":",k
print retrievedItem
print retrievedItem['key3']
----
OUTPUT:
-----
: t
: h
: r
: e
: e
-----
I was expecting ['one', 'two', 'three'], not 'three'.
On Apr 22, 11:21 am, rtweed <rob.tw...@gmail.com> wrote:
> OK I've tested myM/DBreference system and can't find a problem.
>
> > > It makes me wonder ifM/DBsupports multiple values for the SAME
Thank you for looking into this. Here is my boto program.
----
import boto
from boto.sdb import *
from boto.sdb.regioninfo import *
from boto.sdb.connection import *
r = RegionInfo(name='mdb',endpoint='192.168.0.110')
conn =
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
for item in domain:
print item.name
#item = domain.new_item('item1')
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item)
#item.save()
retrievedItem = domain.get_item('item1')
keys = retrievedItem['key3']
for k in keys:
print ":",k
print retrievedItem
print retrievedItem['key3']
----
OUTPUT:
-----
: t
: h
: r
: e
: e
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1'}
three
-----
I was expecting ['one', 'two', 'three'], not 'three'.
On Apr 22, 11:21 am, rtweed <rob.tw...@gmail.com> wrote:
> OK I've tested myM/DBreference system and can't find a problem.
>
> I first used the Bolso client tool to create a domain named "test",
> then created an item named "item1", with two attributes named "attr1"
> and "attr2". "attr1" was given 3 values: "value1", "value2" and
> "value3", while "attr2" was given a single value of "value3"
>
> This data showed up just fine in Bolso. I then tried the boto Python
> client as follows:
>
> import boto
> import boto.sdb
> import boto.sdb.regioninfo
> mdbRegion =
> boto.sdb.regioninfo.SDBRegionInfo(name='mdb',endpoint='127.0.0.1')
> mdb=boto.connect_sdb('myusername','myprivatekey',isSecure=False,region=mdbRegion,path='/
> mdb/request.mgwsi')
>
> query="SELECT * FROM test"
> results=mdb.select("test",query)
> print results
>
> [{u'attr2': u'value3', u'attr1': [u'value1', u'value2', u'value3']}]
>
> ...which is what you'd expect in SimpleDB also, right?
>
> Can you provide some more details on exactly what you're doing which
> just returns a single value? Are you using something other than
> Select, or perhaps a different type of query?
>
> Thanks
>
> Rob
>
> On Apr 22, 12:43 pm, rtweed <rob.tw...@gmail.com> wrote:
>
>
>
>
>
>
>
> > Interesting.M/DB*should* behave identically and it's been designed
> I first used the Bolso client tool to create a domain named "test",
> then created an item named "item1", with two attributes named "attr1"
> and "attr2". "attr1" was given 3 values: "value1", "value2" and
> "value3", while "attr2" was given a single value of "value3"
>
> This data showed up just fine in Bolso. I then tried the boto Python
> client as follows:
>
> import boto
> import boto.sdb
> import boto.sdb.regioninfo
> mdbRegion =
> boto.sdb.regioninfo.SDBRegionInfo(name='mdb',endpoint='127.0.0.1')
> mdb=boto.connect_sdb('myusername','myprivatekey',isSecure=False,region=mdbRegion,path='/
> mdb/request.mgwsi')
>
> query="SELECT * FROM test"
> results=mdb.select("test",query)
> print results
>
> [{u'attr2': u'value3', u'attr1': [u'value1', u'value2', u'value3']}]
>
> ...which is what you'd expect in SimpleDB also, right?
>
> Can you provide some more details on exactly what you're doing which
> just returns a single value? Are you using something other than
> Select, or perhaps a different type of query?
>
> Thanks
>
> Rob
>
> On Apr 22, 12:43 pm, rtweed <rob.tw...@gmail.com> wrote:
>
>
>
>
>
>
>
> > to support multiple values per attribute. Now I understand the issue
> > I'll check it out and see what the problem is
>
> > Rob
>
> > On Apr 22, 12:39 pm, project2501 <darreng5...@gmail.com> wrote:
>
> > > Hi,
> > > I use boto for simpledb andM/DBaccess. In simpledb you can have
> > I'll check it out and see what the problem is
>
> > Rob
>
> > On Apr 22, 12:39 pm, project2501 <darreng5...@gmail.com> wrote:
>
> > > Hi,
> > > multiple values for the same attribute.
> > > boto will translate to and from a list of values in this case. It
> > > works on simpledb, but onM/DBit returns only 1 item from the list.
> > > boto will translate to and from a list of values in this case. It
>
> > > It makes me wonder ifM/DBsupports multiple values for the SAME
> > > attribute or not.
>
> > > Example (again, using boto)
>
> > > SimpleDB result for item:
>
> > > {u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
> > > u'value1'}
>
> > > key3 has multiple values.
>
> > >M/DBresult for item:
>
> > > Example (again, using boto)
>
> > > SimpleDB result for item:
>
> > > {u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
> > > u'value1'}
>
> > > key3 has multiple values.
>
rtweed
Apr 23, 2011, 10:08:29 AM4/23/11
to M/DB Community Forum
OK I've run your test and yes, sure enough, only one value (the last)
is saved for key3. When you inspect the raw HTTP request that is
being sent by boto, the critical part looks like this:
&Attribute.4.Name=key3
&Attribute.4.Replace=true
&Attribute.4.Value=one
&Attribute.5.Name=key3
&Attribute.5.Replace=true
&Attribute.5.Value=two
&Attribute.6.Name=key3
&Attribute.6.Replace=true
&Attribute.6.Value=three
Note that Replace=true. What this tells SimpleDB (and hence M/DB) is
to replace the value of key3, not append a new one. This is according
to the SimpleDB documentation and spec. Can you confirm what that
same test does against SimpleDB? It *should* replace the values, which
is why key3 ends up just holding the value 'three'.
I don't know why boto is setting Replace=true in the request it's
generating - looks like a bug in boto to me!
BTW you can inspect this for yourself if you put M/DB into "trace
mode". To do this, go into the GT.M shell:
cd /usr/local/gtm/ewd
$gtm (yes, add the $ at the beginning!)
GTM>
Now do the following:
set ^zewd("trace")=1
kill ^%zewdTrace
That sets trace mode on and clears down the trace global.
Now run your boto commands, then to inspect what happened, in the GT,M
shell:
for i=1:1 w ^%zewdTrace(i),!
You'll get a listing of what M/DB received over the wire and how it
handled it.
WHen you're done, turn off trace mode again:
set ^zewd("trace")=0
kill ^%zewdTrace
halt
Halt returns you to the Linux shell
Anyway, let me know what you find, but as far as I can see, M/DB is
only doing exactly what it's being told to do by boto!
Rob
Rob
> > mdb=boto.connect_sdb('myusername','myprivatekey',isSecure=False,region=mdbR egion,path='/
is saved for key3. When you inspect the raw HTTP request that is
being sent by boto, the critical part looks like this:
&Attribute.4.Name=key3
&Attribute.4.Replace=true
&Attribute.4.Value=one
&Attribute.5.Name=key3
&Attribute.5.Replace=true
&Attribute.5.Value=two
&Attribute.6.Name=key3
&Attribute.6.Replace=true
&Attribute.6.Value=three
Note that Replace=true. What this tells SimpleDB (and hence M/DB) is
to replace the value of key3, not append a new one. This is according
to the SimpleDB documentation and spec. Can you confirm what that
same test does against SimpleDB? It *should* replace the values, which
is why key3 ends up just holding the value 'three'.
I don't know why boto is setting Replace=true in the request it's
generating - looks like a bug in boto to me!
BTW you can inspect this for yourself if you put M/DB into "trace
mode". To do this, go into the GT.M shell:
cd /usr/local/gtm/ewd
$gtm (yes, add the $ at the beginning!)
GTM>
Now do the following:
set ^zewd("trace")=1
kill ^%zewdTrace
That sets trace mode on and clears down the trace global.
Now run your boto commands, then to inspect what happened, in the GT,M
shell:
for i=1:1 w ^%zewdTrace(i),!
You'll get a listing of what M/DB received over the wire and how it
handled it.
WHen you're done, turn off trace mode again:
set ^zewd("trace")=0
kill ^%zewdTrace
halt
Halt returns you to the Linux shell
Anyway, let me know what you find, but as far as I can see, M/DB is
only doing exactly what it's being told to do by boto!
Rob
Rob
project2501
Apr 24, 2011, 2:44:03 PM4/24/11
to M/DB Community Forum
Rob,
You are awesome. I will try setting replace=False in boto and I
agree it might be a bug.
I will notify Mitch. Oddly enough, it returns the full list when run
against SimpleDB....
I'll report my result after setting the flag and re-trying.
Darren
You are awesome. I will try setting replace=False in boto and I
agree it might be a bug.
I will notify Mitch. Oddly enough, it returns the full list when run
against SimpleDB....
I'll report my result after setting the flag and re-trying.
Darren
project2501
Apr 24, 2011, 2:56:34 PM4/24/11
to M/DB Community Forum
Ok, some more info. Not sure what to make of it though.
Using SimpleDB with replace=True, an attribute is replaced with
the new value, EVEN if it is a list. Leaving the attribute with the
NEW list value.
On M/DB, the behavior is different. With replace=True, only the last
list item
is left in the attribute value.
domain.put_attributes('item1',item,replace=False)
Setting replace=False, causes the attribute to grow with new list
values,
adding to it. Both SimpleDB and M/DB behave the same with
replace=False.
But its insufficient for the use case of replacing the attribute with
a NEW list
of values because it will keep adding to it.
thoughts?
Darren
Using SimpleDB with replace=True, an attribute is replaced with
the new value, EVEN if it is a list. Leaving the attribute with the
NEW list value.
On M/DB, the behavior is different. With replace=True, only the last
list item
is left in the attribute value.
domain.put_attributes('item1',item,replace=False)
Setting replace=False, causes the attribute to grow with new list
values,
adding to it. Both SimpleDB and M/DB behave the same with
replace=False.
But its insufficient for the use case of replacing the attribute with
a NEW list
of values because it will keep adding to it.
thoughts?
Darren
rtweed
Apr 26, 2011, 12:16:34 PM4/26/11
to M/DB Community Forum
Hmmm Sounds like an ambiguity in Amazon's SimpleDB documentation for
PutAttributes. Here's what it says:
"Optionally, the requester can supply the Replace parameter for each
individual attribute. Setting this value to true causes the new
attribute value to replace the existing attribute value(s). For
example, if an item has the attributes { 'a', '1' }, { 'b', '2'} and
{ 'b', '3' } and the requester calls PutAttributes using the
attributes { 'b', '4' } with the Replace parameter set to true, the
final attributes of the item are changed to { 'a', '1' } and { 'b',
'4' }, which replaces the previous values of the 'b' attribute with
the new value."
To me, that says that M/DB is doing what SimpleDB is supposed to do,
but clearly what SimpleDB is doing is more subtle. What does Mitch
make of it, Darren? The bottom line is that M/DB should do exactly
as SimpleDB, so we need to clarify the rules for Replace!
Cheers
Rob
PutAttributes. Here's what it says:
"Optionally, the requester can supply the Replace parameter for each
individual attribute. Setting this value to true causes the new
attribute value to replace the existing attribute value(s). For
example, if an item has the attributes { 'a', '1' }, { 'b', '2'} and
{ 'b', '3' } and the requester calls PutAttributes using the
attributes { 'b', '4' } with the Replace parameter set to true, the
final attributes of the item are changed to { 'a', '1' } and { 'b',
'4' }, which replaces the previous values of the 'b' attribute with
the new value."
To me, that says that M/DB is doing what SimpleDB is supposed to do,
but clearly what SimpleDB is doing is more subtle. What does Mitch
make of it, Darren? The bottom line is that M/DB should do exactly
as SimpleDB, so we need to clarify the rules for Replace!
Cheers
Rob
Rob Tweed
Apr 26, 2011, 12:21:27 PM4/26/11
to M/DB Community Forum
I guess what it all means is that the Replace=true refers to values that are already in the item, not any new ones in the incoming request
--
Rob Tweed
Director, M/Gateway Developments Ltd
http://www.mgateway.com
------------------
iWD: build iPhone Apps faster :
http://www.mgateway.com/iwd.html
Rob
--
You received this message because you are subscribed to the Google Groups "M/DB Community Forum" group.
To post to this group, send an email to mdb-commu...@googlegroups.com.
To unsubscribe from this group, send email to mdb-community-f...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mdb-community-forum?hl=en-GB.
--
Rob Tweed
Director, M/Gateway Developments Ltd
http://www.mgateway.com
------------------
iWD: build iPhone Apps faster :
http://www.mgateway.com/iwd.html
project2501
Apr 26, 2011, 2:16:31 PM4/26/11
to M/DB Community Forum
Interesting. I will take this over to the Mitch/boto and see if he's
doing any "interpretation" there (or maybe a bug).
I'll report back.
On Apr 26, 12:21 pm, Rob Tweed <rob.tw...@gmail.com> wrote:
> I guess what it all means is that the Replace=true refers to values that are
> already in the item, not any new ones in the incoming request
>
> Rob
>
> Director, M/Gateway Developments Ltdhttp://www.mgateway.com
doing any "interpretation" there (or maybe a bug).
I'll report back.
On Apr 26, 12:21 pm, Rob Tweed <rob.tw...@gmail.com> wrote:
> I guess what it all means is that the Replace=true refers to values that are
> already in the item, not any new ones in the incoming request
>
> Rob
>
project2501
Apr 26, 2011, 2:24:01 PM4/26/11
to M/DB Community Forum
My quick take on the above reference documentation is that perhaps the
semantics of Replace=True are different
depending whether SimpleDB receives a "list" of values vs. a single
value for the "same" attribute.
It would be a documentation bug in that case, but it seems SimpleDB
DOES change its behavior in this case.
Once I verify this behavior is correct, may I suggest M/DB adapt to
SimpleDB's behavior? Othewise, it will require two calls
to set/update list values in M/DB, degrading performance.
v/r,
Darren
> Director, M/Gateway Developments Ltdhttp://www.mgateway.com
semantics of Replace=True are different
depending whether SimpleDB receives a "list" of values vs. a single
value for the "same" attribute.
It would be a documentation bug in that case, but it seems SimpleDB
DOES change its behavior in this case.
Once I verify this behavior is correct, may I suggest M/DB adapt to
SimpleDB's behavior? Othewise, it will require two calls
to set/update list values in M/DB, degrading performance.
v/r,
Darren
> I guess what it all means is that the Replace=true refers to values that are
> already in the item, not any new ones in the incoming request
>
> Rob
>
> already in the item, not any new ones in the incoming request
>
> Rob
>
Rob Tweed
Apr 26, 2011, 2:41:32 PM4/26/11
to mdb-commu...@googlegroups.com
Darren
Yes of course - once I understand the rules that SimpleDB is using for the Replace, I'll make M/DB use those same rules. The whole idea of M/DB is for it to behave identically to SimpleDB so that it's "plug-compatible". Clearly the rules are more subtle than the documentation implies.
Perhaps you could try some experiments with SimpleDB along the lines of what happens if (using standard boto):
- an attribute has a single value and is replaced with a list
- an attribute does not exist and is replaced with a list
- an attribute holds a list of values and is replaced with a single value
- an attribute holds a list and is replaced with another list
I think those are all possible permutations and should clarify the rules that Amazon are applying
Cheers
Rob
project2501
Apr 27, 2011, 8:35:10 AM4/27/11
to M/DB Community Forum
Here is Chris Moyer's response[1] from the boto team:
"The SimpleDB implementation is the correct one. When it's updating a
list with replace=True, it should wipe out the old values and replace
them with ALL of the new values, not just the last one. Not sure what
the difference is that M/DB has but I don't know of any special
handling case that we do for SDB that would be different for M/DB"
[1] http://groups.google.com/group/boto-users/browse_thread/thread/aa73ee6b4b7bbf53
v/r
Darren
On Apr 26, 2:41 pm, Rob Tweed <rob.tw...@gmail.com> wrote:
> Darren
>
> Yes of course - once I understand the rules that SimpleDB is using for the
> Replace, I'll make M/DB use those same rules. The whole idea of M/DB is for
> it to behave identically to SimpleDB so that it's "plug-compatible".
> Clearly the rules are more subtle than the documentation implies.
>
> Perhaps you could try some experiments with SimpleDB along the lines of what
> happens if (using standard boto):
>
> - an attribute has a single value and is replaced with a list
> - an attribute does not exist and is replaced with a list
> - an attribute holds a list of values and is replaced with a single value
> - an attribute holds a list and is replaced with another list
>
> I think those are all possible permutations and should clarify the rules
> that Amazon are applying
>
> Cheers
>
> Rob
>
> ...
>
> read more »
"The SimpleDB implementation is the correct one. When it's updating a
list with replace=True, it should wipe out the old values and replace
them with ALL of the new values, not just the last one. Not sure what
the difference is that M/DB has but I don't know of any special
handling case that we do for SDB that would be different for M/DB"
[1] http://groups.google.com/group/boto-users/browse_thread/thread/aa73ee6b4b7bbf53
v/r
Darren
On Apr 26, 2:41 pm, Rob Tweed <rob.tw...@gmail.com> wrote:
> Darren
>
> Yes of course - once I understand the rules that SimpleDB is using for the
> Replace, I'll make M/DB use those same rules. The whole idea of M/DB is for
> it to behave identically to SimpleDB so that it's "plug-compatible".
> Clearly the rules are more subtle than the documentation implies.
>
> Perhaps you could try some experiments with SimpleDB along the lines of what
> happens if (using standard boto):
>
> - an attribute has a single value and is replaced with a list
> - an attribute does not exist and is replaced with a list
> - an attribute holds a list of values and is replaced with a single value
> - an attribute holds a list and is replaced with another list
>
> I think those are all possible permutations and should clarify the rules
> that Amazon are applying
>
> Cheers
>
> Rob
>
>
> read more »
rtweed
Apr 27, 2011, 8:43:02 AM4/27/11
to M/DB Community Forum
Darren
I agree - M/DB should always behave exactly like SimpleDB, not in a
different or special way. So yes, the SimpleDB implementation is
always, by definition, the correct one. The question is exactly how
should the Replace rule be implemented so it always behaves
identically? Hence my suggestion above - can you see how SimpleDB
behaves when using the standard boto client in all those possible
permutations? The documentation, is, to my mind, ambiguous, so I need
to clarify that ambiguity.
If you could find that out for me, I can fix M/DB accordingly
Cheers
Rob
On Apr 27, 1:35 pm, project2501 <darreng5...@gmail.com> wrote:
> Here is Chris Moyer's response[1] from the boto team:
>
> "The SimpleDB implementation is the correct one. When it's updating a
> list with replace=True, it should wipe out the old values and replace
> them with ALL of the new values, not just the last one. Not sure what
> the difference is that M/DB has but I don't know of any special
> handling case that we do for SDB that would be different for M/DB"
>
> [1]http://groups.google.com/group/boto-users/browse_thread/thread/aa73ee...
> ...
>
> read more »
I agree - M/DB should always behave exactly like SimpleDB, not in a
different or special way. So yes, the SimpleDB implementation is
always, by definition, the correct one. The question is exactly how
should the Replace rule be implemented so it always behaves
identically? Hence my suggestion above - can you see how SimpleDB
behaves when using the standard boto client in all those possible
permutations? The documentation, is, to my mind, ambiguous, so I need
to clarify that ambiguity.
If you could find that out for me, I can fix M/DB accordingly
Cheers
Rob
On Apr 27, 1:35 pm, project2501 <darreng5...@gmail.com> wrote:
> Here is Chris Moyer's response[1] from the boto team:
>
> "The SimpleDB implementation is the correct one. When it's updating a
> list with replace=True, it should wipe out the old values and replace
> them with ALL of the new values, not just the last one. Not sure what
> the difference is that M/DB has but I don't know of any special
> handling case that we do for SDB that would be different for M/DB"
>
>
> read more »
project2501
Apr 27, 2011, 8:44:07 AM4/27/11
to M/DB Community Forum
Will do! I'm on it.
> ...
>
> read more »
>
> read more »
project2501
May 1, 2011, 10:17:17 AM5/1/11
to M/DB Community Forum
Rob,
Sorry for the delay on this. Here is the docstring unit test
comparing your
suggested test cases between SimpleDB and M/DB, using boto. Let me
know what else I can do to help.
Darren
------------------------------------------------------------------------
"""
>>> sdb_single_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> mdb_single_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> sdb_no_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> mdb_no_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> sdb_list_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> mdb_list_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> sdb_list_value_replace_value()
{u'key3': u'value3', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
>>> mdb_list_value_replace_value()
{u'key3': u'value3', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
import time
def mdb_single_value_replace_list():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
time.sleep(3)
def sdb_single_value_replace_list():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
time.sleep(3)
def mdb_no_value_replace_list():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
time.sleep(3)
def sdb_no_value_replace_list():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
time.sleep(3)
def mdb_list_value_replace_list():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
time.sleep(3)
def sdb_list_value_replace_list():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
time.sleep(3)
def mdb_list_value_replace_value():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = 'value3'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
def sdb_list_value_replace_value():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = 'value3'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
def _test():
import doctest, mdb
return doctest.testmod(mdb)
if __name__ == "__main__":
_test()
> ...
>
> read more »
Sorry for the delay on this. Here is the docstring unit test
comparing your
suggested test cases between SimpleDB and M/DB, using boto. Let me
know what else I can do to help.
Darren
------------------------------------------------------------------------
"""
>>> sdb_single_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> mdb_single_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> sdb_no_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> mdb_no_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> sdb_list_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> mdb_list_value_replace_list()
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
>>> sdb_list_value_replace_value()
{u'key3': u'value3', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
>>> mdb_list_value_replace_value()
{u'key3': u'value3', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
"""
import boto
from boto.sdb import *
from boto.sdb.regioninfo import *
from boto.sdb.connection import *
from uuid import *
import boto
from boto.sdb import *
from boto.sdb.regioninfo import *
from boto.sdb.connection import *
import time
def mdb_single_value_replace_list():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
conn =
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
try:
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = 'value3'
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def sdb_single_value_replace_list():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain = conn.create_domain('domain')
try:
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = 'value3'
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def mdb_no_value_replace_list():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
conn =
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
try:
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
item['key1'] = 'value1'
item['key2'] = 'value2'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def sdb_no_value_replace_list():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain = conn.create_domain('domain')
try:
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
item['key1'] = 'value1'
item['key2'] = 'value2'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def mdb_list_value_replace_list():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
conn =
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
try:
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = ['one1', 'two1', 'three1']
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def sdb_list_value_replace_list():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain = conn.create_domain('domain')
try:
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = ['one1', 'two1', 'three1']
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = ['one', 'two', 'three']
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def mdb_list_value_replace_value():
r = RegionInfo(name='mdb',endpoint='192.168.56.103')
conn =
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
try:
SDBConnection('adminkey','adminsecret',region=r,is_secure=False,path='/
mdb/request.mgwsi')
domain = conn.create_domain('domain')
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = ['one1', 'two1', 'three1']
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = 'value3'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def sdb_list_value_replace_value():
conn = SDBConnection('YOURKEY','YOURSECRET')
domain = conn.create_domain('domain')
try:
domain.delete_attributes('item1')
time.sleep(3)
except:
pass
item = {}
item['key1'] = 'value1'
item['key2'] = 'value2'
item['key3'] = ['one1', 'two1', 'three1']
item['key1'] = 'value1'
item['key2'] = 'value2'
item['id'] = 'item1'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
item['key3'] = 'value3'
domain.put_attributes('item1',item,replace=True)
time.sleep(3)
retrievedItem = domain.get_item('item1')
print retrievedItem
def _test():
import doctest, mdb
return doctest.testmod(mdb)
if __name__ == "__main__":
_test()
>
> read more »
rtweed
May 1, 2011, 12:14:45 PM5/1/11
to M/DB Community Forum
I'm deep in development of a Node.js version of M/DB and don't have
the time to plough through the detail of your tests and the results.
It would really help if you could just sum up in words how each of the
scenarios I specified plays out.
Many thanks for your efforts! Greatly appreciated
Rob
> ...
>
> read more »
the time to plough through the detail of your tests and the results.
It would really help if you could just sum up in words how each of the
scenarios I specified plays out.
Many thanks for your efforts! Greatly appreciated
Rob
>
> read more »
project2501
May 1, 2011, 2:04:58 PM5/1/11
to M/DB Community Forum
Awesome news about Node.js again. Its amazing tech.
Here is the result of the unit test. If you just run "python test.py"
on the above file, it will print out all the test runs and expected
results automatically, btw.
Bottom line is that M/DB is treating multiple vales supplied
in a single call as separate attributes values processed
individually, when it should be interpreted as a single "list"
value and processed atomically.
hth,
Darren
darren@tungsten:~/cloud$ python mdb.py
**********************************************************************
File "/home/darren/cloud/mdb.py", line 4, in mdb
Failed example:
mdb_single_value_replace_list()
Expected:
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
**********************************************************************
File "/home/darren/cloud/mdb.py", line 8, in mdb
Failed example:
mdb_no_value_replace_list()
Expected:
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
**********************************************************************
File "/home/darren/cloud/mdb.py", line 12, in mdb
Failed example:
mdb_list_value_replace_list()
Expected:
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
**********************************************************************
1 items had failures:
3 of 8 in mdb
***Test Failed*** 3 failures.
> ...
>
> read more »
Here is the result of the unit test. If you just run "python test.py"
on the above file, it will print out all the test runs and expected
results automatically, btw.
Bottom line is that M/DB is treating multiple vales supplied
in a single call as separate attributes values processed
individually, when it should be interpreted as a single "list"
value and processed atomically.
hth,
Darren
darren@tungsten:~/cloud$ python mdb.py
**********************************************************************
File "/home/darren/cloud/mdb.py", line 4, in mdb
Failed example:
mdb_single_value_replace_list()
Expected:
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
Got:
u'value1', u'id': u'item1'}
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
**********************************************************************
File "/home/darren/cloud/mdb.py", line 8, in mdb
Failed example:
mdb_no_value_replace_list()
Expected:
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
Got:
u'value1', u'id': u'item1'}
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
**********************************************************************
File "/home/darren/cloud/mdb.py", line 12, in mdb
Failed example:
mdb_list_value_replace_list()
Expected:
{u'key3': [u'one', u'three', u'two'], u'key2': u'value2', u'key1':
u'value1', u'id': u'item1'}
Got:
u'value1', u'id': u'item1'}
{u'key3': u'three', u'key2': u'value2', u'key1': u'value1', u'id':
u'item1'}
**********************************************************************
1 items had failures:
3 of 8 in mdb
***Test Failed*** 3 failures.
>
> read more »
Reply all
Reply to author
Forward
0 new messages