sdb_con = boto.connect_sdb( credentials[0], credentials[1], region=sdbregion)
sdb_ptr = sdb_con.get_domain( storename)
for each connection object you want to use - you cannot get away with using just a single sdb_con...
>>> So my connection pooling class is created once (not per thread?) or uses classmethods?
Yes, it is neat to use a class to encapsulate all of this, but I'm sure there are other ways of doing it.
>>> and cherrypy kicks off a thread which will eventually call my page handling function
I really believe it is worth making a very tidy formal arrangement for accessing AWS via boto, as boto really doesn't do much by way of retries, and you will want to catch all sorts of low-level exceptions, log them and deal with them. For example, SDB has some real issues if you are trying to do a lot of writes in a short time. You pretty much *must* use batch_write, but there are limits - and the way it responds is to fail the request, and you will have to work out the exceptions that can be retried, and use a back-off and retry algorithm to manage them. All the same, there are real limits to the performance you can get out of SDB (it isn't boto's fault).
If your usage is light though, SDB is extremely flexible.
If your usage is likely to be heavy in terms of writes or deletes (maybe more than say 50 writes/sec), then you should forget SDB and go to DynamoDB (that is what we are up to right now...)
-----Original Message-----
From: cherrypy-users@googlegroups.com on behalf of Mark
Sent: Thu 03/05/2012 17:27
To: cherrypy-users
Subject: [cherrypy-users] Re: Using simpledb with boto
I'm asking on boto-users about connection pooling (thanks) but I'm
still asking here because I'm not sure about how threading works and
how to setup a class as you mentioned.
So I have a class say with a get_conn and release_conn function.
sdb = boto.connect_sdb(...)
creates a connection which I can then reuse if release_conn is called
by returning the same connection to the next get_conn call?
cherrypy is threaded right? I don't understand this. So my connection
pooling class is created once (not per thread?) or uses classmethods?
and cherrypy kicks off a thread which will eventually call my page
handling function:
@cherrypy.expose
def page(self, username):
conn = MyPool.get_conn()
...
MyPool.release(conn)
Now each thread will reuse a connection that has been previously
created and released...
Mark
On May 3, 11:46 am, "Alan Pound" <Alan.Po...@ACULAB.COM> wrote:
> What I do is, while starting up CP, I instantiate a class that builds a pool of SDB connection objects, and I pass a reference to that class to my CP classes.
> When they want to access SDB (or in a similar manner, S3), they obtain a connection object from that shared pool, and return it after use. If the pool gets empty, the class just makes another new one, and that goes back into the pool on completion.
> In fact, the class that maintains the pool, also supplies a bunch of helper functions that use those connection objects..
> Hope this helps.
> Alan
> -----Original Message-----
> From: cherrypy-users@googlegroups.com on behalf of Mark
> Sent: Thu 03/05/2012 16:35
> To: cherrypy-users
> Subject: [cherrypy-users] Using simpledb with boto
> Anyone using Amazon's SimpleDB with boto in CherryPy?
> I have to connect then get a domain:
> import boto
> sdb = boto.connect_sdb(...)
> dom = sdb.get_domain('test_users')
> Then in my page handlers I can look for a user 'shooter' with:
> user2 = dom.get_item('shooter')
> if user2 == None:
> print "no user shooter"
> But I'm not sure where to make the connect and get_domain calls and
> where to save the domain object. Can I just do:
> root.py:
> import boto
> sdb = boto.connect_sdb(...)
> dom = sdb.get_domain('test_users')
> class Root:
> @cherrypy.expose
> def page(self, username):
> user = dom.get_item(username)
> if user != None:
> ...
> --
> You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
> To post to this group, send email to cherrypy-users@googlegroups.com.
> To unsubscribe from this group, send email to cherrypy-users+unsubscribe@googlegroups.com.
> For more options, visit this group athttp://groups.google.com/group/cherrypy-users?hl=en.
> winmail.dat
> 4KViewDownload
--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-users@googlegroups.com.
To unsubscribe from this group, send email to cherrypy-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cherrypy-users?hl=en.
