Recommended database
Rafael SDM Sierra
it, and it's was running pretty good so far, but I have a question.
What is the recommended database for an application that need
transactions (we are talking about run_in_transaction(), right?)? BDB
or MySQL? Does BDB really need Java, it used to be a C API...
I'm trying to fetch some data using MySQL from a set of ~14k records
(in a model with only keys, float, timestamp and boolean), but it is
so slow that nginx timeout waiting for the reply.
I would love to use mongodb, but transactions are required =/
Tobias
First and foremost, thanks for giving TyphoonAE a try! Transaction
support for the Datastore MongoDB API proxy stub is on our roadmap.
Currently, snapshot isolation and transaction serialization are
missing. I'm not certain if I find the time to finish that in the near
future, though. You're very welcome to help on this :)
The BDBDatastore backend uses the Berkeley DB Java Edition. Since it
has not been updated for quite a while, it lacks support for query
cursors and a couple of more recent Datastore API enhancements. Have
you already tested the BDBDatastore with your data set?
Could you provide a sample model and a query for your set of records?
I'd also like to investigate whether there is a way to improve the
performance of our MySQL Datastore stub.
Kind regards,
Tobias
Rafael Sierra
On Thu, Nov 11, 2010 at 8:15 PM, Tobias <tobias....@googlemail.com> wrote:
> Hi Rafael,
>
> First and foremost, thanks for giving TyphoonAE a try! Transaction
> support for the Datastore MongoDB API proxy stub is on our roadmap.
> Currently, snapshot isolation and transaction serialization are
> missing. I'm not certain if I find the time to finish that in the near
> future, though. You're very welcome to help on this :)
Yeah, I saw at mongodb that transactions don't even exists, so that
must be done by application level, and I guess it will be very
hard...the good news is that I needed transactions in gae just because
there is no unique index, but mongodb does :D
I don't know why, but took about 10 tries to import 14k records using
bulkload, seemed to be something allocating Ids (when I put a logging
to see how much ids it was trying to allocate, it worked)
>
> The BDBDatastore backend uses the Berkeley DB Java Edition. Since it
> has not been updated for quite a while, it lacks support for query
> cursors and a couple of more recent Datastore API enhancements. Have
> you already tested the BDBDatastore with your data set?
Yes, I tried, but I got some errors telling that something was
missing, and I was not in the mood to look what it was :P
>
> Could you provide a sample model and a query for your set of records?
> I'd also like to investigate whether there is a way to improve the
> performance of our MySQL Datastore stub.
The model is something like this:
class MyModel(BaseModel):
my_ref = db.ReferenceProperty(AnotherModel)
user = db.ReferenceProperty(User) # This is not the Google Account
user, it's the django user model
cents = db.FloatProperty() # Increased .01 by .01 and must be
unique with my_ref
date_utc = db.DateTimeProperty(auto_now_add=True, name='date')
is_true = db.BooleanProperty(default=False)
Then I got 8 AnotherModel instances and for each I get the last
MyModel (orderd by date_utc desc)
There is more than 500 MyModels for each AnotherModel (sometimes there
are thousands of them)
Anyway, I'm happy with MongoDB and it's unique index :)
Thankyou.
ps.: My app is running 100% with 0 modification, nice job guys, I hope
I can help with the project as soon as I can.
>
> Kind regards,
> Tobias
>
> On Nov 11, 3:53 am, Rafael SDM Sierra <rafaeljs...@gmail.com> wrote:
>> Hi, I just installed TyphoonAE today and I'm trying to run my app on
>> it, and it's was running pretty good so far, but I have a question.
>>
>> What is the recommended database for an application that need
>> transactions (we are talking about run_in_transaction(), right?)? BDB
>> or MySQL? Does BDB really need Java, it used to be a C API...
>>
>> I'm trying to fetch some data using MySQL from a set of ~14k records
>> (in a model with only keys, float, timestamp and boolean), but it is
>> so slow that nginx timeout waiting for the reply.
>>
>> I would love to use mongodb, but transactions are required =/
>
> --
> You received this message because you are subscribed to the Google
> Groups "typhoonae" group.
> To post to this group, send email to typh...@googlegroups.com
> To unsubscribe from this group, send email to
> typhoonae+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/typhoonae?hl=en
--
Rafael Sierra
http://blog.rafaelsdm.com