Node.js version of MDB

[rtweed]

I've released an early build for anyone interested to try out. See
https://github.com/robtweed/node-mdb

As you'll see in the Readme document, I've not yet implemented all the
SimpleDB APIs, but most of the core ones are pretty much complete.

I'll be particularly interested in experiences relating to performance
and scalability. This version appears to be lightning fast and I
suspect it will scale really well.

Feedback welcomed!!

Enjoy!

Rob

[project2501]

Rob,
Just tried it out. Looks good!

When can we expect a new version? Will it support the NextToken? my
app relies heavily on that.

thanks for a great project.

Darren

On May 5, 10:31 am, rtweed <rob.tw...@gmail.com> wrote:
> I've released an early build for anyone interested to try out.  Seehttps://github.com/robtweed/node-mdb

[Rob Tweed]

Excellent!

NextToken will come eventually, though with node-mdb being a local database, IMO you really don't have the concern about using internet bandwidth to return large numbers of results.   It's also never been clear in any of the SDB specs (and answers to my questions in the SDB forums) just how NextToken works: ie does SimpleDB retain state for a resultset, and if so, for how long? ....or is the query rerun and the NextToken used to seed the start of the next set of results. 

The first is efficient, but requires a state-management to retain resultsets and a garbage collection mechanism to determine when to delete the cached results (but apparently this isn't how SimpleDB works).

The second is inefficient but a simple stateless system.

You can see how it might work serially for very simple queries, but for Select queries that are the result of the intersection or aggregation of several sub-query predicates, it becomes difficult to see how to implement NextToken without retaining a resultSet cache.

Anyway, the bottom line is that the NextToken is a non-trivial piece of functionality.

Rob

--
You received this message because you are subscribed to the Google Groups "M/DB Community Forum" group.
To post to this group, send an email to mdb-commu...@googlegroups.com.
To unsubscribe from this group, send email to mdb-community-f...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mdb-community-forum?hl=en-GB.

--
Rob Tweed
Director, M/Gateway Developments Ltd
http://www.mgateway.com
------------------
EWD Mobile: Build mobile applications faster
http://www.mgateway.com/ewd.html

[project2501]

I hear ya. Its definitely non-trivial. I will do some research with
the AWS folks to get some insight.

But in my application, it stores millions of records. The browser app
has paging built in, so they
see things like " Page 1/2500 of 1,255,000 results. To achieve that
it uses NextToken and limit
values. This makes the server query fast in most cases since it only
actually returns the page limit
to the browser and the middle tier querying SDB also, only gets a page
at a time.

Obviously, returning a million results from a select doesn't work in
any architecture.

Good question on how that state is maintained though. I suspect it has
to do with the result
calculus in their query engine caching offsets or something. Or the
nexttoken is a ciphered
instruction for skipping to a certain junction in the result set at
query time before returning
values. This way its done in the query engine before putting any
results on the wire..

On Jun 13, 12:51 pm, Rob Tweed <rob.tw...@gmail.com> wrote:
> Excellent!
>
> NextToken will come eventually, though with node-mdb being a local database,
> IMO you really don't have the concern about using internet bandwidth to
> return large numbers of results.   It's also never been clear in any of the
> SDB specs (and answers to my questions in the SDB forums) just how NextToken
> works: ie does SimpleDB retain state for a resultset, and if so, for how
> long? ....or is the query rerun and the NextToken used to seed the start of
> the next set of results.
>
> The first is efficient, but requires a state-management to retain resultsets
> and a garbage collection mechanism to determine when to delete the cached
> results (but apparently this isn't how SimpleDB works).
>
> The second is inefficient but a simple stateless system.
>
> You can see how it might work serially for very simple queries, but for
> Select queries that are the result of the intersection or aggregation of
> several sub-query predicates, it becomes difficult to see how to implement
> NextToken without retaining a resultSet cache.
>
> Anyway, the bottom line is that the NextToken is a non-trivial piece of
> functionality.
>
> Rob
>

> Director, M/Gateway Developments Ltdhttp://www.mgateway.com