In memory datastore for unit testing instead of aetest and dev_appserver.py.

[Jon]

Hi All,

The development SDK/aetest has been quite problematic for me. It is slow, crashes sporadically leaving orphan/zombie processes and usually produces 'Timeout starting child process' errors. For integration tests I often have to insert time.Sleep into the test code to try and reduce the frequency of dev_appserver.py crashes.

Anyway, the SDK drove me crazy enough that I thought I would create my own pure Go memory based datastore to speed up my tests. I know of one pure Go implementation of the datastore backend, https://github.com/luci/gae, already in existance which also has tonnes of amazing extra functionality as well. I wanted to try and create my own.

I realise my code, which is currently proof of concept, might be useful to others in the future so I am posting it here: https://godoc.org/github.com/qedus/appengine/datastore.

The API basically takes the official Go datastore API and puts it in an interface so implementations are easier to swap in and out - such as a memory based implementation: https://godoc.org/github.com/qedus/appengine/datastore/memds. It also tries to make the use of namespaces a bit more orthogonal to the other datastore concepts.

The code is in its early stages and works well for me with just Get, Put & Delete methods - querying is not so good. However I thought I would ask for constructive criticism of the API to help me make it as simple to use as possible. It's easy to create a beautiful looking API only to find it painful in practice; which is why I stuck so closely to the tried and tested official API.

There are also quite a few production datastore quirks that the dev_appserver.py matches. I would like to replicated these with my code as much as possible so feel free to raise an issue if you know of a quirk that might not have been thought about.

I also tried using Dave Cheney's advice about returning errors that can be 'asserted for behaviour' instead of using the usual appengine.MultiError & datastore.ErrNoSuchEntity when fetching data using Get. It doesn't sit well with me though. Any ideas?

Jon

[Erik Troan]

I've done the same thing

   https://github.com/pendo-io/appwrap

covers a little more than the datastore fwiw.

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jon]

Interesting to see the different approaches in your project and the Chromium one. How come godoc.org only shows your logging code? Something to do with your build flags?

For me at least it was worth the effort building a pure Go test datastore just to have my project's unit tests consistently run in a couple of seconds instead of minutes.

I would be grateful for anyone with API expertise to comment on my abstraction to see how leaky it might be and how it could be improved.