In-memory datastore for unit-testing with Objectify v6 and cloud SDK?

[Matt Burns]

I foolishly tried to simultaneously upgrade from v5 to v6, and from old gradle plugin to cloud SDK.

All my unit tests were failing bizarrely with things like:

    @Test

    public void example_test() {

        List<SCFUser> users = ofy().load().type(SCFUser.class).list();

        createUser(...);

        assertEquals(1, users.size());

    }

java.lang.AssertionError: 

Expected :1

Actual   :22439

Each time I ran the test, the actual count was increasing. I was getting really confused until I realised it was authenticated against my LIVE DATASTORE and writing dummy data to that! GAH!

Luckily it was easy to tidy up but could have been a nightmare.

My question is, how should I be configuring unit tests with objectify v6? I was hoping I could use the in-memory datastore for fast, reproducible tests... I can't find any documentation anywhere for what I want to do.

[Fred Wulff]

The way we're doing it (Jeff might have a better practice): use the ObjectifyFactory constructor that takes a datastore instance. Build that instance with DatastoreOptions.Builder. Use builder.setHost(datastoreHost) to set the host/port from the gcloud datastore emulators run. You can configure the emulator to not save to disk, but there's not a good way to make it run in-memory in the test process so we have some scaffolding to clear out the local datastore between tests.

--
You received this message because you are subscribed to the Google Groups "objectify-appengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appen...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Burns]

Thanks for pointing me in the right direction. For anyone else that ends up here:

Start emulator

    gcloud beta emulators datastore start

Initialise ObjectifyService to point at the emulator

    DatastoreOptions options = DatastoreOptions.newBuilder()

            .setProjectId("dummy")

            .setHost("localhost:8081")

            .setCredentials(NoCredentials.getInstance())

            .setRetrySettings(ServiceOptions.getNoRetrySettings())

            .build();

    Datastore datastore = options.getService();

    ObjectifyService.init(new ObjectifyFactory(datastore));

    // register entities etc.

    closeable = ObjectifyService.begin();

Reset the database between tests

    curl -X POST http://localhost:8081/reset

Cheers,

Matt

To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appengine+unsub...@googlegroups.com.

[Matt Burns]

That should have been: gcloud beta emulators datastore start --no-store-on-disk

[Jeff Schnitzer]

Hey all, sorry I'm in crunch mode and just now catching up on email, PRs, issues, etc...

Take a look at Objectify's test suite. It uses a set of Junit5 extensions to setup/teardown a local test datastore using Google's new LocalDatastoreHelper class:

https://github.com/objectify/objectify/blob/master/src/test/java/com/googlecode/objectify/test/util/LocalDatastoreExtension.java

https://github.com/objectify/objectify/blob/master/src/test/java/com/googlecode/objectify/test/util/ObjectifyExtension.java

If you're using Junit5 (which I strongly recommend over Junit4), you can pretty much copy these extensions as-is. The LocalDatastoreHelper is pretty much doing exactly what your code is doing (start emulator, hit reset endpoint, shut it down) - it's just letting Google write the code. For some reason there doesn't seem to be any official documentation for this outside of the javadocs... don't feel bad about missing it, I had to ask someone at Google.

BTW if you'd like some other random unsolicited advice... drop the assertEquals() stuff and start using either Truth or AssertJ (they're nearly identical): assertThat(users).hasSize(1). Lots of great methods for working with collections. Here's a neat example you can do as a one-liner:

assertThat(users.stream().map(User::getName).map(String::toLowerCase).containsExactlyInAnyOrder("bob", "sally");

Jeff

To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appen...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "objectify-appengine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appen...@googlegroups.com.

[Jeff Schnitzer]

Ugh missed a paren:

assertThat(users.stream().map(User::getName).map(String::toLowerCase)).containsExactlyInAnyOrder("bob", "sally");

[Brian White]

Not sure what's going on. I've created similar classes locally and used @ExtendWith on my test base class to get them operational; the debugger shows everything being called. The LocalDatastoreHelper is being started but the helper.reset() call fails with "Connection refused: connect". Perhaps because I removed the Memcache code?

[Brian White]

Ah...  I needed a setPort(...) call when creating the LocalDatastoreHelper.

[Jeff Schnitzer]

Glad you figured it out.

Cheers,

Jeff

To view this discussion on the web visit https://groups.google.com/d/msgid/objectify-appengine/5f12544c-9523-44b2-b974-865e4f0d306dn%40googlegroups.com.

[Brian White]

For the record if someone else ends up here...  I needed that call because I was creating a connection to a local "gcloud" datastore instance and the helper seemed to be trying to wrap it.  I eventually removed the setup for that and then no longer needed setPort().  It doesn't seem like it's making any changes to a cloud database and I'd expect an exception for trying to access /reset if it did.  Thank goodness for being able to step into library code; I wouldn't have figured it out otherwise.

It would be nice if there was an official ObjectifyTestBase class that could be extended that did all this.