Starting gcloud datastore emulator without storing db to disk

[Teemu Stenhammar]

Hi,

Hopefully this is correct place to ask this question.

Preface:

So, I am working with Node and gcloud sdk 0.27.0 on windows machine. My target was to enable integration/acceptance testing against locally emulated datastore. Everything else is running ok, but to ease my load a bit I tried to use '--store-on-disk=False' option so that all entries to datastore would be wiped out at the end. That option, however, seems not to work. I could not find any value that would result in anything else than '--store-on-disk=True' to be set in the end.

Issue:

I am no Python expert, but after debugging start scripts and reading SO I came to conclusion that Python takes command line arguments of boolean type as True if they are present, and hence, it does not matter what value is given, it will always be True.

Possible fix:

I fixed this issue by editing google-cloud-sdk/lib/surface/emulators/datastore/start.py so that '--store-on-disk' parameter defaults to False. Now I can give '--store-on-disk=True' to make it save locally and then just remove that option to prevent it from saving. It still behaves incorrectly in my opinion as it requires value. I was looking to solve it in a way that '--store-on-disk' would enable it and not having that option would disable.

So, is that option defaulting to True a bug, or is my Python magic insufficient to even provide script with a boolean value?

Br,

Teemu

[Patrick Costello]

Hi Teemu,

I'll need to dig a little more, but this definitely looks like a bug in the gcloud library.

In the meantime, you can always execute the gcd.sh tool directly.

This should look something like:

$GCLOUD_PATH/platform/gcd/gcd.sh start --testing projectDir

--testing is equivalent to the flags: --consistency=1.0 --store_on_disk=false --store_index_configuration_on_disk=false. It's usually what you'd want for running tests.

Thanks,

Patrick

--
You received this message because you are subscribed to the Google Groups "Google Cloud Datastore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gcd-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gcd-discuss/e3c9fb8b-89c2-4e34-be7c-2284a6dbc1b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Teemu Stenhammar]

Hi,

Thanks! I'll use that mean while then.

-Teemu

[Teemu Stenhammar]

Hi again,

I noticed that it seems to actually save the data on disk even with '--store_on_disk=False' and then later removes it somehow. At least when I test using grunt-mocha and any one test fails with the result of grunt-mocha aborting the process, then all data is stored as it was when abort happens. Only when all tests pass will datastore emulator actually clear any data I placed there.

Is there something I can do when running the tests?

-Teemu

[Patrick Costello]

Hi Teemu,

What data do you see getting stored on disk? If store_on_disk gets set when you start up the Datastore emulator via gcd.sh then it should not create the local db file.

-Patrick 

To view this discussion on the web visit https://groups.google.com/d/msgid/gcd-discuss/463bea0d-5f1e-41e8-a6ed-11a17e482ed8%40googlegroups.com.

[Teemu Stenhammar]

Hi,

Ok, so this might be silly from me, but what I am doing is that I have grunt running datastore emulator for continuous development mode with '--store_on_disk=True', and then tests that run the exactly same emulator with '--store_on_disk=False'.

So, when I am starting tests, it says:

    >> WARNING: Reusing existing data in [C:\<mypathhere>\AppData\Roaming\gcloud\emulators\datastore].

    >> Executing: cmd /c C:\<mypathhere>\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\gcd.cmd start --host=localhost --port=8181 --store_on_disk=False --consistency=1.0 --allow_remote_shutdown C:\<mypathhere>\AppData\Roaming\gcloud\emulators\datastore

Which then means, that it sees the data from development but tries to start emulator with '--store_on_disk=False'. This works like I was hoping it would when all tests pass, but like said, when grunt-mocha aborts on failure, it will keep the data in db.

-Teemu

[Patrick Costello]

(Removing the group from the thread)

When you run your test, do you restart the emulator for every run? Setting store_on_disk means that the local_db.bin won't (or at least shouldn't) get created at all. So you should never end up with any entity data on disk.

However, without specifying --testing or --store_index_configuration_on_disk, you'll still get an index.yaml file.

If you don't restart the emulator for each test, the data won't get cleared between runs. You can clear it by sending a post request to localhost:<port>/_ah/admin/datastore with the form encoded arguments "action":"Clear Datastore". (See the Java local service starter as an example).

-Patrick

To view this discussion on the web visit https://groups.google.com/d/msgid/gcd-discuss/49915cf4-fdd1-4cce-aa26-38ad45056192%40googlegroups.com.