SDK Upgrade 1.5.2: --datastore_path ignored?

[Cat]

Hi,

After today's upgrade my datastore is empty and my testdata is not
available.

I start my SDK Server like this (usually with the MacOS Launcher):
dev_appserver.py --datastore_path=/Users/cat/repositories/appengine/
my.datastore -p8080 .

Strangely --datastore_path is ignored and the dev_appserver uses a new
datastore in the tmp directory:
INFO 2011-07-22 07:11:11,606 rdbms_sqlite.py:58] Connecting to
SQLite database '' with file '/var/folders/u5/u5xmrm5gHPGXhfjlyv98u+++
+TI/-Tmp-/dev_appserver.rdbms'

Can anyone tell me which flags to set to get my old test datastore
back?

I would have to create lots of testdata manually that accumlated over
the last months.

Regards,
Cat

[Cat]

Update: (I'm going nuts)
I downgraded to 1.5.1 (restored GoogleAppEngineLauncher.app) and still
the same now ...
It seems that the upgrade to 1.5.2 did some permanent damage.
I am absolutely puzzled why --datastore_path= is ignored suddenly.

Here the full startup log:

cat$ /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
dev_appserver.py -d --datastore_path=/Users/cat/repositories/appengine/
my.datastore -p 8080 .
INFO 2011-07-22 09:15:00,330 appengine_rpc.py:159] Server:
appengine.google.com
INFO 2011-07-22 09:15:21,578 rdbms_sqlite.py:58] Connecting to

SQLite database '' with file '/var/folders/u5/u5xmrm5gHPGXhfjlyv98u+++
+TI/-Tmp-/dev_appserver.rdbms'

INFO 2011-07-22 09:15:21,618 dev_appserver_multiprocess.py:637]
Running application test on port 8080: http://localhost:8080

[Chris Copeland]

I'm having a problem that may be related.  I'm using django-nonrel which sets the datastore path to .gaedata in the project directory. 

When I start the server I get a log message that has the correct path:

INFO:root:Connecting to SQLite database '' with file '/Users/me/workspace/myapp/.gaedata/rdbms'

But when the app starts the datastore is empty.

-Chris

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Chris Copeland]

Cat, do you have backups of you datastore files from before you ran 1.5.2?

I was able to get running again by downgrading the SDK to 1.5.1 and restoring my data files from a backup.

-Chris

On Fri, Jul 22, 2011 at 4:19 AM, Cat <kat...@gmail.com> wrote:

[oakmad]

+1 for me. I have tried deleting the application plist as detailed in
this issue
http://code.google.com/p/googleappengine/issues/detail?id=1187&q=datastore_path&colspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log
but no joy.

[Cat]

I have tried to restore the datastore and the
GoogleAppEngineLauncher.app (with the SDK bundle inside) back to 1.5.0
but I cannot get it to run.
Its always the same error, as if the "downgrade" did not change the
python files.
Does anyone know how to get the SDK-version out of the running
appserver (a --version flag is missing!)?

On 22 Jul., 16:21, oakmad <adam.oak...@gmail.com> wrote:
> +1 for me. I have tried deleting the application plist as detailed in

> this issuehttp://code.google.com/p/googleappengine/issues/detail?id=1187&q=data...
> but no joy.

[c h]

hi all,

i *think* that it is honoring your datastore location (though the log message is incorrect), but the change to rename your application to dev~<your appname> in development has just rendered all of our test data useless.

after re-importing my test data it does look like it is stored where i ask it to be, but under the new application name.

if you are lucky enough to be using sqlite you can connect to the db and rename some tables to get it to work:

sqlite3 local_appname_dev_sqlite.datastore

sqlite> .tables

Apps                                          

IdSeq                                         

Namespaces                                    

appname!!Entities                        

appname!!EntitiesByProperty              

appname!namespace!Entities          

appname!namespace!EntitiesByProperty

sqlite> alter table `appname!!Entities` rename to `dev~appname!!Entities`;

sqlite> alter table `appname!!EntitiesByProperty` rename to `dev~appname!!EntitiesByProperty`;

sqlite> alter table `appname!namespace!Entities` rename to `dev~appname!namespace!Entities`;

sqlite> alter table `appname!namespace!EntitiesByProperty` rename to `dev~appname!namespace!EntitiesByProperty`;

where you substitute 'appname' for your application's name, and 'namespace' for your datanamespace.

cfh

[Matthew Blain]

That's a clever way to update the appid. I do not know if it works for
all cases (e.g. it may not work for all reference properties (stored
keys)) but is a neat trick.

Another way to deal with it is to use the --default_partition="" flag
rather than using an older version of the sdk.

--Matthew

[Chris Copeland]

Thanks, Matthew. 

I was able to update to 1.5.2 and use my existing datastore by adding that flag. 

It would have been useful if the release notes had mentioned that this would be necessary. 

-Chrsi

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

[c h]

yup the sqlite updates don't quite seem to work.

i second that the release notes should be updated, the flag does seem to work for me

cfh

[Joseph Letness]

I tried the --default_partition="" flag but now validation.py is
throwing an exception:

Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 4099, in _HandleRequest
default_partition)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/validation.py", line 360, in __setattr__
value = self.GetValidator(key)(value, key)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/validation.py", line 598, in __call__
return self.Validate(value, key)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/validation.py", line 923, in Validate
'\'%s\'' % (value, key, self.re.pattern))
ValidationError: Value '""~tagassetspro' for application does not
match expression '^(?:[a-z\d\-]{1,100}\~)?(?:(?!\-)[a-z\d\-\.]
{1,100}:)?(?!-)[a-z\d\-]{1,100}$'

This has got to be a bug. It's not like using stored test data with
the dev server is some kind of oddball edge case ;-)

--Joe

On Jul 22, 4:57 pm, Chris Copeland <ch...@cope360.com> wrote:
> Thanks, Matthew.
>
> I was able to update to 1.5.2 and use my existing datastore by adding that
> flag.
>
> It would have been useful if the release notes had mentioned that this would
> be necessary.
>
> -Chrsi
>

[Cat]

One thing I know for sure now is that the following log message is
bogus, it appears even if dev_appserver.py successfully connects and
uses a store at a different location than the TMP directory.
INFO 2011-07-23 07:22:11,762 rdbms_sqlite.py:58] Connecting to

[Cat]

================
SOLUTION
================

1. USE --default_partition= BUT DO NOT INCLUDE THE QUOTES as mentioned
in Matthew's post.
2. IGNORE THE INCORRECT LOG MESSAGE ... rdbms_sqlite.py:58] Connecting
to SQLite database ...

LauncherFlags:
--datastore_path=/Users/cat/repositories/appengine/my.datastore --
default_partition=

Console Flags:
dev_appserver.py --datastore_path=/Users/cat/repositories/appengine/
my.datastore --default_partition= -p 8080 .

That's it.

[Joseph Letness]

That did the trick, Cat. Thanks!

[WallyDD]

Many thanks Cat.

Just adding --default_partition= to the launch settings worked for me.

Using the Google App Engine Launcher;
Edit, Application settings.
In the box "Launch Settings" add --default_partition=

[Robert Kluin]

The file referred to in that log message is used for the experimental
SQL database support, not for the datastore stub.

Robert

[dop]

Very nice move, Google team, very nice... Keeping developers busy with
really exciting stuff (unlike the actual developing), eh?

Joking aside, I was able to see the old data with the --default-
partition option, but because I run django-nonrel it's a bit tricky to
add this option support to every management command, so I'd prefer to
run without it. Now, the question is how do I backup the local
datastore and restore it into the new configuration?

[dop]

I'm just amazed how little noise is there because of this issue. If I
understand it correctly after upgrading to 1.5.2 evenybody's local
datastore becomes empty. Yet very few seem to care. Weird...

[Robert Kluin]

Because they searched the groups and passed the default_partion argument to the dev appserver.