After Migrate to HR I get exception "app s~XXX cannot access app YYY's data"

[Uri Lukach]

Hello,

I followed the instructions "Copying Entities to Another Application" and "A Note for Java Developers"  to migrate to HR application.

Everything went perfectly good!

While conducting some testing to verify the migration process went well, I got some strange exceptions in the form of:

IllegalArgumentException: app s~XXX cannot access app YYY's data 

Were XXX is the  HR application and YYY is the master slave one.

Did you encounter such error?

Thanks.

[Bert]

Hi,

Do you have some hardcoded string encoded keys used in parts of your
application?
Generally you get that error when trying to use a key that belongs to
another application.

Thanks
Rob

On Jul 3, 1:18 pm, Uri Lukach <uluk...@gmail.com> wrote:
> Hello,
>

> I followed the instructions "*Copying Entities to Another Application*" and
> *"A Note for Java Developers"*  to migrate to HR application.

> Everything went perfectly good!
>
> While conducting some testing to verify the migration process went well, I
> got some strange exceptions in the form of:
>

> *IllegalArgumentException: app s~XXX cannot access app YYY's data *

[Uri Lukach]

Hello Bert,

While following Google App Engine instructions in "Copying Entities to Another Application", I used datastore_admin to copy the all entities to the HR application.

When using datastore_admin the copy operation keeps the same entity keys. 

I think that if the entities keys are the same  there should not be any issue with using the wrong keys, in addition I don't have hard coded keys in my application code.

Is there another reason for this error?

Thanks,

Uri

[Bert]

Hi,

Can you paste the stack trace from the logs when you get this error?

I've used datastore_admin to copy data to HR. I'm open to correction
but I think it does create new keys, but the reference properties all
maintain the correct references. Places where I've seen the problem
you have is when I've stored a string encoded version of a key in a
property and have tried to get that string encoded key in the new
application.

Thanks
Rob

On Jul 4, 12:24 am, Uri Lukach <uluk...@gmail.com> wrote:
> Hello Bert,
>

> While following Google App Engine instructions in "*Copying Entities to
> Another Application"*, I used *datastore_admin* to copy the all entities to
> the HR application.
> When using *datastore_admin *the copy operation keeps the same entity keys.

[Uri Lukach]

Hey Rob,

This is the stack trace I am getting (I removed the appID and replaced it with XXX for the HR application and YYY for the master / slave)

NestedThrowablesStackTrace:
java.lang.IllegalArgumentException: app s~XXX cannot access app YYY's data
	at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:36)
	at com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(DatastoreApiHelper.java:98)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:69)
	at com.google.appengine.api.datastore.FutureHelper$CumulativeAggregateFuture.get(FutureHelper.java:136)
	at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:71)
	at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:32)
	at com.google.appengine.api.datastore.DatastoreServiceImpl$1.runInternal(DatastoreServiceImpl.java:78)
	at com.google.appengine.api.datastore.DatastoreServiceImpl$1.runInternal(DatastoreServiceImpl.java:75)
	at com.google.appengine.api.datastore.TransactionRunner.runInTransaction(TransactionRunner.java:31)
	at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java:75)
<continued in next message>

[Bert]

Hi,
Can you include the part of the stack trace which shows where in your
code the error got thrown from?
Thanks
Rob

On Jul 4, 9:35 am, Uri Lukach <uluk...@gmail.com> wrote:
> Hey Rob,
>
> This is the stack trace I am getting (I removed the appID and replaced it

> with *XXX* for the *HR application* and *YYY* for the *master / slave*)

>
> NestedThrowablesStackTrace:
> java.lang.IllegalArgumentException: app s~XXX cannot access app YYY's data
>         at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(Datast oreApiHelper.java:36)

>         at com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(Da tastoreApiHelper.java:98)

>         at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:69)
>         at com.google.appengine.api.datastore.FutureHelper$CumulativeAggregateFuture.g et(FutureHelper.java:136)

>         at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.ja va:71)

>         at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java: 32)
>         at com.google.appengine.api.datastore.DatastoreServiceImpl$1.runInternal(Datas toreServiceImpl.java:78)
>         at com.google.appengine.api.datastore.DatastoreServiceImpl$1.runInternal(Datas toreServiceImpl.java:75)
>         at com.google.appengine.api.datastore.TransactionRunner.runInTransaction(Trans actionRunner.java:31)

>         at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServic eImpl.java:75)
> <continued in next message>

[Uri Lukach]

Hey Bert,

It appears that you are correct.

The  datastore_admin  did not  preserved entity keys, and part of my entities have encoded keys to other entities for 1-1 or 1-Many references.

It seems that after the copying process the encoded keys have the same ending . So I mistakenly thought they were the same.

Is there away to preserve the entity keys? 

I am using encoded keys to maintain references in a lot of places... 

Thanks,

Uri

[Bert]

Hi Uri, Glad you found your issue.

I'm currently trying to convert my application too, so I've had
similar headaches.

In a few cases where I've used string encoded keys, I've been updating/
creating new entities to use key_name or id instead of string encoded
key. Then you can use get_by_key_name or get_by_id or key_from_path
methods to get the keys (and this will work on hrd and ms versions). I
think auto-generated ids are unique per entity group.

Have you been using ReferenceProperties to handle the relationships?
If so, I think those will be updated correctly (i.e. they'll use the
newly created object).

With the many-to-1 relationships, did you use lists of string encoded
keys? I think you can just use db.Key instead and those may be updated
automatically (not sure about this one though).

Good Luck,
Rob

On Jul 4, 10:08 am, Uri Lukach <uluk...@gmail.com> wrote:
> Hey Bert,
>
> It appears that you are correct.
>

> The  *datastore_admin  **did not** * preserved entity keys, and part of my

> entities have encoded keys to other entities for 1-1 or 1-Many references.
>

> It seems that after the copying process the encoded keys have the *same
> ending* . So I mistakenly thought they were the same.

[Uri Lukach]

Hey Rob,

I am using Java / JDO rather than python, although I have read again the documentation and I am confused,

According to the docs:

"Every entity has a key that is unique over all entities in App Engine. A complete key includes several pieces of information, including the application ID, the kind, and an entity ID."

So as I see, a key will always holds the application ID.... 

I guess that when you copy the Key objects in the entities using database-store it updates the application ID ...

I am not sure what to do next.... updating 80% of my data seems like a very challenging task

Thanks again for all your help,

Uri

[Noah McIlraith]

What I would do is iterate over every entity containing keys, and change each key's application id.

No idea how to do that on Java, sorry.

[Uri Lukach]

Hello Noah,

I have a big challenge here. Beyond the fact that I used encoded keys as references to form one to one/one to many and many to many relationships

I used them to save statistic data in data structured like Maps/Lists that I used extensively.

I read somewhere in the forum that Google App Engine team will make an effort in the future to support high - replication. I suppose I should wait for this to happen.

Thanks,

Uri

[Uri Lukach]

I meant that Google App Engine team will make an effort to make master /slave application support high - replication