Migrating from Master/Slave to HRD key issue workaround

[Bert]

Hi,

In my application, running on master/slave, I have model called Total
where key names that are set to a combination of a string encoded key
and a descriptor.

For example, a key name could be set to "UserClicks-%s"%user.key().
So to get the total user clicks for a user, I run
Total.get_by_key_name( "UserClicks-%s"%user.key()).

The problem is that when moving from MS to HRD, the user.key() has
changed but the key name for the Total key is still the old value. So
totals are all wrong.

As a fix for this, should I run through Total table in Master/slave,
create new Totals with keyname = "UserClicks-
%s"%user.key().id_or_name(), and refactor the code where totals are
retrieved and set to use id_or_name instead of key?

Would id_or_name() always return a unique value for the model (I used
auto-generated ids but later changed to setting key_names for User
model but the key_name is never the same value as Id.)

Thanks
Rob

[Robert Kluin]

Hi Rob,
  The string encoded key includes the appid, that's why your seeing the keys change.  A key name or id will be unique within a namespace and kind, so if you can convert your format it should fix your problem.  You will have a problem if you have any overlapping names and ids. If you use namespaces you'll also need to account for that.

Robert

On Jul 2, 2011 6:37 AM, "Bert" <robert...@gmail.com> wrote:

[Bert]

Hi Robert,

Thanks, I've now converted the format, just testing everything now.
Another quick question, are auto-generated Id's unique across entity
groups? So if I set a parent entity on some of models, could they
potentially have overlapping ids with other models (of the same kind,
in the same namespace) that don't share the same parent?

Thanks
Rob

> On Jul 2, 2011 6:37 AM, "Bert" <robertbcur...@gmail.com> wrote:

[Robert Kluin]

As I recall the auto generated ids are unique for the kind; however, I
would not assume that is the case since it is an implementation
detail.

I encountered a similar issue when migrating to HR. My solution was to
write a custom key encoder / decoder that includes everything except
the appid. I basically use db.Key.to_path to get a list of the parts,
then concat and base 64 encode them. I then also use db.Key.from_path
to rebuild the keys when needed.

Robert

On Monday, July 4, 2011, Bert <robert...@gmail.com> wrote:
&gt; Hi Robert,
&gt;
&gt; Thanks, I've now converted the format, just testing everything now.
&gt; Another quick question, are auto-generated Id's unique across entity
&gt; groups? So if I set a parent entity on some of models, could they
&gt; potentially have overlapping ids with other models (of the same kind,
&gt; in the same namespace) that don't share the same parent?
&gt;
&gt; Thanks
&gt; Rob
&gt;
&gt; On Jul 4, 5:51 am, Robert Kluin &lt;robert.kl...@gmail.com&gt; wrote:
&gt;&gt; Hi Rob,
&gt;&gt;   The string encoded key includes the appid, that's why your
seeing the keys
&gt;&gt; change.  A key name or id will be unique within a namespace
and kind, so if
&gt;&gt; you can convert your format it should fix your problem.  You
will have a
&gt;&gt; problem if you have any overlapping names and ids. If you use
namespaces
&gt;&gt; you'll also need to account for that.
&gt;&gt;
&gt;&gt; Robert
&gt;&gt; On Jul 2, 2011 6:37 AM, &quot;Bert&quot;
&lt;robertbcur...@gmail.com&gt; wrote:
&gt;
&gt; --
&gt; You received this message because you are subscribed to the
Google Groups &quot;Google App Engine&quot; group.
&gt; To post to this group, send email to google-a...@googlegroups.com.
&gt; To unsubscribe from this group, send email to
google-appengi...@googlegroups.com.
&gt; For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
&gt;
&gt;

--
------
Robert Kluin
Ezox Systems, LLC

[Bert]

Great, Thanks very much.
Conversion is done and everything *seems* to be working:)

On Jul 5, 2:29 am, Robert Kluin <robert.kl...@gmail.com> wrote:
> As I recall the auto generated ids are unique for the kind; however, I
> would not assume that is the case since it is an implementation
> detail.
>
> I encountered a similar issue when migrating to HR. My solution was to
> write a custom key encoder / decoder that includes everything except
> the appid. I basically use db.Key.to_path to get a list of the parts,
> then concat and base 64 encode them. I then also use db.Key.from_path
> to rebuild the keys when needed.
>
> Robert
>

> &gt; For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en.

[Noah McIlraith]

From what I've noticed, the auto-generated IDs are only unique across entity groups in the SDK.

In production, each entity group has it's own set of IDs.

[Ikai Lan (Google)]

Bert,

Sorry about this. One problem here is that our tools cannot detect that the encoded key is a substring in another string.  Our tools, however, *should* detect a Key if you don't add a prefix or a suffix and do the appropriate conversion. 

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

On Thu, Jul 7, 2011 at 2:14 AM, Noah McIlraith <noah.mc...@gmail.com> wrote:

From what I've noticed, the auto-generated IDs are only unique across entity groups in the SDK.

In production, each entity group has it's own set of IDs.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/O6HAhJr4UBsJ.

To post to this group, send email to google-a...@googlegroups.com.

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.