JOINs and GROUP BYs

[DennisJ]

Hi,
I know that these operations aren't supported and BigTable isn't a
relational datastore but how would I port an application that relies
on these features to the app engine?

Take for example a number of users and a number of comments that each
user has written. How would I implement a query like "Show me all
users who have written more than 5 comments and order the result by
name or by the number of comments"?

Regards,
Dennis

[ryan]

You're right, the datastore doesn't support histograms right now,
which makes operations like these difficult.

We've had good results when we take a step back and think about our
data models from a different angle. Most app developers are accustomed
to designing SQL data models in a certain way, with a normalized
schema, foreign keys, fixed column types, etc. With Google App Engine,
you can often get good results by loosening the normalization and
other restrictions, and often by doing more work in writes instead of
queries.

In this case, instead of attempting to count the number of comments by
each user in a query, consider storing the user's number of comments
inside the user entity, updating it each time they post a comment.
Then your query would look something like SELECT * FROM Users WHERE
num_comments >= 5 ORDER BY num_comments DESC.

[Rif]

What about supporting a real object oriented type of database?

I really think somthing needs to be done around this, if I want to
write B2B sales applications or Profe of Delivery apps linking in with
Gears.. then I dont see how going to model the data with out

a) looping in the code to find the customers details which does not
matter how many server you have never going to be fast, I but had to
do it many times when join data for reporting from many system live
with out a datawarehouse.

b) having master tables that triger updates to transactional tables.

[Romain Vallet]

So that's it?
That's what it has to offer?
No joins? No foreign keys?
Honestly, what kind of application can you build without such basic
features? So if I want to count rows I have to make a specific column?
What if I want a sum? An average? What if I need some more criteria to
select my rows?

I was excited by Google App Engine at first but this is a major
letdown.

On 9 avr, 03:11, ryan <ryanb+appeng...@google.com> wrote:
> You're right, the datastore doesn't support histograms right now,
> which makes operations like these difficult.
>
> We've had good results when we take a step back and think about our
> data models from a different angle. Most app developers are accustomed
> to designing SQL data models in a certain way, with a normalized

> schema,foreignkeys, fixed column types, etc. With Google App Engine,

[Brett Morgan]

Is GMail enough of an application built without such "basic features"?
Google maps?

The truth of the matter is that what you are feeling is an emotional
reaction to change. You are looking to do what you have always done,
and you can't. The truth is, you can built out a lot of functionality,
but instead of doing the work at read time, you do it at write time.

This is a world view change brought about by the convergence of cheap
disks and the need of webapp read requests to be blindingly quick.
Yes, it requires you, the app developer to let go of the way you have
always done things.

The pay off is that you get a scalable application. While ever you
have an SQL server at the heart of your design philosophy you will
never really be able to scale out.

[Niels Egberts]

When you have this application, and then you want to add a feature that count's the ammount of posts a user made to put it in their profile page. You cant do sum() to count their post, and also you don't have a record that counted the posts when they were created. Then you're stuck right?

This approach does not seem to be very scalable (in being able to add features) to me. Or can someone enlighten me?

[Brett Morgan]

On Wed, Apr 9, 2008 at 11:37 PM, Niels Egberts <niels....@gmail.com> wrote:
> When you have this application, and then you want to add a feature that
> count's the ammount of posts a user made to put it in their profile page.
> You cant do sum() to count their post, and also you don't have a record that
> counted the posts when they were created. Then you're stuck right?
>
> This approach does not seem to be very scalable (in being able to add
> features) to me. Or can someone enlighten me?

http://code.google.com/appengine/docs/datastore/gqlqueryclass.html#GqlQuery_count

[bowman...@gmail.com]

Dynamically counting and generating the post count on every view of
the user's profile page is really inefficient. You'd be better off
having a post_count field in the users profile table, and updating it
when posts are created and deleted. Possibly run a routine every once
in while that does the count() to verify the count is correct, but,
that logic on a server based host could be costly if the site became
very active.

On Apr 9, 9:37 am, "Niels Egberts" <niels.egbe...@gmail.com> wrote:
> When you have this application, and then you want to add a feature that
> count's the ammount of posts a user made to put it in their profile page.
> You cant do sum() to count their post, and also you don't have a record that
> counted the posts when they were created. Then you're stuck right?
>
> This approach does not seem to be very scalable (in being able to add
> features) to me. Or can someone enlighten me?
>
>
>
> On Wed, Apr 9, 2008 at 3:00 PM, Brett Morgan <brett.mor...@gmail.com> wrote:
>
> > Is GMail enough of an application built without such "basic features"?
> > Google maps?
>
> > The truth of the matter is that what you are feeling is an emotional
> > reaction to change. You are looking to do what you have always done,
> > and you can't. The truth is, you can built out a lot of functionality,
> > but instead of doing the work at read time, you do it at write time.
>
> > This is a world view change brought about by the convergence of cheap
> > disks and the need of webapp read requests to be blindingly quick.
> > Yes, it requires you, the app developer to let go of the way you have
> > always done things.
>
> > The pay off is that you get a scalable application. While ever you
> > have an SQL server at the heart of your design philosophy you will
> > never really be able to scale out.
>

> > On Wed, Apr 9, 2008 at 7:29 PM, Romain Vallet <romain.val...@gmail.com>

> > wrote:
>
> > >  So that's it?
> > >  That's what it has to offer?
> > >  No joins? No foreign keys?
> > >  Honestly, what kind of application can you build without such basic
> > >  features? So if I want to count rows I have to make a specific column?
> > >  What if I want a sum? An average? What if I need some more criteria to
> > >  select my rows?
>
> > >  I was excited by Google App Engine at first but this is a major
> > >  letdown.
>

> > >  On 9 avr, 03:11, ryan <ryanb+appeng...@google.com<ryanb%2Bappeng...@google.com>>