Modelling data in Java for GAE
37 views
Skip to first unread message
MK Z
Jul 25, 2011, 2:09:06 PM7/25/11
to Google App Engine
Hi,
I'm pretty new to this GAE platffrom and the Google BigTable. Ive been
trying to search for tutorial (with example) on this subject but found
nothing to get me started (found handful of Python examples but my app
is to be written in Java). There are few good examples which I have
tried and its working perfectly on GAE. As I come from RDBMS
background so im totally clueless on how "relationship" between tables
work as Google BigTable stores data differently. I also read
Programming Google App Engine book the example shown is not in-depth.
here is my scenario: If I have table department and employee, a
typical query will be select d.deptName, e.employee name, ... from
department d, employee e WHERE.... as far as my reading, this is not
supported in GQL. another thing, how do I do the data modelling? do I
need to use the mappedBy annotation? Any help will be appreciated.
I'm pretty new to this GAE platffrom and the Google BigTable. Ive been
trying to search for tutorial (with example) on this subject but found
nothing to get me started (found handful of Python examples but my app
is to be written in Java). There are few good examples which I have
tried and its working perfectly on GAE. As I come from RDBMS
background so im totally clueless on how "relationship" between tables
work as Google BigTable stores data differently. I also read
Programming Google App Engine book the example shown is not in-depth.
here is my scenario: If I have table department and employee, a
typical query will be select d.deptName, e.employee name, ... from
department d, employee e WHERE.... as far as my reading, this is not
supported in GQL. another thing, how do I do the data modelling? do I
need to use the mappedBy annotation? Any help will be appreciated.
Pascal Voitot Dev
Jul 26, 2011, 5:12:20 AM7/26/11
to google-a...@googlegroups.com
Hi,
BigTable is a NoSQL DB without join feature that you know in SQL.
So you manage relations manually and do join by issuing several request instead a single one and you join within your application context.
Basically you do more computation in your app than in the DB in this case.
But the DB is more versatile and less contrained which is a big advantage sometimes.
Generally, you don't create normalized models in DB such as BigTable because you can't join.
So you tend to denormalize your models (create join table) and use redundancy.
In GAE, a great feature is the entity grouping where you can associate a child to a parent by its key but it has its limitations also.
One thing to know is that, in the same transaction, you can target entities only in the same group of entities.
One other thing to know is that when you fetch multiple entities in a row, the maximum returned number of entities is 1000 but you can manage it with offsets or better with cursors (look at GAE docs).
There are a lot of other features but begin with that ;)
regards
Pascal
PS: I'm lead developer of siena project providing a simple mapping layer for NoSQL/SQL (http://www.sienaproject.com)
BigTable is a NoSQL DB without join feature that you know in SQL.
So you manage relations manually and do join by issuing several request instead a single one and you join within your application context.
Basically you do more computation in your app than in the DB in this case.
But the DB is more versatile and less contrained which is a big advantage sometimes.
Generally, you don't create normalized models in DB such as BigTable because you can't join.
So you tend to denormalize your models (create join table) and use redundancy.
In GAE, a great feature is the entity grouping where you can associate a child to a parent by its key but it has its limitations also.
One thing to know is that, in the same transaction, you can target entities only in the same group of entities.
One other thing to know is that when you fetch multiple entities in a row, the maximum returned number of entities is 1000 but you can manage it with offsets or better with cursors (look at GAE docs).
There are a lot of other features but begin with that ;)
regards
Pascal
PS: I'm lead developer of siena project providing a simple mapping layer for NoSQL/SQL (http://www.sienaproject.com)
On Mon, Jul 25, 2011 at 8:09 PM, MK Z <v5s12....@gmail.com> wrote:
Hi,
I'm pretty new to this GAE platffrom and the Google BigTable. Ive been
trying to search for tutorial (with example) on this subject but found
nothing to get me started (found handful of Python examples but my app
is to be written in Java). There are f in ew good examples which I have
tried and its working perfectly on GAE. As I come from RDBMS
background so im totally clueless on how "relationship" between tables
work as Google BigTable stores data differently. I also read
Programming Google App Engine book the example shown is not in-depth.
here is my scenario: If I have table department and employee, a
typical query will be select d.deptName, e.employee name, ... from
department d, employee e WHERE.... as far as my reading, this is not
supported in GQL. another thing, how do I do the data modelling? do I
need to use the mappedBy annotation? Any help will be appreciated.
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Jose Montes de Oca
Jul 26, 2011, 3:21:37 PM7/26/11
to google-a...@googlegroups.com
Hi,
You don't need to worry much on the underlying of the datastore (big table) but rather understand how the datastore works as a non relational way of persisting data in your application. If you are using Java you have:
There are also other frameworks that you can check out:
- Objectify: http://code.google.com/p/objectify-appengine/
- slim3: http://code.google.com/p/slim3/
I will recommend to start by reading about the Datastore: http://code.google.com/appengine/docs/java/datastore/overview.html getting familiar with its concept (keys, properties, entities, ancestors ...)
Another good resource of information would be Past Google IO talks:
For the scenario you picture on your example, you are right you can not do join queries. A workaround would be to de-normalize your data model to have the department information within the employee entity, so you could make a query over the kind employee.
Hope this helps you getting started on Datastore and App engine. If you have more questions don't hesitate to ask!
Happy coding!
Best,
Jose Montes de Oca
MK Z
Aug 14, 2011, 10:28:06 AM8/14/11
to google-a...@googlegroups.com
Thanks for all the replies,
just a quick question. Since BigTable is object database so why bother having relationship between entities? I know there are advantages to that. But for small application like school project, I think I dont really need to have all those relationship annotations in relevant data model. I mean you can simply "join" the query result from the app context? For example if I have forum board containing few fields including forum category (e.g Science, Tech, etc) and forum group (adult/teen/kids). In RDBMS, forum category id and forum group id will be stored in Forum table as foreign key which make them related. but for BigTable, the only way to relate them is using the mappedBy annotation or embedded class(not really a relationship).
Im thinking not to have the relationship set in the relevant class and simply tie everything at app level. for example after querying the forum object, it will list and display all forum topics according to category and group. Since forumcategoryid and forumgroupid are stored as Long type in forum object, displaying series of number is meaningless. so basically I can create a function that will query this id to respective objects and return the forum category description based on ID supplied. Any advice?
btw, ive gone through the links u given, but there isnt enough app-like example on one-to-many relationship (only the simple guestbook example)
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-MPXy0Q40QsJ.
Pascal Voitot Dev
Aug 14, 2011, 12:46:19 PM8/14/11
to google-a...@googlegroups.com
In NoSQL, you don't have join as you said.
In NoSQL, managing data is more direct than RDBMS but when you want to merge data from different tables, you need to do it yourself in your app. In a summary, for relations, you do the job of the DB. So, this might seem a drawback but it's also a big advantage because you control everything. It also allows scalability and distribution!
Therefore, you must design your data to allow the kind of request your need in your app. You don't design a perfect normalized model but an efficient one. Generally, this is the kind of job designers have to do with their RDBMS models when they discover their perfect relational normalized models are not very efficient in production. So in NoSQL, you are forced to think like that immediately. I know, it has pros and cons ;)
Anyway, in NoSQL, denormalizing is not necessarily bad and redundancy is also advised in lots of cases. Moreover, using a memory cache is a good solution in many cases in which you won't query the datastore but get the data from your cache. Finally, learn the behavior of GAE datastore for indexing/filtering/ordering to build the right model.
Sorry if my advises are quite generic :p
regards
Pascal
In NoSQL, managing data is more direct than RDBMS but when you want to merge data from different tables, you need to do it yourself in your app. In a summary, for relations, you do the job of the DB. So, this might seem a drawback but it's also a big advantage because you control everything. It also allows scalability and distribution!
Therefore, you must design your data to allow the kind of request your need in your app. You don't design a perfect normalized model but an efficient one. Generally, this is the kind of job designers have to do with their RDBMS models when they discover their perfect relational normalized models are not very efficient in production. So in NoSQL, you are forced to think like that immediately. I know, it has pros and cons ;)
Anyway, in NoSQL, denormalizing is not necessarily bad and redundancy is also advised in lots of cases. Moreover, using a memory cache is a good solution in many cases in which you won't query the datastore but get the data from your cache. Finally, learn the behavior of GAE datastore for indexing/filtering/ordering to build the right model.
Sorry if my advises are quite generic :p
regards
Pascal
Reply all
Reply to author
Forward
0 new messages