[Juggling DB] .all function enhancement

[Drouyer Sebastien]

Hi,

I am working on the .all function because I think some functionnalities are missing on the ORM compared to the one in Ruby On Rails I am used to work with.

My changes will affect abstract-class, adapters (I am working on mysql currently), and since I am new to jugglingDB I would like some help and advice :).

The changes are currently working on my side (it needs some refinements though). I added support for three keys :

• include : allows to load relations the same way that ruby on rails does. For example :
  User.all({include: ['posts'], function(err, users) {  }) will execute two requests:
• SELECT * FROM users
• SELECT * FROM posts WHERE post_id IN (user_id_1, user_id_2, ...)

Therefore, the posts relations is saved on cache and you can load posts with

• users[0].posts
• OR users[0].posts(function(err, posts) {  })

Without any addtionnal request. If the relation is not in cache, if additionnal conditions are added, it has the same behaviour than now.

• joins : allows to add joins to model. For example :
  User.all({joins: ['posts'], where: {'posts.published': 1}}, function(err, users) { }) will execute this request:
• SELECT * FROM users JOIN posts ON (posts.user_id = users.id) WHERE posts.published = 1 GROUP BY users.id

Therefore allowing to add more complex conditions

• groupBy: allows to do a group by on one or more columns, it is needed by the joins key

Like I said these changes I made are working on my side but since I discovered JugglingDB two days ago, I am not sure all changes I made are good :

• First, I need to know the objects relations when I am on the .all function (on adapter), but I didn't find out where to get it. So I added a key on the models, relations, which for users would look like this:
  User.relations = {
      'posts': {
          model: Post,
          foreignKey: 'user_id',
          type: 'hasMany'
      }
  }
  This relations key could be edited in the hasMany on belongsTo function, and the developer wouldn't have to do anything more than now...
  Is there a similar key ? Is this the good way of doing things ?

• I also need to save relation in cache. At the moment, I added a __cache key on my objects where I save relations. But I noticed there is a cachedRelations key which is only use once in the belongsTo method. Do I need to use this key instead ? Wouldn't the name __cachedRelations be better ?

• I am only working on the mysql adapter. I could change also the sqlite adapter, but I don't know any other RDBMS... The changes here won't modify behaviour in other adapters, but mysql would have some additional keys than others, which is not great I think. How do we proceed ? Do I need to make a copy of the current mysql adapter and name it "mysql-enhanced.js" or something like that ?

Thanks,

[Anatoliy Chakkaev]

Hey Sebastien,

Thanks for great stuff you are working on. It would be great to have these things in jugglingdb, once it will work fine for relational dbs i will hook it in redis too, don't hesistate to add tests explaining your functionality to `test/common_test.js` (probably with adapter type checking to not fail in others).

1. User.relations is fine

2. Use __cachedRelations if you need it, since it's not used anywhere now, just declare it in constructor to keep performance high. Note: adding __cache to object in realtime creates new virtual class, which is not good for performance from v8 point of view.

3. While new functionality doesn't break any other (existing) stuff it's fine to update it, only create new adapter when it's not compatible with previous (which was happened to me when I rewritten redis adapter).

Good luck,

Anatoliy

--
The official community of RailwayJS
 
website: railwayjs.com,
source: https://github.com/1602/express-on-railway,
---
You received this message because you are subscribed to the Google Groups "RailwayJS" group.
To post to this group, send email to rail...@googlegroups.com.
Visit this group at http://groups.google.com/group/railwayjs?hl=en.
 
 

--
Thanks,
Anatoliy Chakkaev

[Drouyer Sebastien]

Great!

I will try to finish these functionalities this weekend or next week.

++ :)

[Drouyer Sebastien]

I have a question for the belongsto relationship. If you have a passport linked to a user on a owner belongsTo relationship (same as in the common_test.js), if you call passport.owner(), it currently returns the id. Since I am working on the relation caching functionality, I would like to know if I can return the object if it is cached. I know it is changing the way the function is working, but I think this would be a more logic way... At worst, I can always add some parameters but I would prefer the former solution.

Thanks!

Le mardi 23 octobre 2012 13:50:10 UTC+2, Anatoliy C. a écrit :

[Anatoliy Chakkaev]

Hi Sebastien,

Behaviour of method created by belongsTo should be the same for cached and non-cached state to avoid producing crappy code like:

var owner = passport.owner();

if (owner instanceof Owner) {

    // some logic

} else {

    passport.owner(function (err, owner) {

        if (err) return handleError(err);

        // same logic again

    });

}

it should be always just:

    passport.owner(function (err, owner) {

        if (err) return handleError(err);

        // some logic

    });

even if object already cached.

[Drouyer Sebastien]

Ok thanks, it makes sense. I won't change the behaviour of the belongsTo and hasMany relations ships. Maybe there should be a way to access directly this cache though ; in my pet project using jugglingdb I have a lot of relations, so it makes a lot of writing. I will try to think about it.

Thanks,

[Anatoliy Chakkaev]

No problem. Man, you probably could be interested to watch this project: ui-juggling. This is ui for jugglingdb (one crud for now). Now it's two days old, but i already have found it useful in couple apps. Here's instructions: https://github.com/anatoliychakkaev/ui-juggling

[Abed Halawi]

This is interesting! ui-jugglingdb.. looking forward.

[Drouyer Sebastien]

Seems interesting indeed :).

[brrmm]

Awesome work Drouyer (and Anatoly). I've been hacking around in jugglingDB to do custom / relational queries, but it's not more than a dirty hack. Looking forward to try your implementation. Please let us know when you have something to try, i'll be happy to test it. 

[Drouyer Sebastien]

Thanks! I am busier that expected these weeks, but I will try to add my implementation in the weeks to come (+ associated tests). What DB system do you use ?

[Anatoliy]

I use redis.

--

Thanks,

Anatoliy

[brrmm]

I'm using sqlite. I still find it hard to deal with the asynchrony, like in following example. Your addition will make it easier already. 

    var reviewsAr = [];
    pub.reviews({order: 'reviewdate DESC'}, function(err, reviews){
        if (!reviews.length) {
            cb(reviewsAr);
        }
        for (i in reviews) {
            var countDone = 0;
            var review = reviews[i];
            review.filesArray = [];
            reviewsAr.push(review);
            review.files(function(err, files){
                this.filesArray = files;
                countDone++;
                if (countDone == reviews.length) {
                    cb(reviewsAr);
                }
            }.bind(review));
        }
    });

[Drouyer Sebastien]

ok I am now back to the include function, but I am facing an issue. Since most adapters are now separated from the core, I don't know where to put tests which work on several adapters but not all (I see those tests are now commented in the core, is it still under consideration ?).

Thanks!

[Anatoliy Chakkaev]

Every adapter will be moved to separate repository. I've all commented tests that have adapter-specific code and now I'm trying to get it working everywhere, and only if test working only on specific adapter - I move test to adapter. You can see, that each adapter import same test case and then could redefine or add some test cases. For example, check this test https://github.com/1602/jugglingdb-mongodb/blob/master/test/mongodb.js

It redefines "hasMany should support additional conditions" step, because mongo support id field names as "_id", I know, this is weird inconsistency - i'm working on this issue.

I'm glad to see you again, Sebastien, you are doing great job. Hope we could continue with new structure. I don't wanted to confuse you; it should be much easier to support different adapters and have consistent api.

Cheers!

On Sat, Dec 15, 2012 at 10:26 PM, Drouyer Sebastien <sdr...@gmail.com> wrote:

ok I am now back to the include function, but I am facing an issue. Since most adapters are now separated from the core, I don't know where to put tests which work on several adapters but not all (I see those tests are now commented in the core, is it still under consideration ?).

Thanks!

--

The official community of RailwayJS
 
website: railwayjs.com,
source: https://github.com/1602/express-on-railway,
---
You received this message because you are subscribed to the Google Groups "RailwayJS" group.
To post to this group, send email to rail...@googlegroups.com.
Visit this group at http://groups.google.com/group/railwayjs?hl=en.
 
 

--
Thanks,
Anatoliy Chakkaev

[Drouyer Sebastien]

ok thanks :). I find the new structure better indeed, I just wanted to make sure I understood it well before launching pull requests.

Cheers,

Sébastien

[sh...@activelement.com]

Sorry to pull up a old thread But was this feature done? and if yes then how do i do a join on 2 tables

[Drouyer Sebastien]

As I remember, I only did the include. It joins the two tables, but it loads the relations to memory.

User.all({include: ['posts'], function(err, users) {  })

for instance...

The official community of CompoundJS.
 
website: compoundjs.com
source: https://github.com/1602/compound
bug reports: https://github.com/1602/compound/issues?state=open
---
You received this message because you are subscribed to a topic in the Google Groups "CompoundJS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/compoundjs/YxcIOKEqM8w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to compoundjs+...@googlegroups.com.
To post to this group, send email to compo...@googlegroups.com.
Visit this group at http://groups.google.com/group/compoundjs.
For more options, visit https://groups.google.com/d/optout.

[Abhishek Nair]

Thanks Drouyer,

 

Pls find my code below.

 

var Category = schemaPost.define('Category', EventStructure.CategoryStructure, {restPath: '/category' });

var SubCategory = schemaPost.define('SubCategory', EventStructure.SubCategoryStructure, { restPath: '/subcategory'});

SubCategory.belongsTo(Category, {as: 'cat', foreignKey: 'CategoryId'});

Category.hasMany(SubCategory, {as: 'subcat', foreignKey: 'CategoryId'});

var category = new Category;

var subcategory = new SubCategory;

category.subcat.build;

 

but when I do Category.all({include:['subcat']}, function(err, category) {

 

I am only getting the category information what I want is an inner join with both the tables and for the info from both the tables to be displayed

 

How to I achieve that?

 

Regards

Abhishek