Missing a way to mirror a Collection locally?

[steph643]

I often feel the need to add local fields to a Collection.

I used to use Collection._collection, but it is undocumented and exhibits strange behavior when mixing with collection update.

Here is a use case:

I have a collection with records that are organized as a tree and displayed to the user as a tree. User can expand/collapse tree node using little '+' icons.

Where do I store the expanded/collapsed states of tree nodes? (which are different among users)

I can use the old way and store them in the DOM (as a 'collapsed' class, for example), but they will be lost when reactively refreshing the DOM.

I can go what looks like the natural way: storing them in Collection._collection. But it does not work correctly: local fields are lost if added during an update round trip.

I can set up a local reactive data structure that mirrors the collection. However:

1. This sounds overkill, as it represents the third data structure containing the same information (#1 being the DOM, #2 being the local collection cache).
2. I miss an easy way to locally mirror the collection (with reactive update to insert/remove/update operations, and local fields preservation).

Any thoughts?

[Christian Stewart]

Use a transform function?

--
You received this message because you are subscribed to the Google Groups "meteor-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-core...@googlegroups.com.
To post to this group, send email to meteo...@googlegroups.com.
Visit this group at http://groups.google.com/group/meteor-core.
To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-core/9bee4f62-3b53-4b38-9bf6-eb197752c5c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[steph643]

My understanding is that a 'transform' function allows to modify the result of a query. I don't think it allows to store data. Am I missing something?

[Christian Stewart]

You are adding / modifying things on the returned objects. Is this not what you want to do?

On Wed Nov 26 2014 at 12:31:35 PM steph643 <sylvain...@gmail.com> wrote:

My understanding is that a 'transform' function allows to modify the result of a query. I don't think it allows to store data. Am I missing something?

--

You received this message because you are subscribed to the Google Groups "meteor-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-core...@googlegroups.com.
To post to this group, send email to meteo...@googlegroups.com.
Visit this group at http://groups.google.com/group/meteor-core.

To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-core/974881b6-6eb1-485f-911a-53f15c6ce163%40googlegroups.com.

[steph643]

What I want is to store some additional data on the client-side.

I can use 'transform' to add some data to a query result, but where do I get that data from?

[Christian Stewart]

Right. You want to store additional data on the client side, like a true/false checked value. On the client, add a transform function to add the default value of what you want to add. I'd imagine the object wouldn't be re-created later on.

On Wed Nov 26 2014 at 12:48:44 PM steph643 <sylvain...@gmail.com> wrote:

What I want is to store some additional data on the client-side.

I can use 'transform' to add some data to a query result, but where do I get that data from?

--

You received this message because you are subscribed to the Google Groups "meteor-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-core...@googlegroups.com.
To post to this group, send email to meteo...@googlegroups.com.
Visit this group at http://groups.google.com/group/meteor-core.

To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-core/ad365d35-281d-4b20-b8aa-b6c9fe76a567%40googlegroups.com.

[steph643]

I see 2 issues in what you propose:

1. The static default value you add in 'transform' will be re-applied every time the query is reactively triggered. So the additional field will always have the same value.
   
2. The field you add this way cannot be updated. If you try to modify it, you will immediately create and modify a server-side field instead. And anyway, this server-side field will immediately be overwritten by 'transform' to stick to the static default value.

[Vincent Jaubert]

I think the current pattern is to store Session UI interaction in a session variable.
You can write some wrapper around Session.get and Session.set to avoid accessing the Session (global !) variable directly.
Or you can use a ViewModel http://viewmodel.meteor.com/ which will do the wrapping for you and allow you to inject your JSON object (with the fromJS method) into the viewModel.

[steph643]

Vincent, thanks for your answer and for the ViewModel link (didn't know this package).

Basically you are saying that I should create a 3rd client-side structure  (#1 being the DOM, #2 being the local collection cache), whether in a Session variable or in a ViewModel. 

For reference, I think there are two more options to consider:

1. Creating a independent reactive structure using Tracker.Dependency (I guess ViewModel is implemented this way).
2. Creating an unmanaged (unsynchronized) local collection ("myClientSideCollection = new Mongo.Collection(null)").

But how do I keep this new structure reactively updated, so that it mirrors my server-side collection?

It seems to me ViewModel is based on a static description of a UI, whereas mine (the tree) is dynamic and has to match a changing collection...

[Roman Biblbrok]

Hi,

As a temporary solution sometimes I use LocalCollection on a client as a reactive model source. But it depends on a reqs that you need. To make client only changes you could

Test = new Mongo.Collection('test');

newId = Test._collection.insert({});
Test._collection.update(newId, {$set: {name: 'Roman'}});

Meteor.call('saveTestRecord', Test.findOne(newId), function (err) {
    Test._collection.remove(newId);
});

четверг, 27 ноября 2014 г., 7:05:45 UTC+8 пользователь steph643 написал:

[steph643]

Thanks Roman.

Using _collection does not work for me. I guess that is why it is undocumented. The field "name" you added to the local collection is not guaranteed to be here next time you look. Its value can even be modified without notice.

You can try this code to see the strange ways _collection behaves:

'click .change': function () {

myCollection.update(this._id, { $inc: { serverField: 1 } });

myCollection._collection.update(self._id, { $inc: { clientField: 1 } });  --> Works right now, but will be lost as soon as the previous line (List.update()) returns from round trip

}

[Vincent Jaubert]

Le jeudi 27 novembre 2014 00:05:45 UTC+1, steph643 a écrit :

For reference, I think there are two more options to consider:

1. Creating a independent reactive structure using Tracker.Dependency (I guess ViewModel is implemented this way).
2. Creating an unmanaged (unsynchronized) local collection ("myClientSideCollection = new Mongo.Collection(null)").

I have no experience with local collection, but i suppose that with both these two approaches, you are loosing the ability to keep user ui interactions between hot code pushes.

The benefit of using Session (which is what ViewModel is doing, AFAIK), is those interactions are kept. I find it a great feature. The "only" problem is that Session variable are global, and tend to encourage a sloppy coding style. For the moment, the only workaround i know of, is to wrap the direct access in a function to encapsulate the access, but it's not 100% clean.
 

It seems to me ViewModel is based on a static description of a UI, whereas mine (the tree) is dynamic and has to match a changing collection...

You can mix both styles (ViewModel and vanilla Meteor) or you can do your own wrapping of Session and build your object reactively (which probably mean rebuilding it everytime the db is update, hence the need for some Session persistence mechanism.

We need "official" patterns for this.  Meteor "coding style" is great for small apps, but when it grows it become un-manageable.

[steph643]

I don't understand how I missed it, but the solution lies in a combination of local collection and cursor.observeChanges.

Here it is:

GlobalCollection = new Meteor.Collection('global');

LocalCollection = new Meteor.Collection('');

if (Meteor.isClient) 

{

Template.list.rendered = function()

{

var cursor = GlobalCollection.find({});

this.handle = cursor.observeChanges(

{

added: function(id, doc) { doc._id = id; doc.localField = 666; LocalCollection.insert(doc); },

changed: function(id, fields) { LocalCollection.update(id, { $set: fields }); },

removed: function() { LocalCollection.remove(id); }

});

}

}

Nevertheless, it is a pity we can not use _collection directly.

@Vincent Jaubert: I am working on a big app and so far it is quite manageable.

[Roman Biblbrok]

Notes aboute your sample. When you modify your collection directly (means Collection.update) you make the same as 

Collection._collection.update(data)
Meteor.call('/collection/update/', data, function(){})

It means that Meteor do all the work. It updates your local collection for latency compensation and waits for roundtrip answer.

In my case I use LocalCollection as reactive data source. And all the current collection's data a stored on a client until you resubscribe on the livedata result again.

I mean that after initial subscription is done you could modify/insert/remove data from LocalCollection. And then (When you a ready to save changes!) sync LocalCollection's records with server-side mongo via some Methods, and resubscribe again to get changes from server. 

It is a realy handy way to work with a few collections/records on a client during he time, and save all data only when you really done with.

For examle, in my last graphics application I have had to create collection named Layers (like photoshop layers). Users could insert/change/remove Layers during their work with current design. I couldnt save the data every time user do something (atomic save). Moreover Layers data should be awaiable throught a few templates and routes. Even if I could... - OMG! roundtrip would kill a perfomance. I allow users to work with Layers._collection. And when they save current progress I simply send all Layers data on a server, clears Layers._collection and then resubscribe for Layers again to populate saved data.

четверг, 27 ноября 2014 г., 16:17:48 UTC+8 пользователь steph643 написал:

[steph643]

Roman, thanks a lot for this interesting trick about atomic bunch update of a remote collection.

However, it seems this does not solve the issue that additional local-only fields are not preserved properly in _collection.

[Roman Biblbrok]

What are additional local-only fields? How do you get them when fetching data from the server? I guess that you store some additional (or temporary data) in your models on a client-side. Do you need them after saved all the changes on server-side? I excuse but can't imagine a use case for the issue.

четверг, 27 ноября 2014 г., 18:18:26 UTC+8 пользователь steph643 написал:

[steph643]

Client-only data I need to store are UI-related. 

In my first post (see on top of this thread), I took the example of the 'collapsed' status of tree nodes in a file explorer.

For another use case, see the Meteor Tutorial. There is a todo list where you can add tasks and mark them for deletion with checkboxes. Check status of checkboxes are stored on the server, which is nonsense. Obviously they should be stored on the client, and preserved whenever some other user add or remove tasks. 

As I said earlier, this is easily solved with a combination of local collection and cursor.observeChanges.

[Vincent Jaubert]

Le jeudi 27 novembre 2014 15:47:19 UTC+1, steph643 a écrit :

As I said earlier, this is easily solved with a combination of local collection and cursor.observeChanges.

As long as you don't mind loosing the UI interactions during a hot cod push....

[steph643]

@Vincent, I don't get it, where do you lose UI interactions?

[Vincent Jaubert]

During a hot code push, AFAIK the local collection is lost, am i wrong ?

[steph643]

You are right, good point.

[Roman Biblbrok]

If we talk about hot code push there is no way to store any states, except of amplify.store and amplify pub/sub. But I guess that hot code push is dev-only feature and you can store states in Session variable or ReactiveVar for example. Hot code push in production is a really rare case and it implies global client-side drop.

пятница, 28 ноября 2014 г., 1:39:21 UTC+8 пользователь steph643 написал:

[Vincent Jaubert]

Le vendredi 28 novembre 2014 08:13:51 UTC+1, Roman Biblbrok a écrit :

If we talk about hot code push there is no way to store any states, except of amplify.store and amplify pub/sub.

Isn't it the point of Session ???

 

But I guess that hot code push is dev-only feature and you can store states in Session variable or ReactiveVar for example. Hot code push in production is a really rare case and it implies global client-side drop.

Knowing that you can do an update whenever you want without risking to annoy your users by deleting their current work  is a big plus for me.

[Jan Hendrik Mangold]

On Wednesday, 26 November 2014 12:31:34 UTC-8, steph643 wrote:

My understanding is that a 'transform' function allows to modify the result of a query. I don't think it allows to store data. Am I missing something?

That depends where you want to store it.  You said you wanted local fields, so I make that to mean you want store it in the context of the UI session?

This following works pretty well form me

Session.setDefault('messages', {});

var transform = function(doc) {

    var read = Session.get('messages');

    // add a flag read to the documnent

    doc.read = doc._id in read;

    return doc;

};

Template.hello.helpers({
 messages: function() {
   // make reactive by looking at messages

  var m = Session.get('messages');

  return Messages.find({},{transform: transform});
 }
});

And in my template I have

  <ul>

 {{#each messages}}

 <li>

 {{#if read}}

  <span class="read">{{subject}}</span>

 {{else}}

  {{subject}} <button messageid="{{_id}}" class="markread">Mark Read</button>

 {{/if}}

 </li>

 {{/each}}

  </ul>

Then I add an event for the button that adds the messageid to the Session

[steph643]

@Jan, you suggest to store UI status in a Session variable, which is the usual way. The 'tansform' function is just an optional way to make UI status available through collection queries.

However, please let me point out two important drawbacks of your method (besides the fact that using 'transform' is probably not the right way to go, and obliges you to use a hack to trigger reactivity):

1. In your example, the 'read' flag is false by default. Suppose you want it, by default, to be true or false depending on some conditions. Where would you put this initialization?
2. If docs are added/removed from the database during a session, how do you add/remove them from your session variable? (unless you can afford the session variable to grow indefinitely)

The solution I proposed earlier solves these two issues in a short and generic way.

[Christian Stewart]

Transform is run on every single object. That is where you would set the default value based on the object's properties.

--

You received this message because you are subscribed to the Google Groups "meteor-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-core...@googlegroups.com.
To post to this group, send email to meteo...@googlegroups.com.
Visit this group at http://groups.google.com/group/meteor-core.

To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-core/159687f5-1e4f-4209-a536-70e71fba5a6a%40googlegroups.com.

[Christian Stewart]

Also you can put listeners on the query to watch for the "removed" event to clear things out of your Session variable. You could also set a periodic function to clear out old entries.

On Fri Nov 28 2014 at 11:30:51 AM Christian Stewart <kido...@gmail.com> wrote:

Transform is run on every single object. That is where you would set the default value based on the object's properties.

On Fri Nov 28 2014 at 9:36:28 AM steph643 <sylvain...@gmail.com> wrote:

@Jan, you suggest to store UI status in a Session variable, which is the usual way. The 'tansform' function is just an optional way to make UI status available through collection queries.

However, please let me point out two important drawbacks of your method (besides the fact that using 'transform' is probably not the right way to go, and obliges you to use a hack to trigger reactivity):

1. In your example, the 'read' flag is false by default. Suppose you want it, by default, to be true or false depending on some conditions. Where would you put this initialization?
2. If docs are added/removed from the database during a session, how do you add/remove them from your session variable? (unless you can afford the session variable to grow indefinitely)

The solution I proposed earlier solves these two issues in a short and generic way.

--
You received this message because you are subscribed to the Google Groups "meteor-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to meteor-core+unsubscribe@googlegroups.com.

[steph643]

@Christian: good idea! Hence this alternate solution:

GlobalCollection = new Meteor.Collection('global');

if (Meteor.isClient) 

{

Template.list.rendered = function()

{

var cursor = GlobalCollection.find({}, { fields: { _id: 1 } });

this.handle = cursor.observeChanges(

{

added: function(id, doc) { Session.set('collapsedNode_' + id, false); },   // This is the init

removed: function(id) { delete Session.keys['collapsedNode_' + id]; },   // This is the cleanup

});

}

}