Building offline / online web apps with Meteor

[Paul Scott]

Hi 

Am new to Meteor and have a requirement to build a web app that will not necessarily have an internet connection all the time. Data will need to be captured on the site when offline which will need to be sync'd up to the server when internet connectivity is restored. Seems like Meteor would be a suitable stack to use for this - just want to confirm that I'm on the right track? Any "gotcha's" to look out for?  

[Paul Reiber]

Paul -

My understanding is the reverse.   Clients must be able to contact the server, as the server is the "distributor cap" that keeps all the clients in sync.

I.e. a Meteor webpage wouldn't work at all if it couldn't get to its server.  There may well be some kind of solution for your needs that leverages meteor... but I don't think the basic framework will help you.

Regards,
-Paul Reiber

On Thursday, August 16, 2012 9:43:01 AM UTC-5, Paul Scott wrote:

 

[...] Data will need to be captured on the site when offline which will need to be sync'd up to the server when internet connectivity is restored. Seems like Meteor would be a suitable stack to use for this [...]

[Paul Scott]

Hi Paul

The client would contact the server but that connectivity could be lost and regained at a later point in time. When re-established the data would sync. Have only discovered meteor today but did a test with the "todos" example and it seems to me it would work. If you deploy their "todos" example, open two browser pages, update a few todo items in the one then switch off your internet connection, carry on updating, switch the connection back on after a few seconds the changes are reflected in both.

[Paul Reiber]

I'll have to defer to those more expert than myself regarding Meteor, regarding whether this is a workable solution for your needs.  The "todos" example is VERY simple, and possibly its model lets it work "disconnected" where some other Meteor app models would not.  There are two concerns:

-> javascript-wise, is everything you could possibly need to load all loaded when the page is loaded, or are some things loaded on demand

-> database-wise - if a client issues mongodb db calls and can't reach the server, are those cached? queued? dropped?

So... expert, please chime in - does meteor need a server, or can the server come and go, and the clients can still work?

"Paul and Paul" really want to know! :-)
-pbr

[steeve]

I have an app that is multi-user with Group/Resource based ACL and multiple collections and form processing and everything works offline except Logout because that is a Meteor.method.  My app is growing fast and will have dozens of collections and forms and so far everything seems fine offline.  I am definitely going to implement the Meteor Server Connections api stuff (state, reconnect, connect) similar to how Google shows you when you are online, offline and connecting Try again....4 secs....etc...  Because if the user refreshes the browser or your app router isn't over riding a href and you get an url click everything goes bye bye and the user is SOL!

[luckysmack]

I am new to meteor as well. So I dont know a lot on that front, But as for the site working locally and later updating the db, this is the whole design premise for couchdb. And data can even survive a page refresh. Although I am not sure how differently it would fit into meteor. My guess is that you would need to find a way to make the db changes persist over a refresh, or somehow store the changes locally. With couch, when the db is reconnected, it automatically sends the changed data to the server, where it is then applied, and determined if there is any conflicts. if there is it saves the last good state and save a conflict version of that document so the user can then fix it.

[Nick Martin]

Meteor mostly works offline.

If the connection to the server is lost, the client can still function locally. Database writes will appear to succeed on the client and reflect instantly on the screen. Once the connection is re-established Meteor will re-send all the pending method requests to the server and update the client display with the results from the server. This is all the result of latency compensation, being offline is treated like the server just being very slow.

Clients can monitor the reactive 'Meteor.status()' output to see the status of the current connection. Like steeve suggests, this can be used to drive a 'trying again' popup to indicate the offline status to the user. In fact, this would make a great package for someone to build. It would be pretty sweet, you could just 'meteor add offline-popup', then add {{>offlinePopup}} in your HTML and have a nicely styled popup with the time until reconnect, and a 'reconnect now' button. Perhaps a better name, though =)

Of course, if you hit 'reload', or navigate away from the page, etc, while offline you'll lose your Meteor session and not being able to start again until you regain network. I think this is true of all web apps with offline mode, though, so it shouldn't come as a surprise to users of your app. In the future, we may add caching of pending messages to localStorage, so that even if you reload the app while offline, you don't lose all the changes you made while offline.

Hope that clears things up =)

-- Nick

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.

To view this discussion on the web visit https://groups.google.com/d/msg/meteor-talk/-/3mpcPPU_zocJ.

To post to this group, send email to meteo...@googlegroups.com.
To unsubscribe from this group, send email to meteor-talk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/meteor-talk?hl=en.

[Paul Scott]

Thanks Nick for clearing that up - this is how I understood it to work but just wanted to check. Is there a limit to the amount of data that will be collected when offline. It seems a mongodb instance is run in the client so I assume "a lot" of data could be stored.. 

[Nick Martin]

You're welcome! Glad to help.

The limiting factor on the amount of data stored while offline is RAM on the user's machine. The in-browser 'minimongo' does not yet have a way to persist data to disk, it is only held in memory. While offline, Meteor will store in memory both the current working database and a list of messages to send to the server on reconnect, so potentially up to 2x the size of the data collected while offline. There is no hardcoded limit, but I would expect browsers would start having issues with more than a few hundred megabytes of data waiting to be uploaded. But that's just a guess, it could be much more or less, I haven't done any benchmarks.

Cheers,

-- Nick

[Jared Chung]

That clears it up for me. This is an excellent thread. It might make sense to synthesize this into a section of the docs. Nick, I'd take your first and third paragraphs straight as they are right into the docs. 

[Colin Hendricks]

I'm thinking of using Meteor for an app I need to deliver that will have to work offline for long periods of time (a few hours or more). Clearly, persistent local storage of the user's data would be necessary to make that work. Seems like it wouldn't be all that hard for you guys to store your "mini-mongodb" to disk rather than just in RAM so that it can survive across browser sessions. Conflict resolution would also be something to manage.

Would love to hear if this is on your roadmap and if so, when it might become available.

Regards,

Colin

[matt debergalis]

It is, but there's quite a bit of work between here and there. It's
not likely something we'll have before a 1.0.

> https://groups.google.com/d/msg/meteor-talk/-/JqS9nWeb700J.

[Ali Camarata]

I would love to see offline app support be something built in and easy and then to sync the data up to the server as soon as data connection is available.  this seems almost completely natural and desirable to the concept of meteor's latency compensation and real time data etc.  making offline the default not something you have to devote a lot of work to should be a direction meteor goes in.

[Nicolson Dsouza]

Check this 

play.google.com/store/apps/details?id=com.youiest.tapmatrix

for android and 

https://itunes.apple.com/ms/app/tapmate/id774935608?mt=8

does everything what you talk.

[Morten Henriksen]

Just a note, Depends on offline requirements, but I have a cordova app that requires some offline capabilities, I've build "groundDB" a while back - it simply grounds/caches the Meteor collection in local storage and it also caches outstanding method calls, resuming on connection. But it uses normal Meteor conflict resolution eg. latest data to hit the server wins so thats something we could improve.

I did "ground" the todo example http://groundtodo.meteor.com

more: https://groups.google.com/forum/#!searchin/meteor-talk/groundDB

[txshon Tseng]

This I have to remove the waitOn.... However, not every of the db is grounded.... so when I have waitOn it render first then, loading, pulling data from the server and render again .... How could I solve this.

Sincerely,

Andy

[Morten Henriksen]

I’m currently working on this - we have to change the way subscriptions work when offline and using the iron router,

https://github.com/GroundMeteor/db/issues/4

Venlig hilsen

Morten N. O. Nørgaard Henriksen

Grøn idé ApS - 30 13 12 41

-- 
You received this message because you are subscribed to a topic in the Google Groups "meteor-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/meteor-talk/tGto0cCsvXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to meteor-talk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-talk/026c8c24-4b5d-4156-ae12-a64e158ac336%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[txshon Tseng]

I saw this post... http://blog.groupbuddies.com/posts/45-offline-web-apps-with-meteor

However, my Subs just wouldn't work sometimes ... where should I put it Before, After, Action, or where.

It tells me not to use WaitOn....

Sincerely,

Andy

[Morten Henriksen]

For now you have to think about subscriptions as the data you want offline, this can be difficult - the think I’m working on will allow the iron router to work as it normally does - but it will in a way tell grounddb what data is most important to the user - this allows ground db to be really clever about what data it should try to keep locally - e.g. if hitting quotas.

Notes:

The ground:store package allows us to write cordova file storage - this will allow for a lot of data - but we can still hit quotas, I’ve optimized Minimax a bit allowing for better compression, its still regular EJSON but it saves about 40% depending on data structure. https://github.com/GroundMeteor/db/releases/tag/v0.3.0

So my todo list at the moment for grounddb:

1. Make intelligent subscriptions to help keep important data, and prevent sending data that is already on client (this should be enough to ground the new todos example...)

2. Better support for user login

3. Allow user data to be compressed even more

4. Allow user data to be encrypted locally

At the moment theres no ETA, if anybody out there wants some of this stuff sooner and want to fund some of it - please let me know.

Kind regards Morten,

Venlig hilsen

Morten N. O. Nørgaard Henriksen

Grøn idé ApS - 30 13 12 41

To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-talk/f0e96d5e-fc7a-411d-a5af-68ed65c0b21a%40googlegroups.com.

[txshon Tseng]

I saw this post http://blog.groupbuddies.com/posts/45-offline-web-apps-with-meteor so I also install appcache

But I saw your post saying should use localstorage instead of cache.....

I'm confused!

All of the combinations will eventually work....

However, the problem is the same.... I some times get the db and sometimes I get it and the browser cleans it..(Maybe the file is too big on local)

What should I do to get a constant data.... that will not clean itself.

Sincerely,

Andy

[Morten Henriksen]

Ground:db uses ground:stores to figure out the best storage available, at the moment I’ve only implemented local storage - but it should be fairly trivial to add any since the api is async.

That said, Ive never experienced that the local storage is being reset.

What platform are you experiencing this issue?

Venlig hilsen

Morten N. O. Nørgaard Henriksen

Grøn idé ApS - 30 13 12 41

To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-talk/37c8780a-855b-4775-a945-47450eb6d2a7%40googlegroups.com.

[Andy Tseng 曾郁翔]

I'm experiencing  it in my local host and Heroku
http://wecare2.herokuapp.com

[txshon Tseng]

I'm still having this issue and don't know how to solve it.

So I don't need to install the appcache to in order to make it work?

http://wecare2.herokuapp.com

Sincerely, Andy

[txshon Tseng]

Thanks I have it solved....

This is how I build for offline and online solution

if(Meteor.status().connected===false) this.render();

else{

if(Ground.ready()){

subscribed = false

  Tracker.autorun( function(){

  if(!subscribed){

  Subs.subscribe("weWillVote");

subscribed = true;

  }

  })

}

if(subscribed===true){

preloader.off();

this.render();

}else{

//loading

preloader.on();

this.render('loading'); 

}

}

However, the loading part still doesn't work....

Testing out how to solve it.

Currently, I'm using iron-router and the Ground seems to add hash tag after the address '#!'

Sincerely,

Andy

[Morten Henriksen]

Call all subscriptions asap, when Ground.ready() is true (when all subscriptions are ready in Meteor) ground db will clean up / align with subscriptions. For good/bad this is the current behaviour - its not optimal, I’m working on more intelligent subscriptions and publish methods for ground db. (This will enable resumable subscriptions)

Venlig hilsen

Morten N. O. Nørgaard Henriksen

Grøn idé ApS - 30 13 12 41

-- 
You received this message because you are subscribed to a topic in the Google Groups "meteor-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/meteor-talk/tGto0cCsvXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to meteor-talk...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-talk/7bcaf718-33b2-4d55-b7ce-12501ff72f20%40googlegroups.com.

[Andy Tseng 曾郁翔]

Thanks I'll try out

To view this discussion on the web visit https://groups.google.com/d/msgid/meteor-talk/0764ECF5-2FDB-498A-BDFC-1C0C9DBF65C5%40gi2.dk.

[txshon Tseng]

I was using new  Ground.Collection('cares') in 'global' function...

In this way, the problem occur that when the subscribe is not ready Ground runs! and it get 0 data of it!

For now, I use the traditional Meteor.Collection first and subscribe it... after the subscription is ready()

Then, lookup if I Ground it already.... if not.... ground it!

Here's the code!

var subCares = Subs.subscribe('cares');

Subs.subscribe('likes');

  if(subCares.ready()){

  if(!Ground.lookup('cares'))Ground.Collection(Cares);

  preloader.off();

this.render();

}

else{

preloader.on();

this.render('loading');