Should we build our own caching layer?

[Richard Musiol]

Hi,

I know that you don't want people to ask for this, but we've waited several months now and we got to the point where we released our mobile app to the app stores. Now one the main complaints from our users is that the app starts too slowly. So my question is: Does it make sense for us to implemented our own offline caching layer? I just want to avoid that we spend considerable time on this and a few weeks later we just have to flip the offline storage switch on Firebase. I would really appreciate at least some rough estimation on this. Thanks.

Cheers,

Richard

[Justin Noel]

Hello,

I'd like to know about this as well and have a broader request.

Several times I've asked about Firebase development efforts.  Each time, I've gotten the "we don't have a timeframe" response.  The last time is asked this and got that answer, I spent about 4 days refactoring all my code for a needed feature.  Then, 2 days later, Firebase released that feature. It had to do with auth being different for regular firebase and not being available yet with AngularFire.

So, we know you can't give us a "4 days, 16 hours, and 23 minutes (give or take)" kind of answer.  However, a "we're working on it and MIGHT have it ready in 3 months" would at least give us a clue as to where to go next.  We're developers too and know that timelines always slip.  We get that.  But we also have to answer to customers and sometimes just have to throw out a SWAG.   I'd be happy with these canned responses:

• We are all over this and are about to commit it but we don't want to get caught with our pants down so we're telling you 2 weeks
• We're working on it.  No sooner than 3 months.
• Great idea!   We'll get right on it... not really.  Look for this new feature in 6 years.

Since Firebase seems very reluctant to even expose a roadmap, ANY information other than "Hello mushroom....like the dark?  Good!  Cuz we're gonna keep you in it!" would be greatly appreciated.

Thanks,

Justin

[Jacob Wenger]

Hey Justin,

Apologies on us coming across as unhelpful in this regard. It's a tough line to tow since for every person like you who understands that deadlines slip, there are five people who send us support inquiries when things are past our estimates. I was responsible for the AngularFire auth stuff and I remember it especially well. I really did attempt to be as clear as I could that I was working on it. However, the amount of requests I received regarding it after my initial time estimate slipped made me wary to throw out dates in the future. If you look at the AngularFire Google Group and the AngularFire issue tracker, you will see me answering the same question a dozen times at least.

Ultimately, I think we can do a better job here and I'll be more wary of our generic "we will get to it eventually" responses. I also think there are some things you can do to make your life easier:

• For our open source stuff, start watching the GitHub repos you care about. GitHub makes this super easy to do. I think we do a pretty good job with the AngularFire repo especially in keeping our issues tracker up-to-date. It should give you an indication of what we are currently working on and who is assigned to what. We even try our best to put things in milestones when it makes sense.
• On a similar note, simply looking at the non-master branches out for certain repos, you can see what commits are being made and have an idea of what things are in-flight.
• While we are often wary to throw out dates on forums like Google Groups or Stack Overflow due to it being misconstrued on a large scale, we are usually pretty open through direct email. So try emailing sup...@firebase.com directly for these roadmap-type questions instead of asking them here.
• As with most tech companies, conferences are a good time to launch things. For example, we announced AngularFire 1.0.0, private backups, and SEO support at the ng-conf conference last month. If you think about other conferences that we will logically be involved with (or just follow @Firebase on Twitter for conference announcements), you can expect that we will ship big features during those.

As for this thread's initial question on the caching layer: offline persistence has been in beta on iOS for many months and we just a few weeks ago announced it in beta for Android. So yes, we are actively working on it and I am literally standing next to the dev who is coding away at it on his computer. It will be ready in a couple months, but not weeks. Doing offline right is a hard problem and there are a myriad of edge cases which still need to be figured out. Please try out our beta and help us make it better by logging bugs. We are in this together!

Thanks for the feedback. We really do appreciate it and are always working to make your lives easier.

Jacob

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/12fd3081-2913-485e-a88a-dac94c3ad97d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Zack Morris]

Just wanted to chime in and say that we did something similar for a project we're working on. It stores the current state in an NSMutableDictionary so we could retrieve values through a Firebase wrapper class during things like UITableView reloads when it’s not possible to defer to a later time because the data is needed immediately. We aren’t storing anything between launches, but it would be straightforward to write the state to a JSON file or plist.

In hindsight, conceptually this is good but I don’t know that it's ideal. Firebase really needs a way to query the current state if data is present, and return something kind of like a nonblocking socket’s EAGAIN error if it’s not. That was the motivation behind writing our caching layer, anyway. Unfortunately one of the problems is that we never know when data is no longer needed. That’s probably one of the reasons Firebase listers work the way they do. They tell Firebase what data needs to be retained and what doesn’t. We probably need to keep track of listeners and retire any data in the cache that's no longer being watched.

If I had it to do over again, I might consider using an object-relational mapping (ORM):

http://en.wikipedia.org/wiki/Object-relational_mapping

I used one in Laravel for a different php project with a MySQL backend and it worked great. We basically need a base class or protocol that can be added to a class and tell it which members map to which Firebase paths. Something like a FirebaseModel, where any properties in the model would map automagically and then their getters and setters could be overwritten. Here is a hypothetical example written in objectivejavascriptish with no error checking etc:

interface Person : FirebaseModel
{
Firebase firebase('persons');

String id; // maps to /persons/$id
String nameFirst; // maps to /persons/$id/nameFirst
String nameLast; // maps to /persons/$id/nameLast
String name; // virtual (overridden getter and setter)

Person()
{
this.primary('id');
}
}

implementation Person
{
String name() // override name’s getter
{
return this.nameFirst + ' ' + this.nameLast;
}

setName(String name) // override name’s setter
{
Array parts = name.split(' ');

this.nameFirst = parts[0];
this.nameLast = parts[1];
}
}

// Usage:

FirebaseModelCollection persons = Person.all();
TableView tableView;

persons.on('child_changed', function(Person person) {
tableView.reloadRow(person.id);
});

TableView::reloadCallback(RowView rowView)
{
rowView.text = persons[rowView.id].name;
}

TableView::changedCallback(RowView rowView)
{
persons[rowView.id].name = row.text;
}

Assuming that the models only update synchronously with the main loop (meaning a model could never change during something like a table redraw), then a view controller could set up its view, load all of its models into a collection, and then get/set values from the collection without having to worry about any asynchronous code. Then if something in the model changed, it would be notified by the standard listeners but be passed a model rather than a node value.

I’m sure I overlooked something here, and am maybe skipping over some of the more interesting relationships that can be set up between models with an ORM, but this could be a useful replacement for manually caching things.

Maybe the FirebaseModel could tell Firebase which data to store for offline access. Or more accurately, I guess any listeners activated by the FirebaseModel during a session would alert Firebase that those nodes need to be kept till next time. So if the app didn’t listen on a node during a session, that node in the cache would get freed at the next launch (or more accurately, marked in the cache’s least-recently-used strategy that it was ok to free the node on the next cleanup). I guess the models wouldn’t be necessary for Firebase’s offline storage, they would just help the user visualize what data is being cached.

Zack Morris

[Justin Noel]

Jacob and Firebase,

First, let me apologize a little bit.  I didn't mean for that to come across so strongly.  I was just trying to make it sound fun(ny).  

You've got some good suggestions for keeping up to date.  Hopefully between these ideas and tracking y'all more closely we'll be able to get a better feel on how / when things are progressing.

Thanks,

Justin

--
You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/sI9o3ccljEA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAGcMwsDEi1Zr2g0mmaqr8zk5oAXxOyuVLaekP7-Puvzt0L17jA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,

Justin Noel

http://kidsintouch.com

http://calendee.com

@calendee

@kidsintouch

[Seth Caldwell]

I wrote a web caching layer that saves to local storage so even if offline in phone gap scenario, App will still load. It's funny because in this project, Firebase is the caching layer and the offline persistence is essentially a cache of the cache. 

It only took about 3 hours to Code, so not sure what the hold up for this feature is. There are probably complexities that didn't exist in my project.

Seth

Sent from my iPhone

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAPnJ2-ReVPT299LN%3DXJJcVzLMaf1rE1%2BRHLpKQeXZfwR%2B4EgzA%40mail.gmail.com.

[Admin MyMissionAdmission]

+1 To Jacob's response. We love Firebase as much as we love it's support team!

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAGcMwsDEi1Zr2g0mmaqr8zk5oAXxOyuVLaekP7-Puvzt0L17jA%40mail.gmail.com.