Various questions - Authentication, Offline Capabilities & Firebase Queues

[Olivier Haouchine]

Hello Firebase People,

I have a project that consists in rewriting an existing application using the Firebase platform.

The existing app is an offline messaging app, that allows people to chat using Bluetooth or WiFi, without any internet connection. There is no server of any kind for the moment, no backend at all. The app just works by itself locally (uses network only for authentication with various social networks).

The new version will be usable online, offline capabilities being relegated to a position of secondary importance (but not removed !). The platform chosen to develop the new version will have to provide offline capabilities. We except to have peak of 100k concurrent users. 

I read Firebase documentation, various blog articles, reviews, code snippets etc and finally concluded that Firebase is well-suited for our needs. Even if it may become more complicated in the future, depending of my customer wills, it will allow us to save a lot of time and complexity in the first version. And it seems strong enough for our needs.

For the moment, the imagined project architecture is the following : 

• Some client applications, iOS and Android for the first version, that will use most of Firebase services. The only thing client app will be connected with will be Realtime Database.
• An "admin server", which is a Node.js application that uses Firebase Admin SDK & Firebase Queues. This server will not be reachable from the outside, client app will not communicate directly with it.
• An ElasticSearch instance to index data and allow searches. We will have a lot of geographical queries in the future App, and Firebase Realtime Database does not seem well suited for this kind of operation. ElasticSearch will be used in the same way Flashlight does, but using Firebase Queues for better scalability. 

This architecture seems well-suited to our needs, and prevents us from having a webserver to manage. No need to put in place load balancing, reverse proxies and other fancy things with this solution. 

Unfortunately, and that is the point of this post, there is some grey areas that we need to enlighten before validating Firebase use : 

Firebase Queues

1) How to manage and monitor Firebase queues properly ? I know this question is not directly related to Firebase, but this point is pretty unclear for me. I can of course monitor resources consumption of the various servers hosting the admin app, and that Firebase queues allow easy horizontal scaling by preventing conflicts between workers of the same queue. But I don't have a clue of what to monitor to determine how many app instances I should put in place, and how to determine how many workers I should define for each queue.

Offline Capabilites

As stated earlier in this post, the app will have to be usable offline. We will need a local persistence of data, coupled with a synchronization mechanism. Firebase does provides this kind of features, with setPersistenceEnabled and keepSynced methods.

2) setPersistenceEnabled : What is the exact behavior of this method, behind the scenes ? Is the data stored locally in JSON format ? Can I, as a developer, extract and read this local content and control that everything is going well ? Moreover, which data is stored locally ? Only references that are read or written during online usage ?

3) keepSynced : How does it work ? Does Firebase always keep an active listener on the targeted reference, to be notified of every changes on server ? Does it work in background ?

4) A requested feature is to have the app sending regularly the current GPS position to the backend, in order to provide to the user a list of the other users he met during his walk for example. The idea would be to write these new geographical positions in the Realtime Database, and that a Firebase queue queries ElasticSearch to get nearby users. If found, the admin server will send a push notification to the client, containing the list of nearby users ids. What should I do, when receiving this push notifications, to retrieve the nearby users data in the local database ? Is setPersistenceEnabled enough to  get it working ? If yes, the only thing to do would be, for each nearby user id found, to read the reference in Realtime Database, and doing nothing with it. The only fact of reading it would make Firebase to copy data into local database, because of setPersistenceEnabled. The user would then be able to see the list of users he met, even if he has no internet connection at this moment.

Authentication

Our app will provide auth providers : Email/Password, Facebook, Google, Twitter, LinkedIn and Instagram. While the first three ones are already supported by Firebase, others aren't. 

I found some documentation and examples on the web. It seems pretty simple to integrate custom providers : authenticate with it and retrieve token + user profile, then create a custom token with Admin SDK, for example by concatenating provider name + provider account id. Then return the generated token to the client which will use it to authenticate.

I found two "official" examples of it : https://firebase.googleblog.com/2016/10/authenticate-your-firebase-users-with.html and https://firebase.googleblog.com/2016/11/authenticate-your-firebase-users-with-line-login.html . The first one is a web app, the second one an iOS app. In both case, a server acts as an auth server, taking a request with custom provider auth token, and returning Firebase generated token.

This is kind of problematic for me, since I would have an auth server to manage, that should be able to handle a lot of simultaneous connections. 

5) Would it be possible to develop an asynchronous auth system with Firebase, using queues ? The logic would be the same as Flashlight : the client pushes provider name + provider token, and then listens for the response. The queue processes tasks by retrieving user provider profile, generate custom token and writes it in the reference the client is listening to. When the client gets the generated token, it uses it to authenticate and then delete the response. Would it be viable in your opinion ? The biggest problem of this solution seems to be about security : response reference would have to be readable by everyone.

Offline Authentication

6) How to handle offline authentication with Firebase ? Does the client has to save the last Firebase token he authenticated with, and then call signInWithCredential or signInWithCustomToken ? Is this done automatically ? 

7) What is the behavior when user authenticate offline with an expired token, and then get internet connection back ? Are transactions executed directly, or does Firebase wait for the user to authenticate ? 

8) Documentation states that offline transactions are not persisted across app restarts. Why is that ? Does it means that I cannot rely at all on Firebase local database if used offline ? Should I persist offline created data in another database, and handle this data write into Firebase by myself ?

Auth providers linking

Firebase provides a feature that allows the linking of multiple auth providers with a Firebase user account. This would be really interesting for our app.

9) Simple question : is linkWithCredential usable with some custom providers ? If yes, how ? Is there a linkWithCustomToken method or equivalent ? 

10) I guess the previous question answer is no. If I'm right, then what do you think I should do ? Use Firebase standard linking process for handled providers (Facebook and Google in this project), and to develop my own linking mechanism for others providers, hoping for Firebase team to integrate the providers we use in the future ? Or should I totally get rid of standard linking mechanism and only use my own ? 

That's all folks.

Apologize me for my approximate English.

Maybe I should've split this post into multiple ones. Sorry for that too !

Thanks in advance.

Best Regards,

Olivier

[Mat Williams]

Hi Olivier

have a look at geofire for your location stuff

github.com/firebase/geofire

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/c05bc9b2-2956-4366-91fd-f2f1837ff2ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Olivier Haouchine]

Hello Mat,

Thanks for this message.

I've had a look a geofire while I was studying Firebase. 

Even if it seems nice, I was afraid it would not be scalable enough, this kind of Firebase data querying going against the good practice described in the document. Or maybe I misunderstood something somewhere ? 

Also in our specific case, even if our queries are location-based, there is also some language-based full text search, where ElasticSearch shows its power.

Hi Olivier

To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.

[Olivier Haouchine]

After having done a few tests, I'm answering myself on some questions : 

1) Firebase's new feature, Functions,  will make Firebase Queue obsolete, and will allow us to get rid of the various scalability issues. 

3) Yes it seems to be working this way. Visibly, there is no problem using it while the app is in background, with something like a Service on Android.

5) Same as 1, Functions will allow us to handle auth with custom providers, like shown in this example

6) In any cases, if the token  has expired, the client will be responsible for refreshing the token when network connection becomes available. 

9) No.

10) I think I'm going to develop my own linking system, in order to keep consistency.