Building server side logic for Firebase with AWS

[sonaye]

I am trying to structure an application environment that utilizes resources offered by Firebase and Amazon Web Services (AWS).

Firebase takes care of the front end (client side logic) by providing a database and a hosting service for a web application (static javascript). AWS takes care of the server side logic, which is needed to carry out some tasks that cannot be done or exposed in the front end (examples: sending email, processing checkouts, connecting to third party APIs with private keys, etc), by providing a Node.js server running on Elastic Beanstalk.

I came up with the following logic:

1. Node.js server listens to changes in database
   
2. Web app updates database
   
3. Node.js server detects a change in the database
   
4. Node.js server processes task

I am covered when it comes to building the web application and having it linked with Firebase resources, but I am having some issues with the server side logic, especially since I am not heavily experienced with AWS or Node.js servers.

How can I make the Node.js server listen to the changes in database? meaning that it should always be running and listening for any triggers. The tasks should run in realtime, no queues, I am not interested in a cron-job-solution-like.

I do have a mechanism in mind to make sure that all the updates were indeed processed, let us say that for some reason the server didn't detect an update to the database, it was down for a couple of seconds or something, the server cannot skip any update. This can be easily solved by making the server listening to a separate database node that contains a list of all the triggers, and once processed it will remove that trigger. This is my current idea, can you verify that this approach is solid and won't cause any issues? are there any other ways to establish this all-triggers-must-be-processed checking mechanism?

As a code:

// import firebase

firebase.initializeApp({
    apiKey: '<your-api-key>',
    authDomain: '<your-auth-domain>',
    databaseURL: '<your-database-url>',
    serviceAccount: '<your-private-key>' // full access to db
});

firebase.database().ref('path/to/triggers').on('value', function(triggers) { // is .on('child_added', ..) better?
    triggers.forEach(function(trigger) {
        // run process
        firebase.database().ref('path/to/triggers').child(trigger.key).remove();
    });
});

What do I need to do exactly to keep this piece of code always running on the Node.js server? never disconnect or timeout, reconnect when there is a failure, and how can I establish that on Elastic Beanstalk?

The reason I am doing all of this, is that I don't want my front end/client to have any information about my backend, I cannot afford having any manipulation by the user, just in case you were wondering why not use an http trigger from the client side. 

Overall, does this approach makes sense? I am a heading to the right direction with this?

[Mike Mcdonald]

Excellent diagram :)

I think you're blending a number of different questions together:

• How does the Firebase Realtime Database eventing model work
• How does the Firebase Realtime Database interact with servers
• How do you perform maintenance/operations work to keep your backend up

Generally speaking you're on the right track.

• On the server, you set up listeners to locations in the Realtime Database, and when the data at those locations change, you react to them appropriately. I'd take a look at the example I wrote a while back that does chat message sanitization, which is a great (and simple example of how this interaction could work). It uses Firebase Queue, which handles triggers in a robust manner, and provides a nice scaffold for you to build on.
• Server side set up is explained here, but TL;DR: think of your server as another client--it functions pretty much the same. You can give it an authentication credential with higher privilege though, and use security rules to make sure that it actually has more power (or clients have less).
• This depends a lot on infrastructure (given that we've got experience with it, I typically recommend Node.JS Managed VM's running on Google Cloud Platform), as well as your operations overhead. A simple way to keep things going is to use something like forever.js.

Thanks,

--Mike

[a.]

After plenty of thought, I decided to drop Firebase's hosting service and do all the work on a dedicated Node.js server (or service) that handles all the logic for me. This approach is more stable I think, all in sync and it gives me the best from all the worlds, it also enables the application to scale with ease.

I've been doing some brainstorming.

At the moment, I am investigating two paths. The one in red is where I use AWS as my backbone, I really like AWS but I thought that I should look at other options (in blue), basically using a PaaS. Luckily, all of this can be delayed, I can study it with more detail when my application becomes ready for deployment. Labels in purple are features that are not fundamental at the beginning of the startup, but there is a plan to have them implemented in the future, you can see how easy it is to plug in a new service with a structure like this.

What is beautiful about all of this: it's all javascript.

[a.]

I did take a look at firebase-queue and forever.js, interesting stuff and very close to what I was looking for. I came up with another solution, you can find some details in a reply to the original message I just posted.

I haven't thought about Google's Cloud Platform before to be honest, tried to use it once, and I remember that the first page I encountered (a signup page) was very user unfriendly, so I just closed the tab, I'll take another look at it. Can you provide any info on why should I switch to it instead of using AWS? are there any advantages particularly for using it with Node.js?

Thanks for the reply.