Integrating Search Engine back-end with firebase

[Raj Bhandari]

Hi,

I am new to firebase. And I really like the idea not having to build the infrastructure/plumbing to build an app. 

In this regard, I want to leverage most of what firebase offers to build my app.

My app will be a firebase app with iOS/andriod/web front-end using the firebase database for storing the data.

I would also need a proper search-engine to then index that data (either ElasticSearch or SOLR is fine).

My question is how can I integrate such a back-end (running on either GCE or GAE) with firebase.

I did find this firebase/flashlight (https://github.com/firebase/flashlight) but I am not sure if it gives me enough flexibility

to transform and process data that will be captured in the firebase database before it gets ingested into Elastic Search back-end. 

As the app I am building is mostly search-driven, I want to build most of the business logic of ingestion and querying into the service itself to reduce duplication.

Any pointers on where to look further will be appreciated.

Kind Regards,

~ raj

[Kato Richardson]

Hi Raj,

Not really sure what you're expecting here. Flashlight is a great example of how to link together Firebase and ES. Beyond this, it's really going to depend on the specific constraints and needs of your app, and be highly proprietary.

The transform step is pretty straightforward with a few tweaks to the OSS library, all contained in one function which you can override by passing in as an argument. 

You'll probably need to make some tweaks later, based on your specific needs and data structures, to achieve massive scale, as the app will load all data at paths you ask it to monitor right now. Without any modifications, Flashlight will read all data at the paths it monitors (even things it has read before) if the service is restarted. So naturally, setting up some sort of queue for ingestion, or archiving old records will be needed when you get into millions of records and GBs of data.

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-talk+unsubscribe@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/cebf94a8-873e-4456-82de-a930f72a83c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398