How integrate FireStore Health Check and Dashboard metrics with our internal Company systems

479 views
Skip to first unread message

Jim C

unread,
Apr 29, 2020, 5:05:46 PM4/29/20
to Firebase Google Group
Context: it is my first use of FireStore. I want to use it to push notification status to our Mobile Application. I can see that there is Google Firestore Dashboard under Analytics umbrella. In our company we use mainly three tools for monitoring our applications: Zabbix, Dynatrace and certain internal solution based on Elasticsearch. I need to ntegrate our internal monitoring systems with metrics resulted from our first Firestore project.

What I am looking for: based on personal assumptions:

1) Maybe there might exist either some GET endpoints that a I can connect and poll for information let's say each minute

2) Maybe, following the idea of Database Realtime pushing events accross a long time connection, I can code a Spring Boot application that import Firebase SDK and every day I connect to some specific Firestore endpoint which will push any interested events (eg. delay based on custom logic or dead service)

3) Maybe some plugin I can connect straight to a Kafka hosted in our internal Datacent

4) Some plugin to connect from Firestore/Firebase to either third tools (eg. Zabbix or Dynatrace or Elasticsearch)

5) Some dependency I could import in google-cloud-funtions thiggered from Firestore Healcheck engine in orther to consume some internal end-point posting data

Perhaps there is already some approach universally used for a scenario when you have to connect Firestore to internal monitoring system. I will be highly appreciated if tell me that than I can narrow my googling searchs because I am not finding anything usefull.

Please, it is not part of this question comparing Monitoring approach. It is a very solid fact in our company use internal Dashboards and some custom alerts trigger. I just mentioned the names above to clarify what I mean by internal monitoring tools. The focus on this question is HOW IMPORT/INTEGRATE/OBSERVE/CONSUME Firestore monitoring data. Our internal stack is beyond this question.

Jim C

unread,
Apr 29, 2020, 6:59:59 PM4/29/20
to Firebase Google Group
Context: it is my first use of FireStore. I want to use it to push notification status to our Mobile Application. I can see that there is Google Firestore Dashboard under Analytics umbrella. In our company we use mainly three tools for monitoring our applications: Zabbix, Dynatrace and certain internal solution based on Elasticsearch. I need to ntegrate our internal monitoring systems with metrics resulted from our first Firestore project.

What I am looking for: based on personal assumptions:

1) Maybe there might exist either some GET endpoints that a I can connect and poll for information let's say each minute

2) Maybe, following the idea of Database Realtime pushing events accross a long time connection, I can code a Spring Boot application that import Firebase SDK and every day I connect to some specific Firestore endpoint which will push any interested events (eg. delay based on custom logic or dead service)

3) Maybe some plugin I can connect straight to a Kafka hosted in our internal Datacent

4) Some paid plugin to connect to either Zabbix or Dynatrace or Elasticsearch

I will not get surprise if someone warn that I wrote 4 fullish ideas because there is alread some tool or approach universally used for a scenario when you have to connect Firestore to internal monitoring system. I will be highly appreciated if tell me that than I can narrow my googling searchs because I am not finding anything usefull.

Sam Stern

unread,
Apr 30, 2020, 8:35:16 AM4/30/20
to Firebase Google Group
Hi Jim,

You can use Cloud Monitoring (formerly Stackdriver Monitoring) to monitor some basic information about Firestore:

This will give you data on read/write operations as well as security rules allow/deny/error counts over time.  However Firestore does not currently offer advanced performance monitoring beyond this, for now there are only those basic metrics.

- Sam

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/2fb116c5-7bb5-4ef9-bc41-93f27c0f7699%40googlegroups.com.

Jim C

unread,
May 1, 2020, 10:32:19 AM5/1/20
to fireba...@googlegroups.com
Sam, thanks for your answer. Two related questions:

1 - Is it possible extract these basic metrics from an exposed API?

2 - Is it possible to use either Firebase Cli or gcloud to get these basic metrics?

3 - Are my questions useless in your opinion? I mean, are you aware if Google is planning to expose such information or it doesn't make sense in Cloud world and I am the only one interested in extracting monitoring info about my Firestore Database? I posted three similar questions on Stackoverflow and all three were downvoted and proposed to be closed. It seems I am looking for something that is obvious either unnecessary or useless. 

In case it adds, here are all my three downvoted questions in Stackoverflow:






To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHafJBpDN%2BTCxFZ5jRAzuetAVXOKNckf5OidiuZCQWude4qgZw%40mail.gmail.com.

Sam Stern

unread,
May 1, 2020, 12:33:05 PM5/1/20
to Firebase Google Group
Hi Jim,

Answers inline, hope they're helpful!


On Fri, May 1, 2020 at 10:32 AM Jim C <jimis...@gmail.com> wrote:
Sam, thanks for your answer. Two related questions:

1 - Is it possible extract these basic metrics from an exposed API?

Yes I believe the Cloud Monitoring API allows you to pull the data yourself, although I can't say I have personally tried:
https://cloud.google.com/monitoring/api/v3
 

2 - Is it possible to use either Firebase Cli or gcloud to get these basic metrics?

I know it's not possible in the Firebase CLI and I don't think it's possible with gcloud either although you might want to get a second opinion there.
 

3 - Are my questions useless in your opinion? I mean, are you aware if Google is planning to expose such information or it doesn't make sense in Cloud world and I am the only one interested in extracting monitoring info about my Firestore Database? I posted three similar questions on Stackoverflow and all three were downvoted and proposed to be closed. It seems I am looking for something that is obvious either unnecessary or useless. 

No!  Your questions are good questions and I agree that it would be great for Cloud Firestore to expose much more detailed monitoring metrics.  We are getting better over time here, all of the metrics we have now are somewhat recent launches and I know that we have more tools/metrics on our roadmap.

The reason this is not a dealbreaker for most of our users is because Cloud Firestore is a fully managed database with well-known performance limits.  This means that most of the monitoring burden should be on our side, not yours.  That said there are still many use cases where you'd want to do your own monitoring, as you have noticed.
 

Jim C

unread,
May 1, 2020, 2:41:53 PM5/1/20
to fireba...@googlegroups.com
Sam, thank you so much. I will take a look very carefully to  https://cloud.google.com/monitoring/api/v3. As far as I have read so far, it is possible to use gcloud for creating Monitoring Policies and edit it ( https://cloud.google.com/monitoring/alerts/using-alerting-api) . Nevertheless, it seems gcloud is able only to create and edit policies options not for reading the result from such policies. Please, if you can ask to someone expert on  https://cloud.google.com/monitoring/api/v3 if I can somehow listen alert events or get a list of alert reults I will be very thanked. I sse 
 somehow related to firestore but how to use it and which information can I search for?   

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHafJBoU7ocNzpgjDJ9-rgbLnCL0rLYzEzDH5x9OxbhYdh6s5g%40mail.gmail.com.

Sam Stern

unread,
May 1, 2020, 2:51:27 PM5/1/20
to Firebase Google Group
Hey Jim,

For more detailed answers to those questions I'd try one of the resources here:
https://cloud.google.com/stackdriver/docs/getting-support

- Sam

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAObNognF_%3DMUAjuet3zqJ%2BXJofwzAkdbm9%2BrLk5NnxC9zvuuzQ%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages