Gremlin/Janusgraph Metrics Monitoring
41 views
Skip to first unread message
Marie Diana Tran
Sep 13, 2021, 2:31:27 PM9/13/21
to Gremlin-users
Hello gremlin-users,
We deploy a cluster of
- 4 janusgraph instances served with gremlin
- BigTable as the strage
So far, we have around 50+ millions vertices and plan to have around 2 billions edges.
Since Janusgraph provides gremlin-server-metrics.csv directory with metrics( on the eval, errors & so on), we manage to have a monitoring system setup based on theses gremlin servers metrics.
Now, we want to have the number of vertices and edges stored on BigTable and metrics on the indexes, what will a good way to do it, knowing that counting the number of vertices and edges through g.V.count() will be too slow ?
For example, we want the metrics on the edges and vertices count every 30 sec but g.V.count() would already more than 5min.
Hope to read some answers and have some leads,
Best regards,
Marie
We deploy a cluster of
- 4 janusgraph instances served with gremlin
- BigTable as the strage
- Elasticsearch for indexing
So far, we have around 50+ millions vertices and plan to have around 2 billions edges.
Since Janusgraph provides gremlin-server-metrics.csv directory with metrics( on the eval, errors & so on), we manage to have a monitoring system setup based on theses gremlin servers metrics.
Now, we want to have the number of vertices and edges stored on BigTable and metrics on the indexes, what will a good way to do it, knowing that counting the number of vertices and edges through g.V.count() will be too slow ?
For example, we want the metrics on the edges and vertices count every 30 sec but g.V.count() would already more than 5min.
Hope to read some answers and have some leads,
Best regards,
Marie
Stephen Mallette
Sep 15, 2021, 6:49:59 AM9/15/21
to gremli...@googlegroups.com
This may be a better question for the JanusGraph list, but if you only need the answer every 30 seconds, I suppose you could get such a count with Spark over 50M vertices. Not sure if counts of edges would take longer at 2B but if you dont need an exact count you could probably get a decent estimation if you had some statistics on your graph characteristics.
--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/a698747d-cc41-4caf-af7a-989d354451a7n%40googlegroups.com.
HadoopMarc
Sep 18, 2021, 4:56:15 AM9/18/21
to Gremlin-users
Hi Marie,
Current status on your requirement is kept at:
Best wishes, Marc
Op maandag 13 september 2021 om 20:31:27 UTC+2 schreef tran.mar...@gmail.com:
Reply all
Reply to author
Forward
0 new messages