Is it possible to setup a load balancer for Gremlin Servers running Neo4J?

[ha...@nugit.co]

I have three Gremlin Server instances, each running a Neo4J instance (the HA Cluster), I wanted to know how would I setup a load balancer in front of the Gremlin Server instances what data should I be sending the load balancer to seed a graph.

Is it possible to send traversal bytecode to the load balancer?

I'm using Gremlin Python to generate traversal bytecode.

[Jean-Baptiste Musso]

It should be fairly transparent doing WebSocket load balancing with tools such as HAProxy or nginx, since Gremlin server communicates over WebSocket by default. Technically, you wouldn't really just be sending bytecode *to* the load balancer, but *via* the load balancer *down to* different Gremlin server instances. Note that by using a load balancer, you'll be introducing a single point of failure (SPOF) here and this is considered an anti-pattern for distributed databases.

To avoid this SPOF, have you been considering doing load balancing at the client (= application) level? You could have your applications directly connect to the 3 Gremlin server, then use a naive round-robin algorithm between the 3 Gremlin servers. I'd go that route since it's easier to setup (no need to deploy a load balancer) and a lot more resilient.

See also https://issues.apache.org/jira/browse/TINKERPOP-1249 which added heart beat - I guess some people were successful in putting a LB in front of Gremlin servers though.

Jean-Baptiste

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/0590fec0-4f27-4fa7-8475-3fc68fe9acde%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ha...@nugit.co]

I wasn't considering doing load balancing at the application level. But thank you for pointing it out. It does make things more robust. Also reduces latency of routing requests to the load balancer and getting a request from the database apart from avoiding a SPOF.

On Saturday, February 3, 2018 at 5:44:59 PM UTC+8, Jean-Baptiste Musso wrote:

It should be fairly transparent doing WebSocket load balancing with tools such as HAProxy or nginx, since Gremlin server communicates over WebSocket by default. Technically, you wouldn't really just be sending bytecode *to* the load balancer, but *via* the load balancer *down to* different Gremlin server instances. Note that by using a load balancer, you'll be introducing a single point of failure (SPOF) here and this is considered an anti-pattern for distributed databases.

To avoid this SPOF, have you been considering doing load balancing at the client (= application) level? You could have your applications directly connect to the 3 Gremlin server, then use a naive round-robin algorithm between the 3 Gremlin servers. I'd go that route since it's easier to setup (no need to deploy a load balancer) and a lot more resilient.

See also https://issues.apache.org/jira/browse/TINKERPOP-1249 which added heart beat - I guess some people were successful in putting a LB in front of Gremlin servers though.

Jean-Baptiste

On Fri, Feb 2, 2018 at 7:52 PM, <ha...@nugit.co> wrote:

I have three Gremlin Server instances, each running a Neo4J instance (the HA Cluster), I wanted to know how would I setup a load balancer in front of the Gremlin Server instances what data should I be sending the load balancer to seed a graph.

Is it possible to send traversal bytecode to the load balancer?

I'm using Gremlin Python to generate traversal bytecode.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.