gRPC client fallback from load-balancer to round robin DNS endpoint on LB outage

[Krishna Sai Veera Reddy]

Hello All,

I am working on a gRPC client that connects to a gRPC service fronted by an LB but would like the client to automatically fallback to using round-robin DNS to directly connect to the backend servers when the LB goes down. How do I go about this? Would I need to implement my own custom LB policy? Any help is appreciated and thanks in advance!

[Larry Safran]

This is a perfect scenario for a custom LB policy.  Since LB policies form a tree, you could easily have your policy delegate to either a PickFirst or RoundRobin policy depending on whether or not the LB is up.  Just have your policy define a ForwardingLoadbalanceHelper whose updateBalancingState passes the picker from the oppropriate downstream policy.

On Fri, Nov 3, 2023 at 12:29 PM Krishna Sai Veera Reddy <krishnasaiv...@gmail.com> wrote:

Hello All,

I am working on a gRPC client that connects to a gRPC service fronted by an LB but would like the client to automatically fallback to using round-robin DNS to directly connect to the backend servers when the LB goes down. How do I go about this? Would I need to implement my own custom LB policy? Any help is appreciated and thanks in advance!

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/df8f4a85-5020-4c68-bb94-cbea67d7c75an%40googlegroups.com.

[Krishna Sai Veera Reddy]

Hey Larry,

Thank you for your response! Are there any examples out in the wild on how to do this? Also the part that is confusing me is the name resolution. Do we have to add a custom name resolution policy as well so that we get both the LB and round robin IP addresses in `acceptResolvedAddresses` method in the LB policy?

[Larry Safran]

Yes, you will need a custom NameResolver.  

You could have your NameResolver do one of the following

1)  Extend DnsNameResolver (as is done by the internal GrpcNameResolver) and override the doResolve method

2)  Directly extend NameResolver and have a field with a DnsNameResolver created through the DnsNameResolverProvider (which should be gotten from the registry).  

There is already a load balancer that is switching between 2 child load balancers.  It is experimental and can be specified as "priority_experimental".  What it does is to use a specific attribute that must be set by the NameResolver.  Even if you go with your own load balancer, that would be the best way to differentiate which addresses should be used for which child.  If you want to use it as an example, it is https://github.com/grpc/grpc-java/blob/master/xds/src/main/java/io/grpc/xds/PriorityLoadBalancer.java

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/2e7b95aa-cd66-40f5-bf41-0484fa12892cn%40googlegroups.com.

[Terry Wilson]

Krishna,

I'll mention that we also have a short guide on custom LBs that can help you with the fundamentals.

[Krishna Sai Veera Reddy]

Will take a look. Thank you Larry and Terry! Much appreciated!