[C++/C#] Directly targeting grpclb

[falco...@gmail.com]

Is there any way to directly target GRPCLB right now? I know about the gRFC A5 (Encoding grpclb info in DNS) which will work when using SRV records, but I am using Windows and therefore also have no c-ares support as of now. Also I don't want to setup and manage a DNS server just for this entry. I have an open issue about my base problem with more details here: https://github.com/grpc/grpc/issues/11879

The workaround was planed as using GRPCLB for service discovery. It would already work if it is possible to use GRPCLB without DNS. Like having a name resolver plugin for GRPCLB without going through DNS to find the load balancer which is already well-known. Like grpclb://<ip>:<port>/<endpointname> where ip and port belong to the GRPCLB service.

[falco...@gmail.com]

Also to clarify it some more: The list of services known can change during runtime. So I would need to keep the DNS zone file always updated to include my endpoints and also add SRV records for the load balancer. This should definetly be doable using coreDNS, automatically updating the Corefile and reloading it, but since my process includes the GRPCLB service and that one already knows the endpoints, it would be a huge workaround including more tools than needed... Since all modules managed by one process share the same load balancer, I would prefer to just address the GRPCLB service directly instead of it being looked up from the SRV record for the original target. So the goal is a bit different from what gRFC A5 targeted.

So all my endpoints sharing one GRPCLB run in one process. The GRPCLB endpoint (IP + Port) is fix, the services hosted by the process choose free ports on startup and can change, why I want to use endpoint names to address the services when talking to them from a different service (outside from this process and most commonly on another machine).

[falco...@gmail.com]

No one working with GRPCLB here?

[David Garcia Quintas]

The only official way of enabling gRPCLB is, as you found out, as described in the gRFC A5. This could be (locally) hacked up to forcefully enable gRPCLB, BUT wouldn't it be simpler to implement everything at the application level? That is, do all the service discovery on a regular grpc server, process its responses and then proceed. Re-purposing gRPCLB for this end is essentially that, but having to jump through extra hoops to enable it, having to implement an API (load_balancer.proto) that's not exactly designed for your usecase, etc.

[falco...@gmail.com]

If done at the application level (like I do it right now due to no other options), I lose every benefit of the channels like rediscovery in case the endpoint changes (which can and will happen). Instead it will try to reconnect to the old endpoint, waiting for it to return, which will not happen. That’s why I hoped to use the existing core features of gRPCLB.

The only service with fixed port is my main service which right now works as nameservice and manages my modules.

[David Garcia Quintas]

Well, it may be that c-ares finally works on windows (worth a nudge on https://github.com/grpc/grpc/pull/12416). Barring that, or until that happens, you have two options, both non-trivial and requiring playing with and building the c-core code:

- (HACK HACK HACK) Modify the logic around here to enable gRPCLB based on any condition you like. This is evil and of course not officially supported :)

- Write custom name resolver for the C Core. Something we've been wanting to do for a while is expose a semi-public API to implement custom resolvers (still in the core, in C/C++) but haven't gotten around to doing it. Given that your needs revolve around DNS, you wouldn't have to start from scratch. Simply copy dns_resolver.cc, rename it and hack it up in any way you need, then enable it for you deployment (in the same way that the standard DNS resolver is registered. I/we can help you with this). While we don't have a public resolver API, the current one hasn't changed in quite a while, so having your needs encapsulated in your custom resolver would be the cleanest and more future-proof way of doing things.

[falco...@gmail.com]

Thanks for the feedback. I will look into writing a resolver and do a PR, since I have seen others with similar needs. In Java it's pretty easy to do what I planed, so maybe it helps others in the C++/C# world too. The nice thing is, that when it's done as resolver in core, it's also usable from C# without any needs of changing the C# implementation.

Once c-ares works with windows, I could change my gRPCLB endpoint into a DNS server if I don't need load balacing, right? But since I need both, I would then still need to add DNS information pointing to the gRPCLB endpoint for the load balancing and in the gRPCLB endpoint manage my services available. So for my use case, I think the name resolver will even then be the better option when I know the gRPCLB endpoint will not change.

[falco...@gmail.com]

I just finished implementing the grpclb resolver. Was pretty straight forward. Seems to work. But I don't receive any client stats. I defined a load_balance_token and set client_stats_reporting_interval to a duration of 5 seconds (for testing purpose). I see in the logs, that the token is used. Do I need to configure anything else?

[falco...@gmail.com]

I forgot to mention, I am using latest master (366e23b604d6458787ef8df77a2fed9c6d861f45) and based my changes on top of it. I don't think I have touched anything responsible for the reports. My changes can be seen here: https://github.com/Falco20019/grpc/commit/171441ae85053e48cb9a88495c97abeb2fd57a4d

[falco...@gmail.com]

Found my mistake. I haven't seen that LoadBalanceResponse is a oneOf and responeded with InitialResponse AND ServerList (effectively overwriting InitialResponse and setting it to null). No I also get the client stats :)

[David Garcia Quintas]

Nice! 

On 16 November 2017 at 03:38, <falco...@gmail.com> wrote:

Found my mistake. I haven't seen that LoadBalanceResponse is a oneOf and responeded with InitialResponse AND ServerList (effectively overwriting InitialResponse and setting it to null). No I also get the client stats :)

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/719ab63b-ae11-4897-b7ca-6ce5d1a8201c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.