Eureka as a DNS provider?

[Karthik]

Question on behalf of Guillaume Berche.

Wondering whether there is any eureka clients packaged as DNS resolvers so that legacy applications can benefit from Eureka mid-tier load balancing without source-level changes?

For java legacy application (say one using JDBC drivers, or the maven client library), the eureka client could be wrapped into an additional custom JVM DNS service provider (similar to http://www.xbill.org/dnsjava/dnsjava-current/README) and registered at JVM startup using the sun.net.spi.nameservice.provider.1 JVM properties.

For legacy C programs, one could imagine register a wrapper Eureka client in a dynamic library that overloads the gethostbyname() system calls that gets registered in the LD_LIBRARY_PATH.

Of course, such Eureka wrapper as DNS resolvers won't expose Eureka additional capabilities beyond name resolution (e.g. metadata), but may largely increase the range of existing applications/libraries that can relyin on Eureka mid-tier load balancing.

[Karthik]

This is an interesting idea.  Do you have requirements for which eureka can be useful as a DNS provider?

Also, about the legacy java application, is this the app for which you do not have control over initializing eureka client that you want to do lookups in a eureka-client agnostic way?

[Guillaume Berche]

Thanks Karthik for proxying my comment from http://techblog.netflix.com/2012/09/eureka.html

This is an interesting idea.  Do you have requirements for which eureka can be useful as a DNS provider?

Use-cases into which eureka can be useful as a JVM local DNS service provider:

1- a three-tier application (ELB, Tomcat, mysql cluster with multiple read replicas). Eureka gets used by the tomcat application to dynamically lookup mysql read replicas within the cluster. The mysql read replicas can come can go (possibly using an ASG). The standard JDBC driver would normally resolve an eureka-specific DNS entry (say "read.myslcluster.eureka"). This DNS resolution would in turn be handed to Eureka local JVM DNS resolver that would lookup a corresponding eureka application, and on  round-robin maneer return the next InstanceInfo available.

On the mysql replica-side, an eureka client is installed and actively register with the eureka cluster for the "read.myslcluster.eureka" application. A mini http server is started up to publish a health url as an HTTP endpoint (unless if the health url format at http://netflix.github.com/eureka/javadoc/eureka-client/com/netflix/appinfo/EurekaInstanceConfig.html#getHealthCheckUrl() supports other protocols than http such as TCP:3306 or mysql:)

This way the tomcat layer benefits from automatic fail over (within read replicas or across AWS AZ or regions) through the DNS resolution delegating to Eureka cluster.

2- an SOA-based application, relying on SOAP webservice backends.

Instead of adapting the SOAP stack (e.g. CXF) to integrate with eureka for performing mid-tier load-balancing (similar to what netflix done with the jersey REST client stack), legacy applications could run unmodified CXF stack, and only configure eureka local JVM DNS provider to resolve DNS entries (e.g. "service1.eureka") to benefit from automatic fail over, and possibly in the future additional value added features such as geo resolution (e.g. resolving to the closest application instance in terms of AZ or network latency)
 

Also, about the legacy java application, is this the app for which you do not have control over initializing eureka client that you want to do lookups in a eureka-client agnostic way?

this is either apps for which I have no control over source code (e.g. apps that I instanciate through standard binaries, such as a maven client) that I would like to benefit from Eureka-based mid-tier load balancing and fail over. Those apps would currently be deployed to leverage hardware-based level 4 load balancers such as bigIp.

Or this may be apps that I compile from source code but I wish no yet to deserve more precise lookup than a simple instance host/port resolution. Not requiring to perform source-level changes to use eureka-based mid-tier load balancing allows the same app to run unmodified with a TCP-level load balancer, or with eureka. Only the JVM configuration is changed to configure Eureka as a DNS resolution provider, and the host name to match one of the eureka-register application name.


Having these app directly DiscoveryClient.getApplication(java.lang.String appName) just to select the first returned ApplicationInstance.getHost() would be more of a burden in some cases than just performing a simple implicit DNS resolution from a well known host name such as "service.eureka".

Guillaume.
 

[Karthikeyan Ranganathan]

Excellent, Thanks. Will get back to you soon.

--
 
 

[Karthikeyan Ranganathan]

Now, I understand your requirement completely. Would you be interested in contributing?

[Karthikeyan Ranganathan]

I had a couple of clarifications as well. I'm presuming you will be "pooling" your JDBC connections. When the instances come and go, you may have drop  the existing connections and create new ones for you to balance traffic?

Also, if the instances go away, it takes a couple of mins for the eureka client to know and in that scenario, there might be JDBC failures and you may have to handle that as well.

[Guillaume Berche]

Actually, the same problems issues exist when using a TCP-level load balancer such as bigIp. I'm not sure there would be difference when using eureka as a local JVM dns service provider. The standard JDBC pools offer strategies to deal with failure of the remote server, testing cnx, recycling/rebalancing cnx, e.g. on http://commons.apache.org/dbcp/configuration.html "test on borrow", "validating query" time between eviction...

[Guillaume Berche]

I'll discuss that with my team and see if if I could contribute parts of the java-based JVM local DNS service provider wrapper to Eureka. I'll will keep you posted.

In the meantime, I'd be interested to heat the community feeling about this idea, if there are similar alternatives currently available.

If someones gets started earlier, I'm definitely interested in sharing design ideas/code review/testing it.

Concerning legacy C programs, I'm lacking fresh C development skills to efficiently contribute this.

[Karthikeyan Ranganathan]

Cool. Thanks.

--
 
 

[Hubert Chen]

Along these same lines, I had considered a local eureka proxy(most likely in python). The idea is that I currently have Java and non-Java applications which have configuration options to point to external services. If I had such a proxy I could point my application to localhost:port then use the eureka python proxy to find the server host:port and do a forwarding.

This configuration would give me most of the benefits of eureka without any application level code changes(but I would have to make configuration changes to point to localhost:port). There of course is a proxy latency penalty, but that is acceptable.

Anyone considered anything similar?

hubert

It wouldn't be quite as low level as a DNS resolver level feature.

[Brian Cline]

Hi Guillaume,

I'm generally interested in this use case as well. For something totally transparent to your app layer and still performant, you might consider a PowerDNS plugin? That way it'd still be easily accessible to all apps/clients pointed at that specific DNS server, and it could still service non-Eureka lookups just fine. I'm guessing you'd want to go for a 0-5 second record TTL so client resolvers don't cache data too long beyond the time that Eureka updates its registry.

Seeing as this thread is a bit old I'm also curious if you've had the chance to do more experimentation with this yet.

Brian

[Karthikeyan Ranganathan]

I will look into this week if it is general requirement. But any ideas/contributions welcome.

--
 
 

[Guillaume Berche]

Hi Brian,

I have unfortunately not been able to prototype the JVM DNS service resolver.

Interesting idea about setting DNS entries in the DNS authority server: do you mean an eureka client that would periodically resolving DiscoveryClient.getApplication(java.lang.String appName)and upon changes in the returned InstanceInfos would update DNS A entries in a remote power DNS autority server?

This opens up the solution to other programming languages than java (without messing up with os-specific native libs), but at the expense of additional point of failure: you'd need more than one eureka client polling the eureka cluster. Then how do you resolve conflicts when they are out-of-sync ?

Guillaume.

--
 
 

[Karthikeyan Ranganathan]

Guillaume,

I'm missing your point about the need for more than one eureka client polling from the eureka cluster.

Can you please elaborate?

--
You received this message because you are subscribed to the Google Groups "eureka_netflix" group.
To unsubscribe from this group, send email to eureka_netfli...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Guillaume Berche]

If multiple servers (e.g. web application servers in a cluster) would be relying on a power DNS to lookup a DNS entry (say the mysql master), then it is important that the DNS entry be updated frequently, and avoid single point of failures.

In addition to have the power DNS authority server be redundant to handle the loss of one of its instance, the mechanisms that would poll the eureka cluster and set the DNS entry would need to be tolerant to failures. I was imagining that one or many eureka client instances would poll the eureka cluster to set the DNS entries in the Power DNS server, hence the need for redudancy in these clients.

Another possibility would be to inject a power DNS plugin within the eureka cluster, and rather have the cluster directly assign the DNS entry in the power DNS server. Might be what Brian was suggesting, and that I did not understand at first ?

Guillaume.

To unsubscribe from this group and stop receiving emails from it, send an email to eureka_netfli...@googlegroups.com.

[brane....@gmail.com]

On Saturday, September 15, 2012 at 1:22:27 AM UTC+2, Karthik wrote:
> Question on behalf of Guillaume Berche.
>
>
> Wondering whether there is any eureka clients packaged as DNS resolvers so that legacy applications can benefit from Eureka mid-tier load balancing without source-level changes?

There's eureka-dns-server, which can be embedded in any spring-cloud-netflix enabled service, including eureka server itself:

https://github.com/bfg/eureka-dns-server

Disclaimer: I'm the author.

Best regards,