Why does eureka-client store hostnames in the registry

[Andrew Spyker]

In our work to port Eureka to the IBM Cloud (SoftLayer) [1], we found that eureka-client registers to eureka-server with hostname [2].  In SoftLayer the IP addresses of the private interface aren't automatically resolvable in internal DNS.  Due to this we changed the code [3] to provide the IP address instead when registering and it has worked well for Eureka and Ribbon.

I've noticed on EC2 that, even through all instances are in some internal DNS, not all instance names are resolvable from all other instances [4].  It would seem like this would cause issues with Eureka in general in EC2.  In fact we've seen when a eureka-client uses ribbon to connect to another eureka advertised service with one using ip-* hostname and the other using domU-* hostname the ribbon call fails as the caller can't resolve the domU-* address as shown in the gist.

I was wondering why in Eureka, if it is intended to advertise only internal nodes, that you didn't just use ip addresses instead of using hostnames and relying on DNS?

For what it's worth, you can see the difference in a registration in Eureka in our system (using ip addresses) vs. in yours in EC2 (using hostnames) in this gist [5].

[1] - https://github.com/EmergingTechnologyInstitute/SoftLayer-NetflixOSS/wiki/Micro-service-location-services-documentation#wiki-eureka-client

[2] - https://github.com/Netflix/eureka/blob/master/eureka-client/src/main/java/com/netflix/appinfo/AbstractInstanceConfig.java#L220

[3] - https://github.com/EmergingTechnologyInstitute/eureka/blob/master/eureka-client/src/main/java/com/netflix/appinfo/AbstractInstanceConfig.java#L229

[4] - https://gist.github.com/aspyker/6498193

[5] - https://gist.github.com/aspyker/8606796

[Nitesh Kant]

Hi Andrew,

Eureka uses ec2 metadata API available on all instances, to get the private ip of the instance and then adds the same to the metadata of the instance, when registering to the eureka server. The relevant code is here:

https://github.com/Netflix/eureka/blob/master/eureka-client/src/main/java/com/netflix/appinfo/AmazonInfo.java

Internally inside Netflix, applications can choose to use this private IP or the hostname whichever they deem appropriate. I am not sure whether this feature was ported to the OSS ribbon. If you are interested, I can get you the answer for that.

Why eureka primarily uses public IP, is just for historic reasons.

--
You received this message because you are subscribed to the Google Groups "eureka_netflix" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eureka_netfli...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Andrew Spyker]

Sorry it took so long to get back to this.  I don't think this has much to do with public vs. private ip addresses.  It is more of an issue of using hostnames vs. IP's.

This is what I found (using all defaults for Ribbon and Eureka not specifying a specific datacenter type):

Eureka -

PropertiesInstanceConfig and AbstractInstanceConfig

- Captures service instance data via AbstractInfoConfig.getHostInfo() - hostname via InetAddress.getLocalHost().getHostName(), ip address via InetAddress.getLocalHost().getHostAddress()

- stores host address in InstanceInfo.ipAddr, stores host name in InstanceInfo.hostName, also defaults the various urls (homepage, statuspage, healthcheck) based on the host name (via ${eureka.hostname} replacement in InstanceInfo$Builder

- So by default a majority of the service instance information (except for ipAddr) in Eureka is based on host name

Ribbon (ribbon-eureka specifically) -

DiscoveryEnabledNIWSServerList.obtainServersViaDiscovery

- Queries Eureka based on vipAddress, gets a list of InstanceInfo objects, and creates a DiscoveryEnabledServer (extends Ribbon's Server class) list

- When DiscoveryEnabledServer is constructed it calls the constructor with the InstanceInfo host name and port (and secure port) discovered out of Eureka

- This means when Ribbon load balances by default it does hostname1:port, hostname2, port (vs. ip1:port, ip2:port) based routing

This means that by default all Eureka enabled Ribbon routing is dependent on DNS names gathered on service instances being resolvable on all callers.  I don't think the AmazonInfo class really helps this as while it provides more information (as does our DataCenter extension for SoftLayer), it doesn't change how Ribbon interacts with the returned Eureka information.

Can you confirm that your Netflix specific Ribbon plugins route based on something other than InstanceInfo.hostname or AmazonInfo.publicHostName?  I'd have to imagine based on the issues I've seen with DNS in Amazon that you'd want to route more on AmazonInfo.local-ipv4.

Also, this applies for the URL's as well.  For Asgard and Eureka console to work with the URL's specified (statuspage, homepage, healthcheck url), you'd think you'd use IP's as well.  Can you clarify if there are additional changes to Asgard and Eureka console to somehow pay attention to ip addresses vs. hostnames?

In general after researching I'm confused why Eureka as a discovery service for micro services would encourage use of DNS hostnames as it then adds that as a dependency that is outside of the service discovery contract between the service query client and the service implementation registration.  I think Ribbon is purely a user of the data and not really the issue.  If I understand this right, I'd suggest (as we've done in our changes for SoftLayer) that:

All connectivity information in Eureka should be ip address based with metadata about service instance locally known hostnames.  Likely this metadata about hostname would never be used unless services were somehow DNS load balanced.  All Eureka based routing would be based on the ip addresses stored in place of the hostnames.

To unsubscribe from this group and stop receiving emails from it, send an email to eureka_netflix+unsubscribe@googlegroups.com.

[Andrew Spyker]

Based on our conversation, I learned that in Netflix you don't use the hostName info from Eureka (instead using the ipAddr field).

I submitted a pull request to ribbon-eureka for a Archaius flag to have the same behavior available in OSS:

https://github.com/Netflix/ribbon/pull/102

This fixes 1/2 of the issue (Ribbon used of Eureka information).

The other 1/2 of the problem with the statuspage, homepage, healthcheck url all coming in based on hostName still exists.  I think that would have to be fixed in eureka-client.  Will look at that soon.

[Andrew Spyker]

I have submitted two pull requests to Ribbon to address this (one for 1.x and one for 2.x) and they have been accepted.

Once they make it to releases there will be a property to control this per ribbon client or globally for ribbon called UseIpAddrForServer.  Both hostname and ipAddr will be in eureka, but the ribbon client will use the ipAddr directly when making calls.

[exhe...@gmail.com]

Se, may this help you:
https://github.com/Netflix/Prana/issues/26

[abhin...@gmail.com]

Hi Andrew

We are trying to discover micro services through eureka , for JVM we did.. facing issues with python and c++ rest services(through registrar we did but still port and ip are static ).

I having few question regarding this.

1.We dont want to maintain IP and port of services , so for java(Spring boot)we are able to achieve it but for Non Java .. still facing problem.
although through registrar we are able to achieve it but here we are maintaining IP and port into eureka-client.properties file.

Could you please help me to find out the answer of above question .

Thanks in advance !!

Regards,
Abhinav