GCE instance keeps changing hostname and /etc/hosts file

[Giovanni Francesco]

Hello everyone,

This is similar to another topic I found in this group (https://groups.google.com/forum/#!topic/gce-discussion/olG_nXZ-Jaw) - however since the solutions in that thread did not work for me I am posting here.

I have GCE instance that was deployed using the CentOS 7.3 image.

[root@cponly2 dhcp]# ls -lah

total 16K

drwxr-x---.  3 root root  23 Dec 23 23:47 .

drwxr-xr-x. 87 root root 12K Dec 23 23:44 ..

drwxr-xr-x.  2 root root  45 Dec 23 23:41 dhclient.d

[root@cponly2 dhcp]# ls -lah dhclient.d/

total 8.0K

drwxr-xr-x. 2 root root   45 Dec 23 23:41 .

drwxr-x---. 3 root root   23 Dec 23 23:47 ..

-rwxr-xr-x. 1 root root 2.2K Nov 14 16:18 ntp.sh

-rw-r--r--. 1 root root   41 Dec 23 23:41 set_gfm_hostname.sh <<< this is a file that contains "hostname myownFQDN" 

However when I restart networking services the hostname continues to be the GCE internal one. As you can see above /etc/dhcp/ folder contains no "hooks" or other scripts, there was a set_googlehostname one that I deleted in an attempt to avoid the hostname being reset.

I really need to override the hostname that GCE is forcefully setting to this virtual machine as it is impacting my SMTP/Mailjet settings among other issues. Yes I could set a cronjob that runs every 5 minutes to get around the issue but I would like to find the root of the problem and either strip the code away to prevent the changes from happening or another solution.

/etc/hosts (is being modified and new lines come up "#Added by Google" ) < this is annoying, I even tried to comment the line with a # at the beginning but whatever script runs is smart enough to remove the commented # (I took that advice from a StackOverflow question).

Someone here even said that the FQDN/hostname setting could be set via metadata setting for that specific VM, but the link posted is broken and I found zero docs or examples on doing so (http://stackoverflow.com/questions/25408612/google-compute-engine-how-to-set-hostname-permanently)

Thanks in advance for all the help

Giovanni

[Giovanni Francesco]

Cross referencing to my question on Stackoverflow: http://stackoverflow.com/questions/41309488/hostname-and-etc-hosts-keep-changing-to-instance-project-name-after-network-res

[George (Google Cloud Support)]

Hello Giovanni,

Copied the answer from the Stackoverflow link you provided as it still applies. The following article explains that the "hostname" is part of the default metadata entries and it is not possible to manually edit any of the default metadata pairs. As such, you would need to use a script or a cron job as you mentioned to change the hostname every time the system restarts, otherwise it will automatically get re-synced with the metadata server on every reboot.

You can find information on startup scripts for GCE in this article. You can visit this one for info on how to apply the script to an instance.

I hope this helps.

Sincerely,

George

[Giovanni Francesco]

Thank you. I figured out how to use custom metadata and a cronjob to reset the hostname and fix this behavior that was breaking my application.

I have shared the details on my blog for those that are looking for a guide: https://goo.gl/rli5BF

[James Tt]

ive tried setting metadata and setting up crontab with @reboot task as required.

when the server boot up, it did reset hostname to the desired hostname,

but, somehow, after running server for several days without any reboot (6days later),

the hostname automatically reset back to the original hostname .

what could be the cause?

[James Maina]

Hey Giovanni

Does the DHCP change the hostname every 24 hours in your case? How did you resolve that? I have set the cron job which works on restart but now server is not restarting but hostname keeps changing due to the DHCP lease. Any idea?

Thanks

James

[Richard Terry]

I found this solution to be more helpful than the cronjob 

https://groups.google.com/d/msg/gce-discussion/olG_nXZ-Jaw/Y9HMl4mlBwAJ

On Friday, December 23, 2016 at 7:46:42 PM UTC-5, Giovanni Francesco wrote:

[Todd Grayson]

None of this is working, including the linked discussion.  We need someone from google engineering to produce an authoritative blog.  The terse answer and zero follow up with questions from that link do not produce repeatable results.

We are facing hundreds of clusters being invalidated as google slams our naming and kerberos and TLS configuration is rendered useless (and all certificates useless)

How do we do this... how do we get the hosts to properly return their name?  

[Sirui Sun]

We are actively working on a feature which will allow you to change VM hostname. We plan to make hostname a property on the VM that can be set via UI/gcloud/API, without the need to fiddle with the OS settings or change Metadata Service properties.

In the meantime, we'd love to hear your feedback on what you'd like to see in this feature. That will help us build the right thing for you.

Some specific questions to help kick this off. Please feel free to reach out privately as well - I'm all ears.

• How important is it for you to change hostname (currently set to VM name) vs. domain (currently c.[project_id].internal or [zone].c.[project_id].internal)
  
• How important is it for you to change hostname/domain at VM creation time vs. modify hostname/domain for an existing VM?
• Do you take a dependency on the Google Cloud Internal DNS service?

Thanks,

Sirui Sun

Product Manager | Google Compute Engine

[Gerardo Santoveña]

Hello,

In my case, I have been working with a client that has part of its infrastructure in GCP, and the servers are being provisioned using Chef.

When the hostname is changed and /etc/hosts is updated through the /etc/dhcp/dhclient-exit-hooks, the client key fails to authenticate to the Chef server and we need to manually modify the /etc/hosts to have chef-client working again.

I'm still trying to find a way to avoid this behavior.

Thanks for any suggestion.

Kind regards,

Gerardo Santoveña

[James Tan]

For my side of usage, cpanel license is bounded by hostname, therefore having it changed will causes license to be invalidated.

Yes allowing change of hostname is important as there are times that we may decide to upgrade another server with the original image and promote this new server as the original server hostname. Old server hostname will then be renamed to another name.

Thank you :)

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/fi2j7oP8bGM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/d4dd645b-7aea-4f76-ac27-13ee0561ec29%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

------------------
If any attachment content power point (.pps / .ppt), please use google slideshow to open for safe precaution.

[Sirui Sun]

Update: we've recently released a feature which which allows you to specify a custom FQDN hostname as an optional parameter when creating a VM. This feature is currently in Beta. 

$ gcloud beta compute instances create INSTANCE_NAME --hostname example.hostname

This should work across OSes, and eliminate the need for workaround scripts.

More info in the docs.

Please try it out, and let us know if it works for you.

Sirui Sun

Product Manager | Google Compute Engine

[sourav rakshit]

Hi ,

I tried with the said command. What I observed is its creating vm with the desired fqdn I am mentioning but when I am doing nslookup with ip its mapping from the internal DNS .

/etc/hosts : -

127.0.0.1 localhost::1 localhost ip6-localhost ip6-loopbackff02::1 ip6-allnodesff02::2 ip6-allrouters10.148.0.2 test1.lowes.com test1 # Added by Google169.254.169.254 metadata.google.internal # Added by Google

nslookup :-

root@test1:~# nslookup 10.148.0.2Server: 169.254.169.254Address: 169.254.169.254#53Non-authoritative answer:2.0.148.10.in-addr.arpa name = test1.asia-southeast1-b.c.genuine-ether-221610.internal.Authoritative answers can be found from:

[Sirui Sun]

Hi Sourav - 

What you're seeing is expected. From the documentation for this feature:

VPC networks have an internal DNS service that can be used to access instances by using the default fully qualified domain name (FQDN) hostnames. However, the internal DNS service cannot resolve custom hostnames.

We are considering adding Internal DNS support for custom hostnames in the future.

Could you provide some more details on your use case, and why you need Internal DNS to resolve to your custom hostname? That will help us prioritize the work mentioned above.

Thanks,

-Sirui Sun

Product Manager | Google Compute Engine

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/fi2j7oP8bGM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/9e4d3111-937e-41bb-a289-6eff18e34f6a%40googlegroups.com.

[sourav rakshit]

Hi Sirui,

We have build Cassandra cluster on GCP. And we need to set SSL configuration for our Cassandra cluster , but after getting it signed from our organisation. we found the handshaking between the nodes through ssl certs takes place with the FQDN of the instances.

The handshaking between the nodes fails because the cn name provided by me was with extension of our domain  *.lowes.com but gcp internal dns resolves with *.internal WRT zonal or global.

Hope I am able to make you understand our use case. we need to modify the FQDN with extension of *.lowes.com which will help to spin up instance on our vpc and get SSL signed from our organisation.

[Sirui Sun]

Hi Sourav - 

Thanks for the information. As mentioned before, we are looking into having GCP's internal DNS support addressing instances with custom hostnames, but it is not currently possible.

In the meantime, if it is required that your instances address each other on the network with FQDNs on the *.lowes.com domain, I would recommend looking into setting up your own DNS server which resolves your instances' FQDNs with their IP addresses.

-Sirui 

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/6d00ebfb-f91c-4b63-90be-ca138ba3b328%40googlegroups.com.

[sourav rakshit]

Hi Sirui,

Thanks for the update.But the scenario I provided is something we are working on micro services on GCP , So can you help me in knowing that by when we can expect the same feature which will help us in our production before  we go live .

Regards,

Sourabh

[Sirui Sun]

Hi Sourabh - 

Unfortunately we don't yet have an ETA for this, but I will update this thread when we do.

-Sirui 

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/0846ec59-7606-4b10-bbb8-196da001a79b%40googlegroups.com.

[James]

Hi Sirui,

Do you have any suggestion for exists VMs? 

We have over 200 VMs are already running on production.

Thanks.

Sirui Sun於 2018年11月10日星期六 UTC+8上午2時59分04秒寫道：

[Sirui Sun]

Hi James - 

If you need to change the hostnames for your existing VMs, then your best bet is to manually modify the DHCP exit hooks as suggested earlier in this thread.

We are looking into enabling modifying hostnames for existing VMs as part of our custom hostnames feature. We will take this feedback into consideration. However, I cannot offer any commitments or timelines.

Thanks,

-Sirui Sun

Product Manager | Google Compute Engine

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/fi2j7oP8bGM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/6643bb0f-b27e-4bd6-82fe-fae89a77bb7a%40googlegroups.com.

[James]

HI Sirui,

Thanks for your reply.

I tried CentOS 7 and 6, that still not works at all.

Do we have any workaround in the official document? 

Thanks.

[Taame (Google Cloud Support)]

Hi, 

We don’t have any workaround in the official document for Centos. 

If you could share the steps you followed and the error messages you encountered to help us try to reproduce and see if we can help provide any assistance based on best efforts.

[James]

HI Taame,

I created a bash script located in /etc/dhcp/dhclient-exit-hooks.d/set-custom-hostname.sh

#!/bin/bash

hostname my-hostname.test.com

But after rebooted, the hostname is the old one.

Thanks.

[Anthony (Google Cloud Support)]

Hi James,

As Sirui has mentioned, at this very moment what you're experiencing is expected behavior. Enabling the modification of hostnames for existing VMs is part of our custom hostnames feature, however we cannot guarentee an ETA as when this feature will be implemented.

[Stanislas Lange]

Is there any news on this?

[Richard Terry]

I am was able to make a script to run on start up and exit that sets the hostname as previously describe and even linked a cron that sets it once every 6 to 12 hours depending on the instance and time zone.

On Friday, December 23, 2016 at 7:46:42 PM UTC-5, Giovanni Francesco wrote:

[Gio]

- As an update to this issue from the OP:

I ended up migrating my VMs to Microsoft Azure since they do support my custom hostnames without having to hack around or create cronjobs. It's unfortunate that Google is unable other major VM provider's offerings due to some custom "Google-way" of doing things. 

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/fi2j7oP8bGM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussio...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/1af7aaeb-5e2b-4445-973d-71399abde232%40googlegroups.com.