How to do server updates on Google Compute Engine?

[GCE Fanboy]

I want to be able to keep up on server updates/patches on my Google Compute Engine instance.

In comparison, when logging into an Amazon EC2 server over a terminal, they tell you there are updates available and you simply do # yum install updates. Done!

Upon login to Google's Compute Engine (GCE), there is no indication.  When doing a # yum install updates, it goes out to check and always comes back with no updates.

From what I can gather, it may be necessary to check more or better repositories -- ???

Here's what I get when doing a yum install updates on the CentOS GCE now (default):

# yum install updates

Loaded plugins: downloadonly, fastestmirror, security

Loading mirror speeds from cached hostfile

 * base: mirror.anl.gov

 * epel: mirrors.tummy.com

 * extras: centos.chi.host-engine.com

 * updates: mirror.thelinuxfix.com

Setting up Install Process

No package updates available.

Error: Nothing to do 

-------

What am I not understanding here?

What is the best practice to be sure that the updates/patches are kept up on?

Thanks in advance to he/she who populates the answer(s).

[Wolfram.Sch...@swiss.com]

yum update

Cheers, Wolf

--
© 2014 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/122f8e71-b901-476b-abe4-268d0d4f0a15%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[GCE Fanboy]

Sorry, but that is incorrect.

yum install updates is the correct command.  It is just not finding any updates with the default repository -- thus the purpose of this post/question.

Thanks for trying though.

To post to this group, send email to gce-di...@googlegroups.com.

[Wolfram.Sch...@swiss.com]

yum(8)                                                                  yum(8)

NAME
       yum - Yellowdog Updater Modified

SYNOPSIS
       yum [options] [command] [package ...]

DESCRIPTION
       yum is an interactive, automated update program which can be used for maintaining systems using rpm

       command is one of:
        * install package1 [package2] [...]
        * update [package1] [package2] [...]
        * ... yadda yadda ...

       install
              Is used to install the latest version of a package or group of packages while ensuring that all dependencies are satisfied.  If no
              package matches the given package name(s), they are assumed to be a shell glob and any matches are then installed.

       update If run without any packages, update will update every currently installed package.  If one or more  packages  are  specified,  Yum
              will  only update the listed packages.  While updating packages, yum will ensure that all dependencies are satisfied.  If no pack-
              age matches the given package name(s), they are assumed to be a shell glob and any matches are then installed.

              If the --obsoletes flag is present yum will include package obsoletes in its calculations - this makes it better  for  distro-ver-
              sion changes, for example: upgrading from somelinux 8.0 to somelinux 9.

_____________________________________
Wolf Schlickenrieder
Senior Manager
Head of Revenue Management Innovation Lab

Swiss International Air Lines Ltd.
P.O. Box ZRHLX / KRDI / WSCH
CH-8058 Zurich Airport
Phone +41 44 564 86 37
Fax +41 44 564 86 09
wolfram.sch...@swiss.com
SWISS.COM

---

From: GCE Fanboy [gr...@vitalelement.com]
Sent: 15 June 2014 17:19
To: gce-dis...@googlegroups.com
Cc: gr...@vitalelement.com; SCHLICKENRIEDER, WOLFRAM
Subject: Re: [gce-discussion] How to do server updates on Google Compute Engine?

[Evan Anderson]

The GCE default images have nightly automatic updates enabled via the 'yum-cron' package.

See this Centos forum post for more details on the process: https://www.centos.org/forums/viewtopic.php?t=4296

To post to this group, send email to gce-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/863AE9167659164895FB36D83CC97F351300BFC6%40SW-FRAADS-MBX38.ads.dlh.de.

[Evan Anderson]

Caveat: I don't actually build/manage the GCE images, so these represent my best understanding; if you need more details, I can try to dig up the current ground truth.

If you're concerned that these properties of the image may change, it should be safe (idempotent) to re-apply the automatic updates configuration.

On Mon Jun 16 2014 at 10:25:37 AM, Gr...@VitalElement.com <gr...@vitalelement.com> wrote:

Thank you for your feedback Evan.

Based on your information, are any/all the following true?

1. By default/automatically, Google Compute Engine instances, in this case a CentOS version of the OS, will update any/all packages daily.  Users do not control this, nor do they need to manually keep up with updates and patches as they do when using an Amazon EC2 instance.

NOTE: This is not pertaining to new instances created.  This question pertains to instances that are live, running, and had been spun up in the distant past.

The images created by Google for GCE have automatic updates enabled, where possible.  These rely on the mechanisms provided by the distributions (CentOS, in this case), and may be disabled by user if desired.  (i.e. you can remove or modify the 'yum-cron' configuration to not update certain packages if you want.)

By default, Google tries to ensure that automatic updates are enabled on Google-provided images where "safe" (which is the default for most distributions), because of your next point around security updates.

Note that this might not include rebooting for kernel updates, depending on the distro.  (I don't know what the current status of this is.)

2. The repository chosen by Google is comprehensive and covers installed modules and additional services such as openSSL. Thus, we can be assured that any/all recent openSSL updates are/were updated automatically upon or near their release.

Google uses or mirrors the official distribution repositories, including (for example) the debian-security repos.  In the case of the recent OpenSSL issues, I believe that these should have been automatically updated within 24 hours once available.  (I think the automatic updates run once a day; again, this is without doing much research.)

3. There is no need to add or update the repository referenced for updates and no need to manually update anything on a Google Compute Engine instance - running.

Assuming you're happy with the upstream CentOS/Debian/SuSE/RedHat repositories and don't need to install other software or point to other repos, then you should be covered.  All four distros do a pretty good job of keeping up to date and patching security problems as they are announced, and you should be able to find more info about security patches for all of them in the CVE database: http://cve.mitre.org/

Thanks again for your expertise and attention to this question.

Regards,

[Evan Anderson]

On Mon Jun 16 2014 at 11:14:08 AM, Evan Anderson <arg...@google.com> wrote:

Caveat: I don't actually build/manage the GCE images, so these represent my best understanding; if you need more details, I can try to dig up the current ground truth.

I chatted briefly with one of the engineers who actually builds the images, and he had a few corrections.  Updating inline:

If you're concerned that these properties of the image may change, it should be safe (idempotent) to re-apply the automatic updates configuration.

On Mon Jun 16 2014 at 10:25:37 AM, Gr...@VitalElement.com <gr...@vitalelement.com> wrote:

Thank you for your feedback Evan.

Based on your information, are any/all the following true?

1. By default/automatically, Google Compute Engine instances, in this case a CentOS version of the OS, will update any/all packages daily.  Users do not control this, nor do they need to manually keep up with updates and patches as they do when using an Amazon EC2 instance.

NOTE: This is not pertaining to new instances created.  This question pertains to instances that are live, running, and had been spun up in the distant past.

The images created by Google for GCE have automatic updates enabled, where possible.  These rely on the mechanisms provided by the distributions (CentOS, in this case), and may be disabled by user if desired.  (i.e. you can remove or modify the 'yum-cron' configuration to not update certain packages if you want.)

It turns out that RHEL and CentOS have automatic updates configured, but Debian does not.  Note that on RHEL and CentOS, installing updates DOES NOT restart currently-running servers, while on Debian updates restart associated services.

Until applications which depend on the vulnerable libraries have been restarted, the applications may still be vulnerable even if the on-disk libraries have been patched.  Our security bulletins recommend a reboot to ensure that all applications have picked up the new libraries; obviously, if you know all the running applications that were vulnerable, you can simply restart those apps 'sudo service <foo> restart'.

The security bulletins for the GCE images are here:

https://developers.google.com/compute/docs/security-bulletins

Note that this might not include rebooting for kernel updates, depending on the distro.  (I don't know what the current status of this is.)

Confirmed that this does not include reboots for kernel updates.

2. The repository chosen by Google is comprehensive and covers installed modules and additional services such as openSSL. Thus, we can be assured that any/all recent openSSL updates are/were updated automatically upon or near their release.

Google uses or mirrors the official distribution repositories, including (for example) the debian-security repos.  In the case of the recent OpenSSL issues, I believe that these should have been automatically updated within 24 hours once available.  (I think the automatic updates run once a day; again, this is without doing much research.)

This is accurate; in particular, GCE is working closely with the distributions to ensure that our provided images align closely with each distro's recommended configurations.

3. There is no need to add or update the repository referenced for updates and no need to manually update anything on a Google Compute Engine instance - running.

Assuming you're happy with the upstream CentOS/Debian/SuSE/RedHat repositories and don't need to install other software or point to other repos, then you should be covered.  All four distros do a pretty good job of keeping up to date and patching security problems as they are announced, and you should be able to find more info about security patches for all of them in the CVE database: http://cve.mitre.org/

Thanks again for your expertise and attention to this question.

Regards,

On Mon, Jun 16, 2014 at 1:01 PM, Evan Anderson <arg...@google.com> wrote:

The GCE default images have nightly automatic updates enabled via the 'yum-cron' package.

See this Centos forum post for more details on the process: https://www.centos.org/forums/viewtopic.php?t=4296

wolfram.schlickenrieder@swiss.com
SWISS.COM

---

From: GCE Fanboy [gr...@vitalelement.com]
Sent: 15 June 2014 17:19

To: gce-discussion@googlegroups.com

Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.

---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.

[gr...@vitalelement.com]

Thank you for your follow up Evan -- much appreciated.

To clarify + questions:

1. On any given GCE, CentOS/Apache and module updates and patches are automatically applied daily by default via yum-cron  -- unless disabled by root user.

2. Based on the automatic daily server updates from the yum-cron, any time a root ssh command of yum install updates or yum update is run, su will always get at "No package updates available" and "No packages marked for Update" respectively.  This is due to the fact that the updates were already done by the instance within 24 hours of any given attempt. 

3. Restarting Apache (httpd) should be all that is needed to load any new updates/patches, NOT at total machine/server reboot.  Correct?

We are moving from Amazon EC2 to Google's Compute Engine and need to be sure our 'policies and procedures' for our new server admin are clear and accurate.  Your assistance and feedback is truly appreciated.

Best regards,

[Jimmy Kaplowitz]

Hi Greg,

Answers inline below.

On Jun 17, 2014 5:16 AM, "Gr...@VitalElement.com" <gr...@vitalelement.com> wrote:
> 1. On any given GCE, CentOS/Apache and module updates and patches are automatically applied daily by default via yum-cron  -- unless disabled by root user.

This is currently true for our CentOS images as well as our Red Hat Enterprise Linux images, yes.

> 2. Based on the automatic daily server updates from the yum-cron, any time a root ssh command of yum install updates or yum update is run, su will always get at "No package updates available" and "No packages marked for Update" respectively.  This is due to the fact that the updates were already done by the instance within 24 hours of any given attempt. 

That outcome will only occur if no updates have been made available since the last time packages have been updated. While the daily updates means this will commonly be the case, updates might be available if they were released within the last day or if the instance has not yet run its first daily round of package updates.

> 3. Restarting Apache (httpd) should be all that is needed to load any new updates/patches, NOT at total machine/server reboot.  Correct?

This is true for updates to Apache. However, any component of the system might be updated and need to be restarted, such as the OpenSSH server or the kernel. In the case of a kernel update, a total machine/server reboot is necessary to activate the new kernel. This is true in most Linux environments, not just Google Compute Engine.

> We are moving from Amazon EC2 to Google's Compute Engine and need to be sure our 'policies and procedures' for our new server admin are clear and accurate.  Your assistance and feedback is truly appreciated.

I hope this helps!

- Jimmy

[gr...@vitalelement.com]

Hello Jimmy,

Your clear/detailed feedback is awesome!  These answers have been elusive since my questioning back in June.  Thus, your reply came as a pleasant surprise in my email this morning.

I've been checking the yum.log to see if the dates auto-update/apply and now see that there are updates being automatically processed.  This is pleasing to know.

We'll write into our GCE documentation that 'Apache restarts' and 'machine reboots' will need to be done at regular intervals -- to be sure all updates/patches are applied.

One concern is that machine reboots on GCE's will not restart necessary services.  We noticed that many application critical features are OFF by default on GCE and need to be turned on manually.  We set some to be 'persistent', but will need to be assured that machine reboot's do not turn off services like Java, (remote database) network connect, and sendmail.

We've applied these (below) via ssh but have yet to test if they automatically turn on with a machine reboot.  If you have any suggestions/background regarding getting a machine to boot back to its previous most recent state, that would be helpful.

We've done these on GCE.  Of note, this process is not necessary for Amazon EC2 CentOS instances as they are ON by default and persistent upon reboot.

setsebool -P httpd_execmem 1

setsebool -P httpd_can_network_connect_db 1

setsebool -P httpd_can_sendmail 1

Thank you again,

Greg J.

Greg Johnson

Vital Element, Inc.