Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Experiences with KVM clock drift?

1,117 views
Skip to first unread message

Martin McClure

unread,
Oct 14, 2016, 7:32:51 PM10/14/16
to ganeti
I'm noticing that some (but apparently not all) of my Linux instances
are experiencing clock drift with respect to the node's clock, which is
NTP-synchronized. I've verified that the affected instances are using
the kvm-clock for clocksource.

These instances are not running NTP, as I'd thought that unnecessary.
Further research shows that advice is mixed on this:
https://s19n.net/articles/2011/kvm_clock.html

Do any of you have advice based on your experience?

Thanks,

-Martin

ge...@riseup.net

unread,
Oct 15, 2016, 5:19:51 AM10/15/16
to gan...@googlegroups.com
On 16-10-14 16:32:49, Martin McClure wrote:
> Do any of you have advice based on your experience?

I'm running NTP in all my instances on different clusters, using Xen and
KVM. I've never seen clock drift and / or problems regarding the time.
I'm even running two FreeSWITCH instances as a PBX, works as expected.

Cheers,
Georg
signature.asc

Randy Bush

unread,
Oct 15, 2016, 8:47:00 AM10/15/16
to ge...@riseup.net, gan...@googlegroups.com
>> Do any of you have advice based on your experience?
> I'm running NTP in all my instances on different clusters

as do i, see [0]. but the OP is running ntp on the node and the guests
are configured to use kvm-clock, and this results in guest clock drift.

i had the same problem many moons ago, which is why i went to ntp in
every guest [1]. but i sure would be interested in learning why
kvm-clock does not work.

randy


0 - https://wiki.rg.net/wiki/GuestNTP

1 - well now, for linuxes, it's this half-assed enraging systemdeath
timedatectl thing :(

George K.

unread,
Oct 15, 2016, 9:27:54 AM10/15/16
to gan...@googlegroups.com
We (GRNET) raised this issue during GanetiCon. Our setup is based on qemu-kvm and we've also noticed clock drifts at least with the qemu-versions up to 2.1 that we have tested.

While we our guests use kvm-clock as clock source we also need to run NTPd on them. What we've also noticed are big clock jumps after guest live migration and that's why we have an icinga/check_mk check to get notified of differences between hwclock and sysclock in guests. Obviously we run NTPd on hosts.

For quite a while we've also been using the following options in a kvm-wrapper script: "-global kvm-pit.lost_tick_policy=discard -rtc base=utc,driftfix=slew" in our ~okeanos setup. This seems to have put and end to complaints from mainly windows users but we haven't done extensive testing with newer kernel versions and/or newer qemu-kvm versions.

If anyone has done more extensive testing, please share your results.

Lucas, Sascha

unread,
Oct 17, 2016, 7:46:02 AM10/17/16
to gan...@googlegroups.com
Hi,

interesting topic. I also noticed time drifts in KVM instances (mostly marginal from 1-4s in 100 days). I followed the recommendations of our enterprise distro: use kvm-clock and don't use NTP.

It seems a very mysterious topic, where "many (supposedly) authoritative sources disagree about how clocks should be synced on KVM hosts and VMs".[1]

I'm as curious as you to find out what the "best" way is. Thanks, Sascha.

[1] http://lists.ntp.org/pipermail/questions/2014-October/038966.html

Aufsichtsratsvorsitzender: Norbert Rotter
Geschäftsführung: Michael Krüger
Sitz der Gesellschaft: Halle/Saale
Registergericht: Amtsgericht Stendal | Handelsregister-Nr. HRB 208414
UST-ID-Nr. DE 158253683

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Empfänger sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder des Inhalts dieser Mail sind nicht gestattet. Diese Kommunikation per E-Mail ist nicht gegen den Zugriff durch Dritte geschützt. Die GISA GmbH haftet ausdrücklich nicht für den Inhalt und die Vollständigkeit von E-Mails und den gegebenenfalls daraus entstehenden Schaden. Sollte trotz der bestehenden Viren-Schutzprogramme durch diese E-Mail ein Virus in Ihr System gelangen, so haftet die GISA GmbH - soweit gesetzlich zulässig - nicht für die hieraus entstehenden Schäden.

Ansgar Jazdzewski

unread,
Oct 18, 2016, 6:27:33 AM10/18/16
to Ganeti
Hi,

2016-10-15 15:27 GMT+02:00 George K. <kargig...@gmail.com>:
> -global kvm-pit.lost_tick_policy=discard -rtc base=utc,driftfix=slew

keep in mind that you don need to write a wrapper you can use the
kvm:kvm_extra extra potion inside ganeti

gnt-cluster modify -H kvm:kvm_extra="-smbios
type=1\,manufacturer=google\,product=ganeti -global
kvm-pit.lost_tick_policy=discard -rtc base=utc\,driftfix=slew"
Reply all
Reply to author
Forward
0 new messages