constant_tsc and TSC unstable
Paul Rolland
I've a machine with a Core2Duo CPU. /proc/cpuinfo reports the flag
constant_tsc, but at boot time, I have the log :
..
Total of 2 processors activated (6919.15 BogoMIPS).
ENABLING IO-APIC IRQs
.TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
checking TSC synchronization [CPU#0 -> CPU#1]:
Measured 3978592228 cycles TSC warp between CPUs, turning off TSC clock.
Marking TSC unstable due to: check_tsc_sync_source failed.
Brought up 2 CPUs
..
This machine is running 2.6.23.1-21.fc7. I know I should report to Fedora,
but I was wondering if this is a bug or a feature ;)
Regards,
Paul
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
H. Peter Anvin
> Hello,
>
> I've a machine with a Core2Duo CPU. /proc/cpuinfo reports the flag
> constant_tsc, but at boot time, I have the log :
>
> Total of 2 processors activated (6919.15 BogoMIPS).
> ENABLING IO-APIC IRQs
> checking TSC synchronization [CPU#0 -> CPU#1]:
> Measured 3978592228 cycles TSC warp between CPUs, turning off TSC clock.
> Marking TSC unstable due to: check_tsc_sync_source failed.
> Brought up 2 CPUs
>
> This machine is running 2.6.23.1-21.fc7. I know I should report to Fedora,
> but I was wondering if this is a bug or a feature ;)
>
constant_tsc means the TSC doesn't change with CPU core frequency.
The problem you're having is that the TSCs of your two cores are
completely different, over a second apart. This is a bug, unrelated to
constant_tsc.
-hpa
Michael Tokarev
> Paul Rolland (ポール・ロラン) wrote:
>> Measured 3978592228 cycles TSC warp between CPUs, turning off TSC clock.
>> Marking TSC unstable due to: check_tsc_sync_source failed.
>> but I was wondering if this is a bug or a feature ;)
> The problem you're having is that the TSCs of your two cores are
> completely different, over a second apart. This is a bug, unrelated to
> constant_tsc.
A bug in where - in the CPU or in kernel?
The thing is that all our dual-core machines shows something like
that.
For example, dualcore PentiumD machine:
Nov 7 20:23:56 paltus kernel: Linux version 2.6.22-i686smp (m...@paltus.tls.msk.ru) (gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)) #2.6.22.12 SMP Wed Nov 7 20:02:14 MSK 2007
..
Nov 7 20:23:56 paltus kernel: checking TSC synchronization [CPU#0 -> CPU#1]:
Nov 7 20:23:56 paltus kernel: Measured 112 cycles TSC warp between CPUs, turning off TSC clock.
Nov 7 20:23:56 paltus kernel: Marking TSC unstable due to: check_tsc_sync_source failed.
Nov 7 20:23:56 paltus kernel: Brought up 2 CPUs
(not that huge difference as Paul reported, but still "unstable".
The same happens with 2.6.23)
Note that once TSC is disabled (it's using "jiffies" as far
as I can see), ntpd constantly speeds up and slows down the
clock, it jumps +/- 0.5sec every several minutes or hours -
I guess that's when ntpd process gets moved from one core
to another for whatever reason. And an interesting thing
is that with 64bits kernel this TSC problem does not occur
on this very machine.
Something similar is reported on AMD X2 64 machines as well --
can't check right now.
Thanks.
/mjt
Patrick McHardy
> Paul Rolland (ポール・ロラン) wrote:
>> Hello,
>>
>> I've a machine with a Core2Duo CPU. /proc/cpuinfo reports the flag
>> constant_tsc, but at boot time, I have the log :
>>
>> ...
>> Total of 2 processors activated (6919.15 BogoMIPS).
>> ENABLING IO-APIC IRQs
>> ..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
>> checking TSC synchronization [CPU#0 -> CPU#1]:
>> Measured 3978592228 cycles TSC warp between CPUs, turning off TSC clock.
>> Marking TSC unstable due to: check_tsc_sync_source failed.
>> Brought up 2 CPUs
>> ...
>>
>> This machine is running 2.6.23.1-21.fc7. I know I should report to
>> Fedora,
>> but I was wondering if this is a bug or a feature ;)
>>
>
> constant_tsc means the TSC doesn't change with CPU core frequency.
>
> The problem you're having is that the TSCs of your two cores are
> completely different, over a second apart. This is a bug, unrelated
> to constant_tsc.
On my core2duo I noticed that the TSC of the second core doesn't
start counting until the CPU is activated. Since they're not
synchronized manually anymore this makes the TSC check fail.
Pallipadi, Venkatesh
>-----Original Message-----
>From: linux-ker...@vger.kernel.org
>[mailto:linux-ker...@vger.kernel.org] On Behalf Of Paul
>Rolland (???・???)
>Sent: Thursday, November 29, 2007 8:12 AM
>To: Linux Kernel
>Cc: r...@witbe.net
>Subject: constant_tsc and TSC unstable
>
>Hello,
>
>I've a machine with a Core2Duo CPU. /proc/cpuinfo reports the flag
>constant_tsc, but at boot time, I have the log :
>
>...
>Total of 2 processors activated (6919.15 BogoMIPS).
>ENABLING IO-APIC IRQs
>..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
>checking TSC synchronization [CPU#0 -> CPU#1]:
>Measured 3978592228 cycles TSC warp between CPUs, turning off
>TSC clock.
>Marking TSC unstable due to: check_tsc_sync_source failed.
>Brought up 2 CPUs
>...
>
>This machine is running 2.6.23.1-21.fc7. I know I should
>report to Fedora,
>but I was wondering if this is a bug or a feature ;)
>
TSCs on Core 2 Duo are supposed to be in sync unless CPU supports deep idle states like C2, C3. Can you send the full /proc/cpuinfo and full dmesg.
Thanks,
Venki
Frans Pop
> Total of 2 processors activated (6919.15 BogoMIPS).
> ENABLING IO-APIC IRQs
> checking TSC synchronization [CPU#0 -> CPU#1]:
> Measured 3978592228 cycles TSC warp between CPUs, turning off TSC clock.
> Marking TSC unstable due to: check_tsc_sync_source failed.
> Brought up 2 CPUs
Not sure if this is related, but thought I'd contribute it anyway...
I've got a Pentium D system (dual core, single processor) and I on some
boots I get "Marking TSC unstable due to check_tsc_sync_source failed" with
some cycles warp between CPUs, while most boots are OK. This kind of
inconsistency seems more due to a failure in the kernel to deal with
differences between boots than with something inherent to the hardware.
I conclude that because basically I never have any problems with the system
once it has booted and the TSC has passed.
From my kern.logs since Okt 26, I get the following data:
2.6.23+cfs: 2 passes
2.6.23.1: 1 pass; 1 failure (48 cycles warp)
2.6.24-rc1: 15 passes
2.6.24-rc2: 13 passes; 1 failure (8 cycles warp)
2.6.24-rc3: 5 passes; 3 failures (8, 8 and 16 cycles warp)
Note that this is not a new issue. For 2.6.21/2.6.23-RCx kernels I reported
similar data in http://lkml.org/lkml/2007/9/16/45.
Cheers,
FJP
Paul Rolland
On Fri, 30 Nov 2007 00:26:47 +0300
Michael Tokarev <m...@tls.msk.ru> wrote:
> H. Peter Anvin wrote:
> > Paul Rolland (ポール・ロラン) wrote:
> []
> >> Measured 3978592228 cycles TSC warp between CPUs, turning off TSC clock.
> >> Marking TSC unstable due to: check_tsc_sync_source failed.
> []
> >> but I was wondering if this is a bug or a feature ;)
>
> > The problem you're having is that the TSCs of your two cores are
> > completely different, over a second apart. This is a bug, unrelated to
> > constant_tsc.
>
> A bug in where - in the CPU or in kernel?
Good question !
> The thing is that all our dual-core machines shows something like
> that.
>
> (not that huge difference as Paul reported, but still "unstable".
> The same happens with 2.6.23)
I've been checking my logs, and the difference is quite constant and
huge :
[root@tux log]# grep 'cycles TSC warp' messages*
messages:Nov 26 08:27:56 tux kernel: Measured 4078687691 cycles TSC warp between C
PUs, turning off TSC clock.
messages:Nov 26 17:21:21 tux kernel: Measured 3978592228 cycles TSC warp between C
PUs, turning off TSC clock.
messages.1:Nov 18 22:52:23 tux kernel: Measured 4063102940 cycles TSC warp between
CPUs, turning off TSC clock.
messages.1:Nov 19 07:19:02 tux kernel: Measured 4057192061 cycles TSC warp between
CPUs, turning off TSC clock.
messages.1:Nov 23 20:50:12 tux kernel: Measured 4064589321 cycles TSC warp between
CPUs, turning off TSC clock.
messages.2:Nov 12 08:06:44 tux kernel: Measured 4072130361 cycles TSC warp between
CPUs, turning off TSC clock.
messages.2:Nov 13 19:42:47 tux kernel: Measured 4049899451 cycles TSC warp between
CPUs, turning off TSC clock.
messages.2:Nov 17 09:27:22 tux kernel: Measured 4066629060 cycles TSC warp between
CPUs, turning off TSC clock.
messages.3:Nov 5 08:25:08 tux kernel: Measured 4086386109 cycles TSC warp between
CPUs, turning off TSC clock.
messages.3:Nov 8 13:07:08 tux kernel: Measured 4041945934 cycles TSC warp between
CPUs, turning off TSC clock.
messages.3:Nov 9 23:31:24 tux kernel: Measured 4092303059 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Oct 29 07:28:23 tux kernel: Measured 4096946373 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Oct 31 17:07:21 tux kernel: Measured 4046765372 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Oct 31 17:15:09 tux kernel: Measured 4039328228 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Oct 31 23:19:00 tux kernel: Measured 4069714246 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Nov 1 20:33:02 tux kernel: Measured 4088199726 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Nov 2 11:53:17 tux kernel: Measured 4079927527 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Nov 3 09:37:16 tux kernel: Measured 4071112656 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Nov 3 10:51:29 tux kernel: Measured 3986266219 cycles TSC warp between
CPUs, turning off TSC clock.
messages.4:Nov 4 18:14:56 tux kernel: Measured 4074214144 cycles TSC warp between
CPUs, turning off TSC clock.
> Note that once TSC is disabled (it's using "jiffies" as far
> as I can see), ntpd constantly speeds up and slows down the
> clock, it jumps +/- 0.5sec every several minutes or hours -
> I guess that's when ntpd process gets moved from one core
> to another for whatever reason. And an interesting thing
> is that with 64bits kernel this TSC problem does not occur
> on this very machine.
Hmmmm.... That could make it a problem related to kernel rather than CPU.
> Something similar is reported on AMD X2 64 machines as well --
> can't check right now.
If I recall correctly, issues with AMD X2 where related to TSC being
independant for each core and not constant (speed depending of C state).
But the reason I raise the issue is that the Core2 reports constant TSC,
so there is (IMHO) no reason for that.
Paul
H. Peter Anvin
Well, "constant" doesn't mean "synchronized", but it might very well be
that the Core2 could really benefit from synchronizing the TSCs manually
like we used to.
On the other hand, I notice that most of the TSC warp values are
relatively close to 2^32, so this could be a specific bug.
-hpa