Waiting for bufdaemon
64 views
Skip to first unread message
Jakob Alvermark
Jan 15, 2021, 5:27:19 AM1/15/21
to freebsd-current
Hi,
When rebooting my thinkpad the 'bufdaemon' times out:
Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop
... timed out
This started happening recently (within the last week I think).
Any ideas?
Thanks,
Jakob
_______________________________________________
freebsd...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"
When rebooting my thinkpad the 'bufdaemon' times out:
Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop
... timed out
Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop
... timed out
This started happening recently (within the last week I think).
Any ideas?
Thanks,
Jakob
_______________________________________________
freebsd...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"
Konstantin Belousov
Jan 15, 2021, 5:54:06 AM1/15/21
to Jakob Alvermark, freebsd-current
On Fri, Jan 15, 2021 at 11:26:47AM +0100, Jakob Alvermark wrote:
> Hi,
>
>
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop ...
> timed out
>
>
> This started happening recently (within the last week I think).
>
>
> Any ideas?
It is bare metal install, right?
> Hi,
>
>
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop ...
> timed out
>
>
> This started happening recently (within the last week I think).
>
>
> Any ideas?
Do you see any mis-handling of time or timeouts while the system operates?
Can you bisect it?
Yasuhiro Kimura
Jan 15, 2021, 6:12:01 AM1/15/21
to freebsd...@freebsd.org
From: Jakob Alvermark <ja...@alvermark.net>
Subject: Waiting for bufdaemon
Date: Fri, 15 Jan 2021 11:26:47 +0100
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop
> ... timed out
>
>
> This started happening recently (within the last week I think).
>
>
> Any ideas?
I have been experiencing same problem with my 13-CURRENT amd64
VirtualBox VM for about a month. The conditions that the problem
happens are unclear and all what I can say is
* It happens only after I login in the VM and do something for a
while. If I boot the VM and shut it down immediately, it never
happens.
* When the problem happens, one or more unkillable processes seem to
be left.
---
Yasuhiro Kimura
Subject: Waiting for bufdaemon
Date: Fri, 15 Jan 2021 11:26:47 +0100
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop
> ... timed out
>
>
> This started happening recently (within the last week I think).
>
>
> Any ideas?
VirtualBox VM for about a month. The conditions that the problem
happens are unclear and all what I can say is
* It happens only after I login in the VM and do something for a
while. If I boot the VM and shut it down immediately, it never
happens.
* When the problem happens, one or more unkillable processes seem to
be left.
---
Yasuhiro Kimura
Jakob Alvermark
Jan 15, 2021, 6:14:47 AM1/15/21
to Konstantin Belousov, freebsd-current
On 1/15/21 11:53 AM, Konstantin Belousov wrote:
> On Fri, Jan 15, 2021 at 11:26:47AM +0100, Jakob Alvermark wrote:
>> Hi,
>>
>>
>> When rebooting my thinkpad the 'bufdaemon' times out:
>>
>> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ... timed out
>>
>> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop ...
>> timed out
>>
>> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop ...
>> timed out
>>
>> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop ...
>> timed out
>>
>> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop ...
>> timed out
>>
>> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop ...
>> timed out
>>
>> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop ...
>> timed out
>>
>>
>> This started happening recently (within the last week I think).
>>
>>
>> Any ideas?
> It is bare metal install, right?
> Do you see any mis-handling of time or timeouts while the system operates?
My webcam (using webcamd) seems to be a little flakey, but that's about
the only thing I have noticed.
>
> Can you bisect it?
I could try, but it will take some time, it seems this only happens when
system has been running for a while. Rebooting right after a boot does
not show this behavior.
Jakob
Juraj Lutter
Jan 15, 2021, 6:17:35 AM1/15/21
to freebsd...@freebsd.org
My observation was that after putting some load on the machine, mysterious mrsas(4)
timeouts started to happen, processes were unkillable, reboots could not complete in time...
For sake of bisect, main-c255656-gb500c184b656 is known to work OK for me as of now.
Santiago Martinez
Jan 15, 2021, 6:53:12 AM1/15/21
to Juraj Lutter, freebsd...@freebsd.org, s...@codenetworks.net
Hi, there, im having the same issue here ( AMD Ryzen7). i do not use mrsas but in my case even if i try to reboot immediately after a fresh reboot i get the timeouts. Regards Santiago
-------- Original message --------From: Juraj Lutter <ot...@FreeBSD.org> Date: 15/01/2021 12:17 (GMT+01:00) To: freebsd...@freebsd.org Subject: Re: Waiting for bufdaemon > On 15 Jan 2021, at 12:10, Yasuhiro Kimura <ya...@utahime.org> wrote:> > From: Jakob Alvermark <ja...@alvermark.net>> Subject: Waiting for bufdaemon> Date: Fri, 15 Jan 2021 11:26:47 +0100> >> When rebooting my thinkpad the 'bufdaemon' times out:>> >> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop>> ... timed out>> >> >> This started happening recently (within the last week I think).>> >> >> Any ideas?> > I have been experiencing same problem with my 13-CURRENT amd64> VirtualBox VM for about a month. The conditions that the problem> happens are unclear and all what I can say is> > * It happens only after I login in the VM and do something for a> while. If I boot the VM and shut it down immediately, it never> happens.> * When the problem happens, one or more unkillable processes seem to> be left.I have been fighting with situations like this for month or two, opened more PRs about it.My observation was that after putting some load on the machine, mysterious mrsas(4)timeouts started to happen, processes were unkillable, reboots could not complete in time...For sake of bisect, main-c255656-gb500c184b656 is known to work OK for me as of now._____________________________...@freebsd.org mailing listhttps://lists.freebsd.org/mailman/listinfo/freebsd-currentTo unsubscribe, send any mail to "freebsd-curre...@freebsd.org"
-------- Original message --------From: Juraj Lutter <ot...@FreeBSD.org> Date: 15/01/2021 12:17 (GMT+01:00) To: freebsd...@freebsd.org Subject: Re: Waiting for bufdaemon > On 15 Jan 2021, at 12:10, Yasuhiro Kimura <ya...@utahime.org> wrote:> > From: Jakob Alvermark <ja...@alvermark.net>> Subject: Waiting for bufdaemon> Date: Fri, 15 Jan 2021 11:26:47 +0100> >> When rebooting my thinkpad the 'bufdaemon' times out:>> >> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop>> ... timed out>> >> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop>> ... timed out>> >> >> This started happening recently (within the last week I think).>> >> >> Any ideas?> > I have been experiencing same problem with my 13-CURRENT amd64> VirtualBox VM for about a month. The conditions that the problem> happens are unclear and all what I can say is> > * It happens only after I login in the VM and do something for a> while. If I boot the VM and shut it down immediately, it never> happens.> * When the problem happens, one or more unkillable processes seem to> be left.I have been fighting with situations like this for month or two, opened more PRs about it.My observation was that after putting some load on the machine, mysterious mrsas(4)timeouts started to happen, processes were unkillable, reboots could not complete in time...For sake of bisect, main-c255656-gb500c184b656 is known to work OK for me as of now._____________________________...@freebsd.org mailing listhttps://lists.freebsd.org/mailman/listinfo/freebsd-currentTo unsubscribe, send any mail to "freebsd-curre...@freebsd.org"
Mikaël Urankar
Jan 15, 2021, 9:56:19 AM1/15/21
to freebsd-current
On 15/01/2021 11:26, Jakob Alvermark wrote:
> Hi,
>
>
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop
> ... timed out
>
>
> This started happening recently (within the last week I think).
Hi,
> Hi,
>
>
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ...
> timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-0' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-1' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-2' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-3' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-4' to stop
> ... timed out
>
> Waiting (max 60 seconds) for system thread 'bufspacedaemon-5' to stop
> ... timed out
>
>
> This started happening recently (within the last week I think).
I'm also affected. I have an AMD Ryzen 9 3900X cpu, running bare metal.
5844bd058aed6f3d0c8cbbddd6aa95993ece0189 (jobc: rework detection of
orphaned groups) "seems" ok
cd240c9cf100bec3def38ceb4a320611b1d02693 (x86 vdso gettc: Add RDTSCP
support), affected by the timeout.
I haven't tried the intermediate commit yet.
My intel machine doesn't seem to be affected
Konstantin Belousov
Jan 15, 2021, 10:02:35 AM1/15/21
to Mikaël Urankar, freebsd-current
If you revert only 9e680e4005b7, is it fixed ?
Jakob Alvermark
Jan 15, 2021, 10:03:56 AM1/15/21
to freebsd...@freebsd.org
I observed the same here. It seems to only happen on my AMD laptop.
I have an Intel laptop which seems fine.
Jakob
Mikaël Urankar
Jan 15, 2021, 10:30:39 AM1/15/21
to freebsd-current
tests, I can do more if you want)
Konstantin Belousov
Jan 15, 2021, 10:45:33 AM1/15/21
to Mikaël Urankar, freebsd-current
kern.timecounter
kern.eventtimer
and first 100 lines of dmesg from the verbose boot (that contains the CPU
ident lines).
Mikaël Urankar
Jan 15, 2021, 11:19:16 AM1/15/21
to Konstantin Belousov, freebsd-current
I put the /var/run/dmesg.boot file on freefall /home/mikael/dmesg.boot
(it seems to be truncated though, I don't know how to retrieve the full log)
sysctl kern.timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.choice: ACPI-fast(900) HPET(950) i8254(0) TSC-low(1000)
dummy(-1000000)
kern.timecounter.hardware: TSC-low
kern.timecounter.alloweddeviation: 5
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.counter: 1470549582
kern.timecounter.tc.ACPI-fast.mask: 4294967295
kern.timecounter.tc.HPET.quality: 950
kern.timecounter.tc.HPET.frequency: 14318180
kern.timecounter.tc.HPET.counter: 378058131
kern.timecounter.tc.HPET.mask: 4294967295
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.counter: 20425
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.TSC-low.quality: 1000
kern.timecounter.tc.TSC-low.frequency: 1900039387
kern.timecounter.tc.TSC-low.counter: 2386729797
kern.timecounter.tc.TSC-low.mask: 4294967295
sysctl kern.eventtimer
kern.eventtimer.choice: LAPIC(600) HPET(350) HPET1(350) HPET2(350)
i8254(100) RTC(0)
kern.eventtimer.et.HPET2.quality: 350
kern.eventtimer.et.HPET2.frequency: 14318180
kern.eventtimer.et.HPET2.flags: 3
kern.eventtimer.et.HPET1.quality: 350
kern.eventtimer.et.HPET1.frequency: 14318180
kern.eventtimer.et.HPET1.flags: 3
kern.eventtimer.et.HPET.quality: 350
kern.eventtimer.et.HPET.frequency: 14318180
kern.eventtimer.et.HPET.flags: 3
kern.eventtimer.et.RTC.quality: 0
kern.eventtimer.et.RTC.frequency: 32768
kern.eventtimer.et.RTC.flags: 17
kern.eventtimer.et.i8254.quality: 100
kern.eventtimer.et.i8254.frequency: 1193182
kern.eventtimer.et.i8254.flags: 1
kern.eventtimer.et.LAPIC.quality: 600
kern.eventtimer.et.LAPIC.frequency: 50001034
kern.eventtimer.et.LAPIC.flags: 7
kern.eventtimer.periodic: 0
kern.eventtimer.timer: LAPIC
kern.eventtimer.idletick: 0
kern.eventtimer.singlemul: 2
Konstantin Belousov
Jan 15, 2021, 11:35:55 AM1/15/21
to Mikaël Urankar, freebsd-current
It is clipped at the start, and that was the information which I need.
Add something like
kern.msgbufsize=1048576
to /boot/loader.conf and try again. I need to see the lines starting with
CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (1992.08-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x806ea Family=0x6 Model=0x8e Stepping=10
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,C
MOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
...
(well, this is my w/s, your CPU would be different of course).
Mikaël Urankar
Jan 15, 2021, 11:47:14 AM1/15/21
to Konstantin Belousov, freebsd-current
Freddie Cash
Jan 15, 2021, 12:27:06 PM1/15/21
to Mikaël Urankar, Konstantin Belousov, freebsd-current
On Fri., Jan. 15, 2021, 8:47 a.m. Mikaël Urankar, <mik...@freebsd.org>
wrote:
wrote:
> Thanks, I've updated the file on freefall.
/var/run/dmesg.boot includes all output from dmesg for the current boot. No
need to manually redirect dmesg output to a file or play with buffer sizes.
:) Just upload that file.
Cheers,
Freddie
Typos due to smartphone keyboard.
Juraj Lutter
Jan 15, 2021, 12:41:30 PM1/15/21
to Freddie Cash, Mikaël Urankar, Konstantin Belousov, freebsd-current
> On 15 Jan 2021, at 18:26, Freddie Cash <fjw...@gmail.com> wrote:
>
>
> /var/run/dmesg.boot includes all output from dmesg for the current boot. No
> need to manually redirect dmesg output to a file or play with buffer sizes.
> :) Just upload that file.
buffer can already be rewritten.
Freddie Cash
Jan 15, 2021, 12:56:52 PM1/15/21
to Juraj Lutter, FreeBSD-Current
On Fri., Jan. 15, 2021, 9:41 a.m. Juraj Lutter, <ot...@freebsd.org> wrote:
> > On 15 Jan 2021, at 18:26, Freddie Cash <fjw...@gmail.com> wrote:
> >
> > /var/run/dmesg.boot includes all output from dmesg for the current boot.
> No
> > need to manually redirect dmesg output to a file or play with buffer
> sizes.
> > :) Just upload that file.
>
> On boot, dmesg is saved to dmesg.boot quite lately and start of the
> circular
> buffer can already be rewritten.
>
Really? Interesting. Haven't run into that (not saying it can't happen). I
> > On 15 Jan 2021, at 18:26, Freddie Cash <fjw...@gmail.com> wrote:
> >
> > /var/run/dmesg.boot includes all output from dmesg for the current boot.
> No
> > need to manually redirect dmesg output to a file or play with buffer
> sizes.
> > :) Just upload that file.
>
> On boot, dmesg is saved to dmesg.boot quite lately and start of the
> circular
> buffer can already be rewritten.
>
always thought it was essentially streamed to that file once / or /var was
mounted.
Now I'll have to read some RC scripts to see. :)
Thanks for the info.
Cheers,
Freddie
Typos due to smartphone keyboard.
>
Mark Millard
Jan 15, 2021, 1:13:47 PM1/15/21
to Konstantin Belousov, Current FreeBSD
In the interest of documenting some variability . . .
While not readily reproducible, on a ThreadRipper 1950X
that I have access to, a recent reboot had one "system
thread" time out: bufspacedaemon-3 . The rest of the
"system thread" list behaved normally.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
While not readily reproducible, on a ThreadRipper 1950X
that I have access to, a recent reboot had one "system
thread" time out: bufspacedaemon-3 . The rest of the
"system thread" list behaved normally.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Rainer Hurling
Jan 15, 2021, 1:49:27 PM1/15/21
to Konstantin Belousov, freebsd-current
Am 15.01.21 um 16:45 schrieb Konstantin Belousov:
I also have this timeout issue only on AMD, here Ryzen 3950X:
[...]
Waiting for PIDS: 2905
90 seconds watchdog timeout expired. Shutdown terminated.
Fri Jan 15 19:21:01 xxxxxx init[1]: /etc/rc.shutdown terminated
abnormally, going to single user mode
Fri Jan 15 19:21:01 xxxxxx syslogd: exiting on signal 15
wg0: link state changed to DOWN
Waiting (max 69 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 22 23 23 1 1 0 0 0 done
Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-6' to stop...
You will find the wanted output from sysctl and boot -v in the attachment.
HTH,
Rainer Hurling
[...]
Waiting for PIDS: 2905
90 seconds watchdog timeout expired. Shutdown terminated.
Fri Jan 15 19:21:01 xxxxxx init[1]: /etc/rc.shutdown terminated
abnormally, going to single user mode
Fri Jan 15 19:21:01 xxxxxx syslogd: exiting on signal 15
wg0: link state changed to DOWN
Waiting (max 69 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 22 23 23 1 1 0 0 0 done
Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-0' to stop...
done
Waiting (max 60 seconds) for system thread `bufspacedaemon-1' to stop...
done
Waiting (max 60 seconds) for system thread `bufspacedaemon-2' to stop...
done
Waiting (max 60 seconds) for system thread `bufspacedaemon-3' to stop...
timeout
Waiting (max 60 seconds) for system thread `bufspacedaemon-4' to stop...
timeout
Waiting (max 60 seconds) for system thread `bufspacedaemon-5' to stop...
done
Waiting (max 60 seconds) for system thread `bufspacedaemon-6' to stop...
You will find the wanted output from sysctl and boot -v in the attachment.
HTH,
Rainer Hurling
Yasuhiro Kimura
Jan 15, 2021, 2:04:08 PM1/15/21
to freebsd...@freebsd.org
From: Yasuhiro Kimura <ya...@utahime.org>
Subject: Re: Waiting for bufdaemon
Date: Fri, 15 Jan 2021 20:10:30 +0900 (JST)
> I have been experiencing same problem with my 13-CURRENT amd64
> VirtualBox VM for about a month. The conditions that the problem
> happens are unclear and all what I can say is
>
> * It happens only after I login in the VM and do something for a
> while. If I boot the VM and shut it down immediately, it never
> happens.
> * When the problem happens, one or more unkillable processes seem to
> be left.
CPU of my host is not AMD but Intel. According to the
/var/run/dmesg.boot of VM, information of CPU is as following.
CPU: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz (3000.09-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x906ed Family=0x6 Model=0x9e Stepping=13
Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x5eda2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,RDRAND>
AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
AMD Features2=0x121<LAHF,ABM,Prefetch>
Structured Extended Features=0x842421<FSGSBASE,AVX2,INVPCID,NFPUSG,RDSEED,CLFLUSHOPT>
Structured Extended Features3=0x30000400<MD_CLEAR,L1DFL,ARCH_CAP>
IA32_ARCH_CAPS=0x29<RDCL_NO,SKIP_L1DFL_VME,MDS_NO>
TSC: P-state invariant
Just FYI.
Subject: Re: Waiting for bufdaemon
Date: Fri, 15 Jan 2021 20:10:30 +0900 (JST)
> I have been experiencing same problem with my 13-CURRENT amd64
> VirtualBox VM for about a month. The conditions that the problem
> happens are unclear and all what I can say is
>
> * It happens only after I login in the VM and do something for a
> while. If I boot the VM and shut it down immediately, it never
> happens.
> * When the problem happens, one or more unkillable processes seem to
> be left.
/var/run/dmesg.boot of VM, information of CPU is as following.
CPU: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz (3000.09-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x906ed Family=0x6 Model=0x9e Stepping=13
Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x5eda2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,RDRAND>
AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
AMD Features2=0x121<LAHF,ABM,Prefetch>
Structured Extended Features=0x842421<FSGSBASE,AVX2,INVPCID,NFPUSG,RDSEED,CLFLUSHOPT>
Structured Extended Features3=0x30000400<MD_CLEAR,L1DFL,ARCH_CAP>
IA32_ARCH_CAPS=0x29<RDCL_NO,SKIP_L1DFL_VME,MDS_NO>
TSC: P-state invariant
Just FYI.
Rebecca Cran
Jan 16, 2021, 12:45:37 AM1/16/21
to Jakob Alvermark, freebsd-current
On 1/15/21 3:26 AM, Jakob Alvermark wrote:
>
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ...
> timed out
I'm glad other people are seeing this: I was about to report it to the list!
>
> When rebooting my thinkpad the 'bufdaemon' times out:
>
> Waiting (max 60 seconds) for system thread 'bufdaemon' to stop ...
> timed out
I'm running an AMD Threadripper 2990WX. I can trigger it by running a
buildworld/buildkernel and then rebooting.
--
Rebecca Cran
Rainer Hurling
Jan 16, 2021, 1:41:35 PM1/16/21
to Konstantin Belousov, freebsd-current
Am 15.01.21 um 19:48 schrieb Rainer Hurling:
During another shutdown after heavy usage of the box, the following
messages were also seen:
[...]
Syncing disks, vnodes remaining... 22 EFI rt_settime call faulted, error 14
efirtc0: CLOCK_SETTIME error 14
Konstantin Belousov
Jan 16, 2021, 11:33:32 PM1/16/21
to rhu...@freebsd.org, freebsd-current
On Sat, Jan 16, 2021 at 07:41:01PM +0100, Rainer Hurling wrote:
> During another shutdown after heavy usage of the box, the following
> messages were also seen:
>
>
> [...]
> Syncing disks, vnodes remaining... 22 EFI rt_settime call faulted, error 14
> efirtc0: CLOCK_SETTIME error 14
This means that BIOS code faulted during RTC settime call. I doubt that
> During another shutdown after heavy usage of the box, the following
> messages were also seen:
>
>
> [...]
> Syncing disks, vnodes remaining... 22 EFI rt_settime call faulted, error 14
> efirtc0: CLOCK_SETTIME error 14
it is related.
On the other hand, it is good that the onfault EFI RT code got tested finally.
Rainer Hurling
Jan 17, 2021, 4:37:43 AM1/17/21
to Konstantin Belousov, freebsd-current
Am 17.01.21 um 05:33 schrieb Konstantin Belousov:
Any chance of getting a fix for the AMD CPUs in the foreseeable future?
Or should I revert commit 9e680e4005b7 on affected boxes until further
notice (as a workaround)?
> On Sat, Jan 16, 2021 at 07:41:01PM +0100, Rainer Hurling wrote:
>> During another shutdown after heavy usage of the box, the following
>> messages were also seen:
>>
>>
>> [...]
>> Syncing disks, vnodes remaining... 22 EFI rt_settime call faulted, error 14
>> efirtc0: CLOCK_SETTIME error 14
>
> This means that BIOS code faulted during RTC settime call. I doubt that
> it is related.
>
> On the other hand, it is good that the onfault EFI RT code got tested finally.
>
Thanks for clarification :)
>> During another shutdown after heavy usage of the box, the following
>> messages were also seen:
>>
>>
>> [...]
>> Syncing disks, vnodes remaining... 22 EFI rt_settime call faulted, error 14
>> efirtc0: CLOCK_SETTIME error 14
>
> This means that BIOS code faulted during RTC settime call. I doubt that
> it is related.
>
> On the other hand, it is good that the onfault EFI RT code got tested finally.
>
Any chance of getting a fix for the AMD CPUs in the foreseeable future?
Or should I revert commit 9e680e4005b7 on affected boxes until further
notice (as a workaround)?
Konstantin Belousov
Jan 17, 2021, 4:50:09 AM1/17/21
to rhu...@freebsd.org, freebsd-current
On Sun, Jan 17, 2021 at 10:37:18AM +0100, Rainer Hurling wrote:
> Am 17.01.21 um 05:33 schrieb Konstantin Belousov:
> > On Sat, Jan 16, 2021 at 07:41:01PM +0100, Rainer Hurling wrote:
> >> During another shutdown after heavy usage of the box, the following
> >> messages were also seen:
> >>
> >>
> >> [...]
> >> Syncing disks, vnodes remaining... 22 EFI rt_settime call faulted, error 14
> >> efirtc0: CLOCK_SETTIME error 14
> >
> > This means that BIOS code faulted during RTC settime call. I doubt that
> > it is related.
> >
> > On the other hand, it is good that the onfault EFI RT code got tested finally.
> >
>
> Thanks for clarification :)
>
>
> Any chance of getting a fix for the AMD CPUs in the foreseeable future?
>
> Or should I revert commit 9e680e4005b7 on affected boxes until further
> notice (as a workaround)?
I am working on it, no ETA.
> Am 17.01.21 um 05:33 schrieb Konstantin Belousov:
> > On Sat, Jan 16, 2021 at 07:41:01PM +0100, Rainer Hurling wrote:
> >> During another shutdown after heavy usage of the box, the following
> >> messages were also seen:
> >>
> >>
> >> [...]
> >> Syncing disks, vnodes remaining... 22 EFI rt_settime call faulted, error 14
> >> efirtc0: CLOCK_SETTIME error 14
> >
> > This means that BIOS code faulted during RTC settime call. I doubt that
> > it is related.
> >
> > On the other hand, it is good that the onfault EFI RT code got tested finally.
> >
>
> Thanks for clarification :)
>
>
> Any chance of getting a fix for the AMD CPUs in the foreseeable future?
>
> Or should I revert commit 9e680e4005b7 on affected boxes until further
> notice (as a workaround)?
Interesting point would be to check on machines of other testers,
if the following hides the problem.
diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c
index 85924df98312..5700a8ca98e5 100644
--- a/sys/x86/x86/tsc.c
+++ b/sys/x86/x86/tsc.c
@@ -639,7 +639,7 @@ init_TSC_tc(void)
* on Intel, and MFENCE;RDTSC on AMD.
* - For really old CPUs, just use RDTSC.
*/
- if ((cpu_vendor_id == CPU_VENDOR_AMD ||
+ if (false && (cpu_vendor_id == CPU_VENDOR_AMD ||
cpu_vendor_id == CPU_VENDOR_HYGON) &&
CPUID_TO_FAMILY(cpu_id) >= 0x17) {
tsc_timecounter.tc_get_timecount = shift > 0 ?
Rainer Hurling
Jan 17, 2021, 11:42:24 AM1/17/21
to Konstantin Belousov, freebsd-current
Am 17.01.21 um 10:49 schrieb Konstantin Belousov:
I tried the above patch on a Ryzen 3950X.
After reboot (again with long waiting for bufdaemon and more) the box
had some heavy load with several jobs (llvm on Poudriere, building
qt5-webengine, libreoffice and some more in parallel). All went fine.
Afterwards I rebooted again. This time, the shutdown was fast without
any unusual delays :)
Many thanks!
After reboot (again with long waiting for bufdaemon and more) the box
had some heavy load with several jobs (llvm on Poudriere, building
qt5-webengine, libreoffice and some more in parallel). All went fine.
Afterwards I rebooted again. This time, the shutdown was fast without
any unusual delays :)
Many thanks!
Yasuhiro Kimura
Jan 17, 2021, 3:03:10 PM1/17/21
to freebsd...@freebsd.org
From: Konstantin Belousov <kost...@gmail.com>
> I am working on it, no ETA.
>
> Interesting point would be to check on machines of other testers,
> if the following hides the problem.
I tried this patch but unfortunately the problem still happens with my
environment.
---
Yasuhiro Kimura
Subject: Re: Waiting for bufdaemon
Date: Sun, 17 Jan 2021 11:49:40 +0200
> I am working on it, no ETA.
>
> Interesting point would be to check on machines of other testers,
> if the following hides the problem.
environment.
---
Yasuhiro Kimura
Jakob Alvermark
Jan 20, 2021, 5:02:51 AM1/20/21
to Konstantin Belousov, freebsd-current
No waiting on reboot, and the webcam works better.
Jakob
Konstantin Belousov
Jan 20, 2021, 5:19:14 AM1/20/21
to Jakob Alvermark, freebsd-current
On Wed, Jan 20, 2021 at 11:02:21AM +0100, Jakob Alvermark wrote:
> This patch hides the problem for me. The system seems to work better now.
>
> No waiting on reboot, and the webcam works better.
I am curious what do you mean by the above reference to webcam.
> This patch hides the problem for me. The system seems to work better now.
>
> No waiting on reboot, and the webcam works better.
Can you explain it with more details, even if only the impressions?
I probably going to commit the following patch in the next 24 hours.
commit 02505d07bca320a638c96918ac9076c6eece2fff
Author: Konstantin Belousov <k...@FreeBSD.org>
Date: Wed Jan 20 11:32:21 2021 +0200
AMD Zen CPUs: switch TSC timecounter to RDTSCP
Reported by: many
MFC after: 1 weel
Sponsored by: The FreeBSD Foundation
diff --git a/lib/libc/x86/sys/__vdso_gettc.c b/lib/libc/x86/sys/__vdso_gettc.c
index 7f224f8758cb..7a64f2a0b556 100644
--- a/lib/libc/x86/sys/__vdso_gettc.c
+++ b/lib/libc/x86/sys/__vdso_gettc.c
@@ -125,7 +125,7 @@ struct tsc_selector_tag {
};
static const struct tsc_selector_tag tsc_selector[] = {
- [0] = { /* Intel or AMD Zen+, LFENCE */
+ [0] = { /* Intel, LFENCE */
.ts_rdtsc32 = rdtsc32_mb_lfence,
.ts_rdtsc_low = rdtsc_low_mb_lfence,
},
@@ -164,9 +164,6 @@ tsc_selector_idx(u_int cpu_feature)
do_cpuid(1, p);
cpu_id = p[0];
- if (amd_cpu && CPUID_TO_FAMILY(cpu_id) >= 0x17)
- return (0);
-
if (cpu_feature != 0) {
do_cpuid(0x80000000, p);
cpu_exthigh = p[0];
diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c
index 85924df98312..de0a1505c2f6 100644
--- a/sys/x86/x86/tsc.c
+++ b/sys/x86/x86/tsc.c
@@ -633,19 +633,12 @@ init_TSC_tc(void)
/*
* Timecounter implementation selection, top to bottom:
- * - For AMD Zens and newer, use LFENCE;RDTSC.
* - If RDTSCP is available, use RDTSCP.
* - If fence instructions are provided (SSE2), use LFENCE;RDTSC
* on Intel, and MFENCE;RDTSC on AMD.
* - For really old CPUs, just use RDTSC.
*/
- if ((cpu_vendor_id == CPU_VENDOR_AMD ||
- cpu_vendor_id == CPU_VENDOR_HYGON) &&
* - For really old CPUs, just use RDTSC.
*/
- if ((cpu_vendor_id == CPU_VENDOR_AMD ||
- CPUID_TO_FAMILY(cpu_id) >= 0x17) {
- tsc_timecounter.tc_get_timecount = shift > 0 ?
- tsc_get_timecount_low_lfence :
- tsc_get_timecount_lfence;
- } else if ((amd_feature & AMDID_RDTSCP) != 0) {
+ if ((amd_feature & AMDID_RDTSCP) != 0) {
tsc_timecounter.tc_get_timecount = shift > 0 ?
tscp_get_timecount_low : tscp_get_timecount;
} else if ((cpu_feature & CPUID_SSE2) != 0 && mp_ncpus > 1) {
Jakob Alvermark
Jan 20, 2021, 5:37:27 AM1/20/21
to Konstantin Belousov, freebsd-current
now that everyone is working from home. (MS Teams, Google Meet, Slack,
Zoom, whatever...).
It is supported by multimedia/webcamd and has been working mostly fine.
(just some rare, occasional hickups)
When I started to notice the bufdaemon problems, I also noticed the
webcam not behaving as before.
It sometimes took me two or three tries to start the camera when joining
a meeting and once it started my video would freeze sometimes, being
frozen for a minute or so. Sometimes when I noticed that it frooze I
would try to restart it, and again it might take a couple of tries to
get it working again.
With the patch it seems to work better again.
Jakob
Rainer Hurling
Jan 20, 2021, 7:18:16 AM1/20/21
to Konstantin Belousov, freebsd-current
Am 20.01.21 um 11:18 schrieb Konstantin Belousov:
bufdaemon, I also have problems with several apps:
After a high system load, several programs only react very slowly (e.g.
Firefox). Several dockable apps from WindowMaker, but also e.g. conky do
not update their windows anymore, they freeze. After some time, Firefox
updates its screen content only after switching back from another window ...
When such frozen programs are killed and restarted, they run normally
again for an indefinite time before they freeze again.
These symptoms completely disappeared, after I patched the Ryzen box as
suggested on 01/17:
diff --git a/sys/x86/tsc.c b/sys/x86/tsc.c
index 85924df98312..5700a8ca98e5 100644
--- a/sys/x86/x86/tsc.c
+++ b/sys/x86/x86/tsc.c
@@ -639,7 +639,7 @@ init_TSC_tc(void)
Rainer
> On Wed, Jan 20, 2021 at 11:02:21AM +0100, Jakob Alvermark wrote:
>> This patch hides the problem for me. The system seems to work better now.
>>
>> No waiting on reboot, and the webcam works better.
> I am curious what do you mean by the above reference to webcam.
> Can you explain it with more details, even if only the impressions?
I should mention, that beside the already discussed timing problem with
>> This patch hides the problem for me. The system seems to work better now.
>>
>> No waiting on reboot, and the webcam works better.
> I am curious what do you mean by the above reference to webcam.
> Can you explain it with more details, even if only the impressions?
bufdaemon, I also have problems with several apps:
After a high system load, several programs only react very slowly (e.g.
Firefox). Several dockable apps from WindowMaker, but also e.g. conky do
not update their windows anymore, they freeze. After some time, Firefox
updates its screen content only after switching back from another window ...
When such frozen programs are killed and restarted, they run normally
again for an indefinite time before they freeze again.
These symptoms completely disappeared, after I patched the Ryzen box as
suggested on 01/17:
diff --git a/sys/x86/tsc.c b/sys/x86/tsc.c
index 85924df98312..5700a8ca98e5 100644
--- a/sys/x86/x86/tsc.c
+++ b/sys/x86/x86/tsc.c
@@ -639,7 +639,7 @@ init_TSC_tc(void)
* on Intel, and MFENCE;RDTSC on AMD.
* - For really old CPUs, just use RDTSC.
*/
- if ((cpu_vendor_id == CPU_VENDOR_AMD ||
* - For really old CPUs, just use RDTSC.
*/
- if ((cpu_vendor_id == CPU_VENDOR_AMD ||
+ if (false && (cpu_vendor_id == CPU_VENDOR_AMD ||
cpu_vendor_id == CPU_VENDOR_HYGON) &&
CPUID_TO_FAMILY(cpu_id) >= 0x17) {
cpu_vendor_id == CPU_VENDOR_HYGON) &&
CPUID_TO_FAMILY(cpu_id) >= 0x17) {
tsc_timecounter.tc_get_timecount = shift > 0 ?
HTH,
Rainer
Konstantin Belousov
Jan 20, 2021, 7:34:37 AM1/20/21
to rhu...@freebsd.org, freebsd-current
On Wed, Jan 20, 2021 at 01:17:51PM +0100, Rainer Hurling wrote:
> Am 20.01.21 um 11:18 schrieb Konstantin Belousov:
> > On Wed, Jan 20, 2021 at 11:02:21AM +0100, Jakob Alvermark wrote:
> >> This patch hides the problem for me. The system seems to work better now.
> >>
> >> No waiting on reboot, and the webcam works better.
> > I am curious what do you mean by the above reference to webcam.
> > Can you explain it with more details, even if only the impressions?
>
> I should mention, that beside the already discussed timing problem with
> bufdaemon, I also have problems with several apps:
>
>
> After a high system load, several programs only react very slowly (e.g.
> Firefox). Several dockable apps from WindowMaker, but also e.g. conky do
> not update their windows anymore, they freeze. After some time, Firefox
> updates its screen content only after switching back from another window ...
>
> When such frozen programs are killed and restarted, they run normally
> again for an indefinite time before they freeze again.
>
> These symptoms completely disappeared, after I patched the Ryzen box as
> suggested on 01/17:
Do you load latest microcode update from devcpu-data?
> Am 20.01.21 um 11:18 schrieb Konstantin Belousov:
> > On Wed, Jan 20, 2021 at 11:02:21AM +0100, Jakob Alvermark wrote:
> >> This patch hides the problem for me. The system seems to work better now.
> >>
> >> No waiting on reboot, and the webcam works better.
> > I am curious what do you mean by the above reference to webcam.
> > Can you explain it with more details, even if only the impressions?
>
> I should mention, that beside the already discussed timing problem with
> bufdaemon, I also have problems with several apps:
>
>
> After a high system load, several programs only react very slowly (e.g.
> Firefox). Several dockable apps from WindowMaker, but also e.g. conky do
> not update their windows anymore, they freeze. After some time, Firefox
> updates its screen content only after switching back from another window ...
>
> When such frozen programs are killed and restarted, they run normally
> again for an indefinite time before they freeze again.
>
> These symptoms completely disappeared, after I patched the Ryzen box as
> suggested on 01/17:
It might be not enough, which means that additionally latest BIOS needs
to be flushed.
Rainer Hurling
Jan 20, 2021, 7:57:18 AM1/20/21
to Konstantin Belousov, freebsd-current
Am 20.01.21 um 13:34 schrieb Konstantin Belousov:
/boot/loader.conf
cpu_microcode_load="YES"
cpu_microcode_name="/boot/firmware/intel-ucode.bin"
But isn't this just for Intel (i387 and amd64), not AMD cpus?
> It might be not enough, which means that additionally latest BIOS needs
> to be flushed.
>
I am running latest Firmware F31 on a "Gigabyte\ Aorus\ X570\ Elite"
mainboard.
> On Wed, Jan 20, 2021 at 01:17:51PM +0100, Rainer Hurling wrote:
>> Am 20.01.21 um 11:18 schrieb Konstantin Belousov:
>>> On Wed, Jan 20, 2021 at 11:02:21AM +0100, Jakob Alvermark wrote:
>>>> This patch hides the problem for me. The system seems to work better now.
>>>>
>>>> No waiting on reboot, and the webcam works better.
>>> I am curious what do you mean by the above reference to webcam.
>>> Can you explain it with more details, even if only the impressions?
>>
>> I should mention, that beside the already discussed timing problem with
>> bufdaemon, I also have problems with several apps:
>>
>>
>> After a high system load, several programs only react very slowly (e.g.
>> Firefox). Several dockable apps from WindowMaker, but also e.g. conky do
>> not update their windows anymore, they freeze. After some time, Firefox
>> updates its screen content only after switching back from another window ...
>>
>> When such frozen programs are killed and restarted, they run normally
>> again for an indefinite time before they freeze again.
>>
>> These symptoms completely disappeared, after I patched the Ryzen box as
>> suggested on 01/17:
>
> Do you load latest microcode update from devcpu-data?
Yes, sysutils/devcpu-data is installed and the following to lines are in
>> Am 20.01.21 um 11:18 schrieb Konstantin Belousov:
>>> On Wed, Jan 20, 2021 at 11:02:21AM +0100, Jakob Alvermark wrote:
>>>> This patch hides the problem for me. The system seems to work better now.
>>>>
>>>> No waiting on reboot, and the webcam works better.
>>> I am curious what do you mean by the above reference to webcam.
>>> Can you explain it with more details, even if only the impressions?
>>
>> I should mention, that beside the already discussed timing problem with
>> bufdaemon, I also have problems with several apps:
>>
>>
>> After a high system load, several programs only react very slowly (e.g.
>> Firefox). Several dockable apps from WindowMaker, but also e.g. conky do
>> not update their windows anymore, they freeze. After some time, Firefox
>> updates its screen content only after switching back from another window ...
>>
>> When such frozen programs are killed and restarted, they run normally
>> again for an indefinite time before they freeze again.
>>
>> These symptoms completely disappeared, after I patched the Ryzen box as
>> suggested on 01/17:
>
> Do you load latest microcode update from devcpu-data?
/boot/loader.conf
cpu_microcode_load="YES"
cpu_microcode_name="/boot/firmware/intel-ucode.bin"
But isn't this just for Intel (i387 and amd64), not AMD cpus?
> It might be not enough, which means that additionally latest BIOS needs
> to be flushed.
>
mainboard.
Konstantin Belousov
Jan 20, 2021, 8:12:55 AM1/20/21
to rhu...@freebsd.org, freebsd-current
update.
I think that early boot update should work on AMD, bit for this you need to
select and put right blob. It is enough to load late to answer my question.
Rainer Hurling
Jan 20, 2021, 8:36:27 AM1/20/21
to Konstantin Belousov, freebsd-current
Am 20.01.21 um 14:12 schrieb Konstantin Belousov:
Ah, ok. Thanks for clarification. I also put cpu_microcode_load="YES" in
/etc/rc.conf for a late update.
Should I try again without your patch of sys/x86/tsc.c, whether the
problem still occurs?
And for the early boot update, how do I know about the right blob?
/etc/rc.conf for a late update.
Should I try again without your patch of sys/x86/tsc.c, whether the
problem still occurs?
And for the early boot update, how do I know about the right blob?
Konstantin Belousov
Jan 20, 2021, 8:52:44 AM1/20/21
to rhu...@freebsd.org, freebsd-current
>
>
> And for the early boot update, how do I know about the right blob?
the blob against your CPU family/model id manually.
Santiago Martinez
Jan 20, 2021, 8:58:38 AM1/20/21
to freebsd...@freebsd.org
Hi Everyone, i have exactly the same behavior as Rainer described.
"After a high system load, several programs only react very slowly (e.g.
Firefox)......"
I will try with the patch and see if it also clear this on my machine
(AMD R7).
Santiago
"After a high system load, several programs only react very slowly (e.g.
I will try with the patch and see if it also clear this on my machine
(AMD R7).
Santiago
Santiago Martinez
Jan 20, 2021, 9:22:49 AM1/20/21
to freebsd...@freebsd.org
Hi, excellent the patch clear the problem for me when rebooting after a
kern compile, etc, etc.
Also seems that Firefox and pycharm or any other java related tools are
not freezing any more, but its maybe to early to confirm this 100%.
Will keep you posted.
Santiago
kern compile, etc, etc.
Also seems that Firefox and pycharm or any other java related tools are
not freezing any more, but its maybe to early to confirm this 100%.
Will keep you posted.
Santiago
Rainer Hurling
Jan 20, 2021, 9:37:08 AM1/20/21
to Konstantin Belousov, freebsd-current
Am 20.01.21 um 14:52 schrieb Konstantin Belousov:
Unfornately, without the patch from 01/17 the problem is _not_ solved.
Next I will try your patch from today, f lib/libc/x86/sys/__vdso_gettc.c
an lib/libc/x86/sys/__vdso_gettc.c ...
Next I will try your patch from today, f lib/libc/x86/sys/__vdso_gettc.c
an lib/libc/x86/sys/__vdso_gettc.c ...
Mark Johnston
Jan 20, 2021, 9:42:52 AM1/20/21
to Konstantin Belousov, rhu...@freebsd.org, freebsd-current
for lack of a test system at the time.
Rainer Hurling
Jan 20, 2021, 9:55:02 AM1/20/21
to Mark Johnston, freebsd-current
Am 20.01.21 um 15:42 schrieb Mark Johnston:
Thanks for the info. So for now, I can remove
microcode_update_enable="YES" in /boot/loader.conf ...
Is there anything, I can test for you (without having skills in the area
;) )?
microcode_update_enable="YES" in /boot/loader.conf ...
Is there anything, I can test for you (without having skills in the area
;) )?
Rainer Hurling
Jan 20, 2021, 1:05:31 PM1/20/21
to Konstantin Belousov, freebsd-current
Am 20.01.21 um 15:36 schrieb Rainer Hurling:
I can confirm that this patch also works for me on Ryzen 3950X. No more
bufdaemon waitings, no frozen apps, ...
bufdaemon waitings, no frozen apps, ...
Yasuhiro Kimura
Jan 27, 2021, 3:03:27 PM1/27/21
to freebsd...@freebsd.org
Date: Sat, 16 Jan 2021 04:03:23 +0900 (JST)
> From: Yasuhiro Kimura <ya...@utahime.org>
> Subject: Re: Waiting for bufdaemon
> Date: Fri, 15 Jan 2021 20:10:30 +0900 (JST)
>
>> I have been experiencing same problem with my 13-CURRENT amd64
>> VirtualBox VM for about a month. The conditions that the problem
>> happens are unclear and all what I can say is
>>
>> * It happens only after I login in the VM and do something for a
>> while. If I boot the VM and shut it down immediately, it never
>> happens.
>> * When the problem happens, one or more unkillable processes seem to
>> be left.
>
> CPU of my host is not AMD but Intel. According to the
> /var/run/dmesg.boot of VM, information of CPU is as following.
>
> CPU: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz (3000.09-MHz K8-class CPU)
> Origin="GenuineIntel" Id=0x906ed Family=0x6 Model=0x9e Stepping=13
> Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
> Features2=0x5eda2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,RDRAND>
> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
> AMD Features2=0x121<LAHF,ABM,Prefetch>
> Structured Extended Features=0x842421<FSGSBASE,AVX2,INVPCID,NFPUSG,RDSEED,CLFLUSHOPT>
> Structured Extended Features3=0x30000400<MD_CLEAR,L1DFL,ARCH_CAP>
> IA32_ARCH_CAPS=0x29<RDCL_NO,SKIP_L1DFL_VME,MDS_NO>
> TSC: P-state invariant
>
> Just FYI.
It took for a while to investigate, but according to the result of
bisect following commit is the source of problem in my case.
----------------------------------------------------------------------
commit 84eaf2ccc6a
Author: Konstantin Belousov <k...@FreeBSD.org>
Date: Mon Dec 21 19:02:31 2020 +0200
x86: stop punishing VMs with low priority for TSC timecounter
I suspect that virtualization techniques improved from the time when we
have to effectively disable TSC use in VM. For instance, it was reported
(complained) in https://github.com/JuliaLang/julia/issues/38877 that
FreeBSD is groundlessly slow on AWS with some loads.
Remove the check and start watching for complaints.
Reviewed by: emaste, grehan
Discussed with: cperciva
----------------------------------------------------------------------
I confirmed the problem still happens with 5c325977b11 but reverting
above commit fixes it.
> From: Yasuhiro Kimura <ya...@utahime.org>
> Subject: Re: Waiting for bufdaemon
> Date: Fri, 15 Jan 2021 20:10:30 +0900 (JST)
>
>> I have been experiencing same problem with my 13-CURRENT amd64
>> VirtualBox VM for about a month. The conditions that the problem
>> happens are unclear and all what I can say is
>>
>> * It happens only after I login in the VM and do something for a
>> while. If I boot the VM and shut it down immediately, it never
>> happens.
>> * When the problem happens, one or more unkillable processes seem to
>> be left.
>
> CPU of my host is not AMD but Intel. According to the
> /var/run/dmesg.boot of VM, information of CPU is as following.
>
> CPU: Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz (3000.09-MHz K8-class CPU)
> Origin="GenuineIntel" Id=0x906ed Family=0x6 Model=0x9e Stepping=13
> Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
> Features2=0x5eda2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,RDRAND>
> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
> AMD Features2=0x121<LAHF,ABM,Prefetch>
> Structured Extended Features=0x842421<FSGSBASE,AVX2,INVPCID,NFPUSG,RDSEED,CLFLUSHOPT>
> Structured Extended Features3=0x30000400<MD_CLEAR,L1DFL,ARCH_CAP>
> IA32_ARCH_CAPS=0x29<RDCL_NO,SKIP_L1DFL_VME,MDS_NO>
> TSC: P-state invariant
>
> Just FYI.
bisect following commit is the source of problem in my case.
----------------------------------------------------------------------
commit 84eaf2ccc6a
Author: Konstantin Belousov <k...@FreeBSD.org>
Date: Mon Dec 21 19:02:31 2020 +0200
x86: stop punishing VMs with low priority for TSC timecounter
I suspect that virtualization techniques improved from the time when we
have to effectively disable TSC use in VM. For instance, it was reported
(complained) in https://github.com/JuliaLang/julia/issues/38877 that
FreeBSD is groundlessly slow on AWS with some loads.
Remove the check and start watching for complaints.
Reviewed by: emaste, grehan
Discussed with: cperciva
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27629
----------------------------------------------------------------------
I confirmed the problem still happens with 5c325977b11 but reverting
above commit fixes it.
Yasuhiro Kimura
Jan 29, 2021, 5:41:34 PM1/29/21
to freebsd...@freebsd.org
Date: Thu, 28 Jan 2021 05:02:42 +0900 (JST)
> It took for a while to investigate, but according to the result of
> bisect following commit is the source of problem in my case.
>
> ----------------------------------------------------------------------
> commit 84eaf2ccc6a
> Author: Konstantin Belousov <k...@FreeBSD.org>
> Date: Mon Dec 21 19:02:31 2020 +0200
>
> x86: stop punishing VMs with low priority for TSC timecounter
>
> I suspect that virtualization techniques improved from the time when we
> have to effectively disable TSC use in VM. For instance, it was reported
> (complained) in https://github.com/JuliaLang/julia/issues/38877 that
> FreeBSD is groundlessly slow on AWS with some loads.
>
> Remove the check and start watching for complaints.
>
> Reviewed by: emaste, grehan
> Discussed with: cperciva
> Sponsored by: The FreeBSD Foundation
> Differential Revision: https://reviews.freebsd.org/D27629
> ----------------------------------------------------------------------
>
> I confirmed the problem still happens with 5c325977b11 but reverting
> above commit fixes it.
I submitted this problem to Bugzilla.
Timeout of bufdaemon happens at shutdown time with -CURRENT amd64 and VirtulaBox VM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253087
> It took for a while to investigate, but according to the result of
> bisect following commit is the source of problem in my case.
>
> ----------------------------------------------------------------------
> commit 84eaf2ccc6a
> Author: Konstantin Belousov <k...@FreeBSD.org>
> Date: Mon Dec 21 19:02:31 2020 +0200
>
> x86: stop punishing VMs with low priority for TSC timecounter
>
> I suspect that virtualization techniques improved from the time when we
> have to effectively disable TSC use in VM. For instance, it was reported
> (complained) in https://github.com/JuliaLang/julia/issues/38877 that
> FreeBSD is groundlessly slow on AWS with some loads.
>
> Remove the check and start watching for complaints.
>
> Reviewed by: emaste, grehan
> Discussed with: cperciva
> Sponsored by: The FreeBSD Foundation
> Differential Revision: https://reviews.freebsd.org/D27629
> ----------------------------------------------------------------------
>
> I confirmed the problem still happens with 5c325977b11 but reverting
> above commit fixes it.
Timeout of bufdaemon happens at shutdown time with -CURRENT amd64 and VirtulaBox VM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253087
Yasuhiro Kimura
Mar 5, 2021, 2:04:06 AM3/5/21
to freebsd...@freebsd.org
Dear src committers,
Would someone please revert above commit from main, stable/13 and
releng/13.0? As I wrote previous mail I submitted this problem to
Bugzilla as bug 253087 and added the committer to Cc. But there is no
response for 34 days.
I confirmed the problem still happens with 37cd6c20dbc of main and
13.0-RC1.
Best Regards.
releng/13.0? As I wrote previous mail I submitted this problem to
Bugzilla as bug 253087 and added the committer to Cc. But there is no
response for 34 days.
I confirmed the problem still happens with 37cd6c20dbc of main and
13.0-RC1.
Best Regards.
Konstantin Belousov
Mar 5, 2021, 3:44:30 PM3/5/21
to Yasuhiro Kimura, freebsd...@freebsd.org
the VMWare and KVM users, which are majority, use fast timecounter,
comparing to the more niche hypervisors like VirtualBox.
For you, a simple but manual workaround, setting the timecounter to
ACPI (?) or might be HPET, with a loader tunable, should do it.
Yasuhiro Kimura
Mar 5, 2021, 6:35:21 PM3/5/21
to freebsd...@freebsd.org
From: Konstantin Belousov <kost...@gmail.com>
> My belief is that this commit helps more users than it hurts. Namely,
> the VMWare and KVM users, which are majority, use fast timecounter,
> comparing to the more niche hypervisors like VirtualBox.
>
> For you, a simple but manual workaround, setting the timecounter to
> ACPI (?) or might be HPET, with a loader tunable, should do it.
Then please let me know the name of it.
I have experienced same situation several time. That is, I faced a
problem and asked for it on ML. Then I was told to try some tunable.
So I thought there may be tunable that can be used as workaround in
this case. But for those who isn't familiar with kernel internal, it
it quite hard to find it without knowing its name. If all tunable were
listed with brief explanation in one document, then I saw it and could
pick up possible candidates. But actually they are documented
separately among many man pages. So the first difficulty is to find
man page in which possible tunable may be explained. If the problem is
releted to some device, it is most hopeful to check its man page. But
in this case, even after reading the commit message, I had no idea
which man page to check.
---
Yasuhiro Kimura
Subject: Re: Waiting for bufdaemon
Date: Fri, 5 Mar 2021 22:43:58 +0200
> My belief is that this commit helps more users than it hurts. Namely,
> the VMWare and KVM users, which are majority, use fast timecounter,
> comparing to the more niche hypervisors like VirtualBox.
>
> For you, a simple but manual workaround, setting the timecounter to
> ACPI (?) or might be HPET, with a loader tunable, should do it.
I have experienced same situation several time. That is, I faced a
problem and asked for it on ML. Then I was told to try some tunable.
So I thought there may be tunable that can be used as workaround in
this case. But for those who isn't familiar with kernel internal, it
it quite hard to find it without knowing its name. If all tunable were
listed with brief explanation in one document, then I saw it and could
pick up possible candidates. But actually they are documented
separately among many man pages. So the first difficulty is to find
man page in which possible tunable may be explained. If the problem is
releted to some device, it is most hopeful to check its man page. But
in this case, even after reading the commit message, I had no idea
which man page to check.
---
Yasuhiro Kimura
Michael Gmelin
Mar 5, 2021, 6:57:25 PM3/5/21
to Yasuhiro Kimura, freebsd...@freebsd.org
On Sat, 06 Mar 2021 08:33:23 +0900 (JST)
Yasuhiro Kimura <ya...@utahime.org> wrote:
> From: Konstantin Belousov <kost...@gmail.com>
> Subject: Re: Waiting for bufdaemon
> Date: Fri, 5 Mar 2021 22:43:58 +0200
>
> > My belief is that this commit helps more users than it hurts.
> > Namely, the VMWare and KVM users, which are majority, use fast
> > timecounter, comparing to the more niche hypervisors like
> > VirtualBox.
> >
> > For you, a simple but manual workaround, setting the timecounter to
> > ACPI (?) or might be HPET, with a loader tunable, should do it.
>
> Then please let me know the name of it.
>
> I have experienced same situation several time. That is, I faced a
> problem and asked for it on ML. Then I was told to try some tunable.
> So I thought there may be tunable that can be used as workaround in
> this case. But for those who isn't familiar with kernel internal, it
> it quite hard to find it without knowing its name. If all tunable were
> listed with brief explanation in one document, then I saw it and could
> pick up possible candidates. But actually they are documented
> separately among many man pages. So the first difficulty is to find
> man page in which possible tunable may be explained. If the problem is
> releted to some device, it is most hopeful to check its man page. But
> in this case, even after reading the commit message, I had no idea
> which man page to check.
>
https://www.freebsd.org/cgi/man.cgi?query=timecounters
-m
--
Michael Gmelin
Mark Millard via freebsd-current
Mar 5, 2021, 8:35:48 PM3/5/21
to ya...@utahime.org, freebsd-current
Yasuhiro Kimura yasu at utahime.org wrote on
Fri Mar 5 23:34:59 UTC 2021 :
> From: Konstantin Belousov <kostikbel at gmail.com>
> Subject: Re: Waiting for bufdaemon
> Date: Fri, 5 Mar 2021 22:43:58 +0200
>
> > My belief is that this commit helps more users than it hurts. Namely,
> > the VMWare and KVM users, which are majority, use fast timecounter,
> > comparing to the more niche hypervisors like VirtualBox.
> >
> > For you, a simple but manual workaround, setting the timecounter to
> > ACPI (?) or might be HPET, with a loader tunable, should do it.
>
> Then please let me know the name of it.
>
> I have experienced same situation several time. That is, I faced a
> problem and asked for it on ML. Then I was told to try some tunable.
> So I thought there may be tunable that can be used as workaround in
> this case. But for those who isn't familiar with kernel internal, it
> it quite hard to find it without knowing its name. If all tunable were
> listed with brief explanation in one document, then I saw it and could
> pick up possible candidates. But actually they are documented
> separately among many man pages. So the first difficulty is to find
> man page in which possible tunable may be explained. If the problem is
> releted to some device, it is most hopeful to check its man page. But
> in this case, even after reading the commit message, I had no idea
> which man page to check.
Its somewhat messy but there is a technique of using
the "timecounter" in kib's wording to explore:
# sysctl -a | grep -i timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.choice: ACPI-fast(900) i8254(0) HPET(950) TSC-low(1000) dummy(-1000000)
kern.timecounter.hardware: TSC-low
kern.timecounter.alloweddeviation: 5
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.counter: 3054367693
kern.timecounter.tc.ACPI-fast.mask: 4294967295
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.counter: 43913
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.HPET.quality: 950
kern.timecounter.tc.HPET.frequency: 14318180
kern.timecounter.tc.HPET.counter: 3575335307
kern.timecounter.tc.HPET.mask: 4294967295
kern.timecounter.tc.TSC-low.quality: 1000
kern.timecounter.tc.TSC-low.frequency: 1696849832
kern.timecounter.tc.TSC-low.counter: 590106679
kern.timecounter.tc.TSC-low.mask: 4294967295
Given the references to ACPI and HPET in kib's
wording, notable seems to be (from one of my
contexts):
kern.timecounter.choice: ACPI-fast(900) i8254(0) HPET(950) TSC-low(1000) dummy(-1000000)
kern.timecounter.hardware: TSC-low
The descriptions of those two look like:
# sysctl -ad kern.timecounter.choice kern.timecounter.hardware
kern.timecounter.choice: Timecounter hardware detected
kern.timecounter.hardware: Timecounter hardware selected
The "selected" wording suggests that kern.timecounter.hardware
might be able to be assigned --and kib's wording would imply
that it can be.
Looking at the descriptions without also looking at the
values need not be clear:
# sysctl -ad | grep -i timecounter
kern.timecounter:
kern.timecounter.tsc_shift: Shift to pre-apply for the maximum TSC frequency
kern.timecounter.smp_tsc_adjust: Try to adjust TSC on APs to match BSP
kern.timecounter.smp_tsc: Indicates whether the TSC is safe to use in SMP mode
kern.timecounter.invariant_tsc: Indicates whether the TSC is P-state invariant
kern.timecounter.fast_gettime: Enable fast time of day
kern.timecounter.tick: Approximate number of hardclock ticks in a millisecond
kern.timecounter.choice: Timecounter hardware detected
kern.timecounter.hardware: Timecounter hardware selected
kern.timecounter.alloweddeviation: Allowed time interval deviation in percents
kern.timecounter.timehands_count: Count of timehands in rotation
kern.timecounter.stepwarnings: Log time steps
kern.timecounter.tc:
kern.timecounter.tc.ACPI-fast: timecounter description
kern.timecounter.tc.ACPI-fast.quality: goodness of time counter
kern.timecounter.tc.ACPI-fast.frequency: timecounter frequency
kern.timecounter.tc.ACPI-fast.counter: current timecounter value
kern.timecounter.tc.ACPI-fast.mask: mask for implemented bits
kern.timecounter.tc.i8254: timecounter description
kern.timecounter.tc.i8254.quality: goodness of time counter
kern.timecounter.tc.i8254.frequency: timecounter frequency
kern.timecounter.tc.i8254.counter: current timecounter value
kern.timecounter.tc.i8254.mask: mask for implemented bits
kern.timecounter.tc.HPET: timecounter description
kern.timecounter.tc.HPET.quality: goodness of time counter
kern.timecounter.tc.HPET.frequency: timecounter frequency
kern.timecounter.tc.HPET.counter: current timecounter value
kern.timecounter.tc.HPET.mask: mask for implemented bits
kern.timecounter.tc.TSC-low: timecounter description
kern.timecounter.tc.TSC-low.quality: goodness of time counter
kern.timecounter.tc.TSC-low.frequency: timecounter frequency
kern.timecounter.tc.TSC-low.counter: current timecounter value
kern.timecounter.tc.TSC-low.mask: mask for implemented bits
Looking at both can tend to narrow things down.
Not exactly direct, but useful. It might have been
harder before having kib's wording that used
terminology (notation) involved in the use of the
systctl commands.
I'll note that in the context I'm using for this:
# sysctl -ad | wc
11153 57922 604736
# sysctl -a | wc
13080 29457 449667
So: not a trivial amount of material.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Fri Mar 5 23:34:59 UTC 2021 :
> From: Konstantin Belousov <kostikbel at gmail.com>
> Subject: Re: Waiting for bufdaemon
> Date: Fri, 5 Mar 2021 22:43:58 +0200
>
> > My belief is that this commit helps more users than it hurts. Namely,
> > the VMWare and KVM users, which are majority, use fast timecounter,
> > comparing to the more niche hypervisors like VirtualBox.
> >
> > For you, a simple but manual workaround, setting the timecounter to
> > ACPI (?) or might be HPET, with a loader tunable, should do it.
>
> Then please let me know the name of it.
>
> I have experienced same situation several time. That is, I faced a
> problem and asked for it on ML. Then I was told to try some tunable.
> So I thought there may be tunable that can be used as workaround in
> this case. But for those who isn't familiar with kernel internal, it
> it quite hard to find it without knowing its name. If all tunable were
> listed with brief explanation in one document, then I saw it and could
> pick up possible candidates. But actually they are documented
> separately among many man pages. So the first difficulty is to find
> man page in which possible tunable may be explained. If the problem is
> releted to some device, it is most hopeful to check its man page. But
> in this case, even after reading the commit message, I had no idea
> which man page to check.
the "timecounter" in kib's wording to explore:
# sysctl -a | grep -i timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.choice: ACPI-fast(900) i8254(0) HPET(950) TSC-low(1000) dummy(-1000000)
kern.timecounter.hardware: TSC-low
kern.timecounter.alloweddeviation: 5
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.counter: 3054367693
kern.timecounter.tc.ACPI-fast.mask: 4294967295
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.counter: 43913
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.HPET.quality: 950
kern.timecounter.tc.HPET.frequency: 14318180
kern.timecounter.tc.HPET.counter: 3575335307
kern.timecounter.tc.HPET.mask: 4294967295
kern.timecounter.tc.TSC-low.quality: 1000
kern.timecounter.tc.TSC-low.frequency: 1696849832
kern.timecounter.tc.TSC-low.counter: 590106679
kern.timecounter.tc.TSC-low.mask: 4294967295
Given the references to ACPI and HPET in kib's
wording, notable seems to be (from one of my
contexts):
kern.timecounter.choice: ACPI-fast(900) i8254(0) HPET(950) TSC-low(1000) dummy(-1000000)
kern.timecounter.hardware: TSC-low
The descriptions of those two look like:
# sysctl -ad kern.timecounter.choice kern.timecounter.hardware
kern.timecounter.choice: Timecounter hardware detected
kern.timecounter.hardware: Timecounter hardware selected
The "selected" wording suggests that kern.timecounter.hardware
might be able to be assigned --and kib's wording would imply
that it can be.
Looking at the descriptions without also looking at the
values need not be clear:
# sysctl -ad | grep -i timecounter
kern.timecounter:
kern.timecounter.tsc_shift: Shift to pre-apply for the maximum TSC frequency
kern.timecounter.smp_tsc_adjust: Try to adjust TSC on APs to match BSP
kern.timecounter.smp_tsc: Indicates whether the TSC is safe to use in SMP mode
kern.timecounter.invariant_tsc: Indicates whether the TSC is P-state invariant
kern.timecounter.fast_gettime: Enable fast time of day
kern.timecounter.tick: Approximate number of hardclock ticks in a millisecond
kern.timecounter.choice: Timecounter hardware detected
kern.timecounter.hardware: Timecounter hardware selected
kern.timecounter.alloweddeviation: Allowed time interval deviation in percents
kern.timecounter.timehands_count: Count of timehands in rotation
kern.timecounter.stepwarnings: Log time steps
kern.timecounter.tc:
kern.timecounter.tc.ACPI-fast: timecounter description
kern.timecounter.tc.ACPI-fast.quality: goodness of time counter
kern.timecounter.tc.ACPI-fast.frequency: timecounter frequency
kern.timecounter.tc.ACPI-fast.counter: current timecounter value
kern.timecounter.tc.ACPI-fast.mask: mask for implemented bits
kern.timecounter.tc.i8254: timecounter description
kern.timecounter.tc.i8254.quality: goodness of time counter
kern.timecounter.tc.i8254.frequency: timecounter frequency
kern.timecounter.tc.i8254.counter: current timecounter value
kern.timecounter.tc.i8254.mask: mask for implemented bits
kern.timecounter.tc.HPET: timecounter description
kern.timecounter.tc.HPET.quality: goodness of time counter
kern.timecounter.tc.HPET.frequency: timecounter frequency
kern.timecounter.tc.HPET.counter: current timecounter value
kern.timecounter.tc.HPET.mask: mask for implemented bits
kern.timecounter.tc.TSC-low: timecounter description
kern.timecounter.tc.TSC-low.quality: goodness of time counter
kern.timecounter.tc.TSC-low.frequency: timecounter frequency
kern.timecounter.tc.TSC-low.counter: current timecounter value
kern.timecounter.tc.TSC-low.mask: mask for implemented bits
Looking at both can tend to narrow things down.
Not exactly direct, but useful. It might have been
harder before having kib's wording that used
terminology (notation) involved in the use of the
systctl commands.
I'll note that in the context I'm using for this:
# sysctl -ad | wc
11153 57922 604736
# sysctl -a | wc
13080 29457 449667
So: not a trivial amount of material.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Chris
Mar 5, 2021, 9:54:04 PM3/5/21
to Yasuhiro Kimura, freebsd...@freebsd.org
On 2021-03-05 15:33, Yasuhiro Kimura wrote:
> From: Konstantin Belousov <kost...@gmail.com>
> Subject: Re: Waiting for bufdaemon
> Date: Fri, 5 Mar 2021 22:43:58 +0200
>
>> My belief is that this commit helps more users than it hurts. Namely,
>> the VMWare and KVM users, which are majority, use fast timecounter,
>> comparing to the more niche hypervisors like VirtualBox.
>>
>> For you, a simple but manual workaround, setting the timecounter to
>> ACPI (?) or might be HPET, with a loader tunable, should do it.
>
> Then please let me know the name of it.
>
> I have experienced same situation several time. That is, I faced a
> problem and asked for it on ML. Then I was told to try some tunable.
> So I thought there may be tunable that can be used as workaround in
> this case. But for those who isn't familiar with kernel internal, it
> it quite hard to find it without knowing its name. If all tunable were
> listed with brief explanation in one document, then I saw it and could
> pick up possible candidates.
Not exactly what you're asking for. But sysctl sysctl(3) and loader(8)
> From: Konstantin Belousov <kost...@gmail.com>
> Subject: Re: Waiting for bufdaemon
> Date: Fri, 5 Mar 2021 22:43:58 +0200
>
>> My belief is that this commit helps more users than it hurts. Namely,
>> the VMWare and KVM users, which are majority, use fast timecounter,
>> comparing to the more niche hypervisors like VirtualBox.
>>
>> For you, a simple but manual workaround, setting the timecounter to
>> ACPI (?) or might be HPET, with a loader tunable, should do it.
>
> Then please let me know the name of it.
>
> I have experienced same situation several time. That is, I faced a
> problem and asked for it on ML. Then I was told to try some tunable.
> So I thought there may be tunable that can be used as workaround in
> this case. But for those who isn't familiar with kernel internal, it
> it quite hard to find it without knowing its name. If all tunable were
> listed with brief explanation in one document, then I saw it and could
> pick up possible candidates.
will provide some good clues.
HTH
--Chris
Yasuhiro Kimura
Mar 6, 2021, 5:02:11 AM3/6/21
to freebsd...@freebsd.org
From: Yasuhiro Kimura <ya...@utahime.org>
>> My belief is that this commit helps more users than it hurts. Namely,
>> the VMWare and KVM users, which are majority, use fast timecounter,
>> comparing to the more niche hypervisors like VirtualBox.
>>
>> For you, a simple but manual workaround, setting the timecounter to
>> ACPI (?) or might be HPET, with a loader tunable, should do it.
>
> Then please let me know the name of it.
From: Chris <bsd-...@bsdforge.com>
Subject: Re: Waiting for bufdaemon
Date: Fri, 05 Mar 2021 18:54:05 -0800
> Not exactly what you're asking for. But sysctl sysctl(3) and loader(8)
> will provide some good clues.
Thank you for reply.
On the system in question 'kern.timecounter.choice' and
'kern.timecounter.hardware' tunables have following values.
----------------------------------------------------------------------
yasu@rolling-vm-freebsd1[1002]% sysctl kern.timecounter.choice
kern.timecounter.choice: ACPI-fast(900) i8254(0) TSC-low(-100) dummy(-1000000)
yasu@rolling-vm-freebsd1[1003]% sysctl kern.timecounter.hardware
kern.timecounter.hardware: ACPI-fast
yasu@rolling-vm-freebsd1[1004]%
----------------------------------------------------------------------
So I tried setting the latter to 'i8254', 'TSC-low' and 'dummy', and
checked if the problem disappear. But unfortunately it still happened.
On the contrary changing the value from default made thing worse.
If it is set to either 'i8254' or 'TSC-low', timeout of bufdaemon also
happens when I shutdown the system just after bootstrap is completed.
And if it is set to 'dummy', the sytem hung up in the middle of
bootstrap.
So setting 'kern.timecounter.hardware' tunable doesn't work in my
case. Then is there any way to try?
Subject: Re: Waiting for bufdaemon
Date: Sat, 06 Mar 2021 08:33:23 +0900 (JST)
>> My belief is that this commit helps more users than it hurts. Namely,
>> the VMWare and KVM users, which are majority, use fast timecounter,
>> comparing to the more niche hypervisors like VirtualBox.
>>
>> For you, a simple but manual workaround, setting the timecounter to
>> ACPI (?) or might be HPET, with a loader tunable, should do it.
>
> Then please let me know the name of it.
Date: Sat, 6 Mar 2021 00:56:43 +0100
> see `man 4 timecounters':
>
> https://www.freebsd.org/cgi/man.cgi?query=timecounters
> see `man 4 timecounters':
>
> https://www.freebsd.org/cgi/man.cgi?query=timecounters
Date: Fri, 5 Mar 2021 17:35:14 -0800
> Its somewhat messy but there is a technique of using
> the "timecounter" in kib's wording to explore:
...
> Its somewhat messy but there is a technique of using
> the "timecounter" in kib's wording to explore:
From: Chris <bsd-...@bsdforge.com>
Subject: Re: Waiting for bufdaemon
Date: Fri, 05 Mar 2021 18:54:05 -0800
> Not exactly what you're asking for. But sysctl sysctl(3) and loader(8)
> will provide some good clues.
On the system in question 'kern.timecounter.choice' and
'kern.timecounter.hardware' tunables have following values.
----------------------------------------------------------------------
yasu@rolling-vm-freebsd1[1002]% sysctl kern.timecounter.choice
kern.timecounter.choice: ACPI-fast(900) i8254(0) TSC-low(-100) dummy(-1000000)
yasu@rolling-vm-freebsd1[1003]% sysctl kern.timecounter.hardware
kern.timecounter.hardware: ACPI-fast
yasu@rolling-vm-freebsd1[1004]%
----------------------------------------------------------------------
So I tried setting the latter to 'i8254', 'TSC-low' and 'dummy', and
checked if the problem disappear. But unfortunately it still happened.
On the contrary changing the value from default made thing worse.
If it is set to either 'i8254' or 'TSC-low', timeout of bufdaemon also
happens when I shutdown the system just after bootstrap is completed.
And if it is set to 'dummy', the sytem hung up in the middle of
bootstrap.
So setting 'kern.timecounter.hardware' tunable doesn't work in my
case. Then is there any way to try?
Alexander Leidinger via freebsd-current
Mar 6, 2021, 5:47:52 AM3/6/21
to freebsd...@freebsd.org
Quoting Konstantin Belousov <kost...@gmail.com> (from Fri, 5 Mar
2021 22:43:58 +0200):
intend to change the code to use a more suitable TC if VirtualBox is
detected?
Bye,
Alexander.
--
http://www.Leidinger.net Alex...@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org netc...@FreeBSD.org : PGP 0x8F31830F9F2772BF
Mark Millard via freebsd-current
Mar 6, 2021, 5:26:20 PM3/6/21
to Alex...@leidinger.net, freebsd-current
Alexander Leidinger Alexander at leidinger.net wrote on
Sat Mar 6 10:47:29 UTC 2021 :
> Quoting Konstantin Belousov <kostikbel at gmail.com> (from Fri, 5 Mar
kern.timecounter.choice alternatives [ACPI-fast(900)
i8254(0) TSC-low(-100) dummy(-1000000)] failed to make a
usable environment.
Sat Mar 6 10:47:29 UTC 2021 :
> Quoting Konstantin Belousov <kostikbel at gmail.com> (from Fri, 5 Mar
> 2021 22:43:58 +0200):
> . . .
> > For you, a simple but manual workaround, setting the timecounter to
> > ACPI (?) or might be HPET, with a loader tunable, should do it.
>
>
> Do you propose this to him as a test if this solves the issue with the
> intend to change the code to use a more suitable TC if VirtualBox is
> detected?
Turns out to not matter: Yasuhiro reported that all the
> > ACPI (?) or might be HPET, with a loader tunable, should do it.
>
>
> Do you propose this to him as a test if this solves the issue with the
> intend to change the code to use a more suitable TC if VirtualBox is
> detected?
kern.timecounter.choice alternatives [ACPI-fast(900)
i8254(0) TSC-low(-100) dummy(-1000000)] failed to make a
usable environment.
Kyle Evans
Mar 7, 2021, 3:26:16 PM3/7/21
to Yasuhiro Kimura, FreeBSD Current
cannot, AFAIK, influence the timecounter chosen with kenv; it has to
be set post-boot via sysctl, and if you're really unlucky it could be
that bufdaemon wedges before you manage to change it (though it seems
unlikely).
I've had some success with setting it to ACPI-fast in sysctl.conf here.
Mark Millard via freebsd-current
Mar 7, 2021, 6:00:01 PM3/7/21
to ya...@utahime.org, freebsd-current
did not cover some sysctl command line options that
are useful:
-T Display only variables that are settable via loader
(CTLFLAG_TUN).
and:
-W Display only writable variables that are not statistical. Useful
for determining the set of runtime tunable sysctls.
So:
# sysctl -aT | grep timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.alloweddeviation: 5
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.timehands_count: 2
# sysctl -aW | grep timecounter
kern.timecounter.fast_gettime: 1
kern.timecounter.hardware: TSC-low
kern.timecounter.alloweddeviation: 5
kern.timecounter.stepwarnings: 0
(kern.timecounter.choice is not settable and so
is not in either list. But I do not see an
option for listing read-only, non-loader-tunable
variables separately.)
Between the -aT output and the -aW output,
it indicates that /etc/sysctl.conf is appropriate
for kern.timecounter.hardware assignment but
/boot/loader.conf is not: writable but not a loader
tunable.
For reference:
# sysctl -aT | wc
768 1519 20989
# sysctl -aW | wc
1506 3130 44717
(Some may be in both lists.)
Konstantin Belousov
Mar 7, 2021, 7:03:52 PM3/7/21
to Kyle Evans, Yasuhiro Kimura, FreeBSD Current
On Sun, Mar 07, 2021 at 02:25:45PM -0600, Kyle Evans wrote:
> I'm going to be pedantic here, but note that this isn't a tunable. You
> cannot, AFAIK, influence the timecounter chosen with kenv; it has to
> be set post-boot via sysctl, and if you're really unlucky it could be
> that bufdaemon wedges before you manage to change it (though it seems
> unlikely).
>
> I've had some success with setting it to ACPI-fast in sysctl.conf here.
Right, I forgot that timecounter.hardware is not tunable, I thought that
> I'm going to be pedantic here, but note that this isn't a tunable. You
> cannot, AFAIK, influence the timecounter chosen with kenv; it has to
> be set post-boot via sysctl, and if you're really unlucky it could be
> that bufdaemon wedges before you manage to change it (though it seems
> unlikely).
>
> I've had some success with setting it to ACPI-fast in sysctl.conf here.
it was fixed.
https://reviews.freebsd.org/D29122
Yasuhiro Kimura
Mar 8, 2021, 10:59:02 AM3/8/21
to freebsd...@freebsd.org
From: Konstantin Belousov <kost...@gmail.com>
> On Sun, Mar 07, 2021 at 02:25:45PM -0600, Kyle Evans wrote:
>> I'm going to be pedantic here, but note that this isn't a tunable. You
>> cannot, AFAIK, influence the timecounter chosen with kenv; it has to
>> be set post-boot via sysctl, and if you're really unlucky it could be
>> that bufdaemon wedges before you manage to change it (though it seems
>> unlikely).
>>
>> I've had some success with setting it to ACPI-fast in sysctl.conf here.
> Right, I forgot that timecounter.hardware is not tunable, I thought that
> it was fixed.
>
> https://reviews.freebsd.org/D29122
I applied the patch of D29122 to 705d06b289e of main, do `make kernel`
and reboot.
Then I tried the test that consists of following 6 steps.
1. Edit /boot/loader.conf.local
2. shutdow -r now
3. Login as root
4. cd /usr/src
5. make -j 4 -s buildworld
6. shutdown -r now
Though I didn't intend it at first, I repeated this test 3 times
changing how /boot/loader.conf.local was edited at step 1. And I got
the result that was surprising and confusing, at least for me.
At first trial I changed /boot/loader.conf.local so it includes
'kern.timecounter.hardware=i8254' in it. Then timeout of bufdaemon
didn't happen at step 6. This means adding the line works as workaound
for the problem.
Next I replaced 'kern.timecounter.hardware=i8254' with
'kern.timecounter.hardware=ACPI-fast'and did test. As I wrote previous
mail, 'ACPI-fast' is the initial value of kern.timecounter.hardware on
the system in question. So I tried it for comparison purpose expecting
the problem happens. But the result is unexpected one. Timeout of
bufdaemon didn't happen at step 6 with this case either.
So tried yet another case. I removed
'kern.timecounter.hardware=ACPI-fast' from /boot/loader.conf.local and
did test again. In this case 'kern.timecounter.hardware' isn't touched
at all. So the result should have been same as 2nd case. But it was
different. Timeout of bufdeamon happend at step 6 in this case.
The result surprised and confused me with following 2 points.
(a) Setting initial value (2nd case) should be same as not touching at
all (3rd case). But actually they caused different results.
(b) 2nd case should have caused same result as one without patch. But
actually it caused different one.
I found the reason when I execute `sysctl kernel.timecount` and
checked the output.
If I do it with unpatched kernel, I get following result.
----------------------------------------------------------------------
yasu@rolling-vm-freebsd1[1005]% sysctl kern.timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 0
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.TSC-low.quality: -100
kern.timecounter.tc.TSC-low.frequency: 1500035122
kern.timecounter.tc.TSC-low.counter: 4026452692
kern.timecounter.tc.TSC-low.mask: 4294967295
yasu@rolling-vm-freebsd1[1006]%
----------------------------------------------------------------------
But if I do it with patched kernel, I get a bit different one.
----------------------------------------------------------------------
yasu@rolling-vm-freebsd1[1001]% sysctl kern.timecounter
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.TSC-low.quality: 1000
kern.timecounter.tc.TSC-low.frequency: 1500035491
kern.timecounter.tc.TSC-low.counter: 581590125
kern.timecounter.tc.TSC-low.mask: 4294967295
yasu@rolling-vm-freebsd1[1002]%
----------------------------------------------------------------------
Comparing two output there are two notable differences.
The first is the value of kern.timecounter.hardware. With unpatched
kernel it is 'ACPI-fast'. On the other hand, with patched kernel it is
'TSC-low'. It means the initial value is changed by applying the
patch. it explains why results of 2nd case and 3rd one are different.
The second is the value of kern.timecounter.tc.ACPI-fast.counter. With
unpatched kernel it is '1311282421' and with patch one it is
'401131388'. I don't know what this value means. But I guess the
difference is the reason that the result of 2nd case is different from
the one of unpatched kernel.
So now I know why unexpected result (a) and (b) happen. Furthermore,
I comfirmed that setting kern.timecounter.hardware to either 'i8254'
or 'ACPI-fast' works as workround of the problem. It is good news
anyway.
But still one question remains. Why have the value of
kern.timecounter.hardware and kern.timecounter.tc.ACPI-fast.counter
changed by applying the patch? My understanding is that it only makes
'kern.timecounter.hardware' loader tunable that can be configured with
loader.conf. Is it unexpected side effect? Or is everything expected
result?
---
Yasuhiro Kimura
Subject: Re: Waiting for bufdaemon
Date: Mon, 8 Mar 2021 02:03:13 +0200
> On Sun, Mar 07, 2021 at 02:25:45PM -0600, Kyle Evans wrote:
>> I'm going to be pedantic here, but note that this isn't a tunable. You
>> cannot, AFAIK, influence the timecounter chosen with kenv; it has to
>> be set post-boot via sysctl, and if you're really unlucky it could be
>> that bufdaemon wedges before you manage to change it (though it seems
>> unlikely).
>>
>> I've had some success with setting it to ACPI-fast in sysctl.conf here.
> Right, I forgot that timecounter.hardware is not tunable, I thought that
> it was fixed.
>
> https://reviews.freebsd.org/D29122
and reboot.
Then I tried the test that consists of following 6 steps.
1. Edit /boot/loader.conf.local
2. shutdow -r now
3. Login as root
4. cd /usr/src
5. make -j 4 -s buildworld
6. shutdown -r now
Though I didn't intend it at first, I repeated this test 3 times
changing how /boot/loader.conf.local was edited at step 1. And I got
the result that was surprising and confusing, at least for me.
At first trial I changed /boot/loader.conf.local so it includes
'kern.timecounter.hardware=i8254' in it. Then timeout of bufdaemon
didn't happen at step 6. This means adding the line works as workaound
for the problem.
Next I replaced 'kern.timecounter.hardware=i8254' with
'kern.timecounter.hardware=ACPI-fast'and did test. As I wrote previous
mail, 'ACPI-fast' is the initial value of kern.timecounter.hardware on
the system in question. So I tried it for comparison purpose expecting
the problem happens. But the result is unexpected one. Timeout of
bufdaemon didn't happen at step 6 with this case either.
So tried yet another case. I removed
'kern.timecounter.hardware=ACPI-fast' from /boot/loader.conf.local and
did test again. In this case 'kern.timecounter.hardware' isn't touched
at all. So the result should have been same as 2nd case. But it was
different. Timeout of bufdeamon happend at step 6 in this case.
The result surprised and confused me with following 2 points.
(a) Setting initial value (2nd case) should be same as not touching at
all (3rd case). But actually they caused different results.
(b) 2nd case should have caused same result as one without patch. But
actually it caused different one.
I found the reason when I execute `sysctl kernel.timecount` and
checked the output.
If I do it with unpatched kernel, I get following result.
----------------------------------------------------------------------
yasu@rolling-vm-freebsd1[1005]% sysctl kern.timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 0
kern.timecounter.invariant_tsc: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.choice: ACPI-fast(900) i8254(0) TSC-low(-100) dummy(-1000000)
kern.timecounter.hardware: ACPI-fast
kern.timecounter.alloweddeviation: 5
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.counter: 1311282421
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.mask: 4294967295
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.counter: 9931
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.TSC-low.quality: -100
kern.timecounter.tc.TSC-low.frequency: 1500035122
kern.timecounter.tc.TSC-low.counter: 4026452692
kern.timecounter.tc.TSC-low.mask: 4294967295
yasu@rolling-vm-freebsd1[1006]%
----------------------------------------------------------------------
But if I do it with patched kernel, I get a bit different one.
----------------------------------------------------------------------
yasu@rolling-vm-freebsd1[1001]% sysctl kern.timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.choice: ACPI-fast(900) i8254(0) TSC-low(1000) dummy(-1000000)
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.hardware: TSC-low
kern.timecounter.alloweddeviation: 5
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.counter: 401131388
kern.timecounter.alloweddeviation: 5
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.mask: 4294967295
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.counter: 9975
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.TSC-low.quality: 1000
kern.timecounter.tc.TSC-low.frequency: 1500035491
kern.timecounter.tc.TSC-low.counter: 581590125
kern.timecounter.tc.TSC-low.mask: 4294967295
yasu@rolling-vm-freebsd1[1002]%
----------------------------------------------------------------------
Comparing two output there are two notable differences.
The first is the value of kern.timecounter.hardware. With unpatched
kernel it is 'ACPI-fast'. On the other hand, with patched kernel it is
'TSC-low'. It means the initial value is changed by applying the
patch. it explains why results of 2nd case and 3rd one are different.
The second is the value of kern.timecounter.tc.ACPI-fast.counter. With
unpatched kernel it is '1311282421' and with patch one it is
'401131388'. I don't know what this value means. But I guess the
difference is the reason that the result of 2nd case is different from
the one of unpatched kernel.
So now I know why unexpected result (a) and (b) happen. Furthermore,
I comfirmed that setting kern.timecounter.hardware to either 'i8254'
or 'ACPI-fast' works as workround of the problem. It is good news
anyway.
But still one question remains. Why have the value of
kern.timecounter.hardware and kern.timecounter.tc.ACPI-fast.counter
changed by applying the patch? My understanding is that it only makes
'kern.timecounter.hardware' loader tunable that can be configured with
loader.conf. Is it unexpected side effect? Or is everything expected
result?
---
Yasuhiro Kimura
Yasuhiro Kimura
Mar 8, 2021, 11:51:26 AM3/8/21
to freebsd...@freebsd.org
From: Yasuhiro Kimura <ya...@utahime.org>
> But still one question remains. Why have the value of
> kern.timecounter.hardware and kern.timecounter.tc.ACPI-fast.counter
> changed by applying the patch? My understanding is that it only makes
> 'kern.timecounter.hardware' loader tunable that can be configured with
> loader.conf. Is it unexpected side effect? Or is everything expected
> result?
Oops, I made a mistake.
> If I do it with unpatched kernel, I get following result.
This isn't correct. I did it with reverted kernel (i.e. 705d06b289e of
main + revert of 84eaf2ccc6a). If I do it with really unpatch kernel,
I get same result as patched one.
Sorry for noise.
Subject: Re: Waiting for bufdaemon
Date: Tue, 09 Mar 2021 00:57:32 +0900 (JST)
> But still one question remains. Why have the value of
> kern.timecounter.hardware and kern.timecounter.tc.ACPI-fast.counter
> changed by applying the patch? My understanding is that it only makes
> 'kern.timecounter.hardware' loader tunable that can be configured with
> loader.conf. Is it unexpected side effect? Or is everything expected
> result?
> If I do it with unpatched kernel, I get following result.
main + revert of 84eaf2ccc6a). If I do it with really unpatch kernel,
I get same result as patched one.
Sorry for noise.
Kyle Evans
Mar 8, 2021, 12:07:55 PM3/8/21
to Yasuhiro Kimura, FreeBSD Current
On Mon, Mar 8, 2021 at 10:51 AM Yasuhiro Kimura <ya...@utahime.org> wrote:
>
> From: Yasuhiro Kimura <ya...@utahime.org>
> Subject: Re: Waiting for bufdaemon
> Date: Tue, 09 Mar 2021 00:57:32 +0900 (JST)
>
> > But still one question remains. Why have the value of
> > kern.timecounter.hardware and kern.timecounter.tc.ACPI-fast.counter
> > changed by applying the patch? My understanding is that it only makes
> > 'kern.timecounter.hardware' loader tunable that can be configured with
> > loader.conf. Is it unexpected side effect? Or is everything expected
> > result?
>
> Oops, I made a mistake.
>
> > If I do it with unpatched kernel, I get following result.
>
> This isn't correct. I did it with reverted kernel (i.e. 705d06b289e of
> main + revert of 84eaf2ccc6a). If I do it with really unpatch kernel,
> I get same result as patched one.
>
> Sorry for noise.
>
I've tried tracking down exactly what the problem is that causes the
>
> From: Yasuhiro Kimura <ya...@utahime.org>
> Subject: Re: Waiting for bufdaemon
> Date: Tue, 09 Mar 2021 00:57:32 +0900 (JST)
>
> > But still one question remains. Why have the value of
> > kern.timecounter.hardware and kern.timecounter.tc.ACPI-fast.counter
> > changed by applying the patch? My understanding is that it only makes
> > 'kern.timecounter.hardware' loader tunable that can be configured with
> > loader.conf. Is it unexpected side effect? Or is everything expected
> > result?
>
> Oops, I made a mistake.
>
> > If I do it with unpatched kernel, I get following result.
>
> This isn't correct. I did it with reverted kernel (i.e. 705d06b289e of
> main + revert of 84eaf2ccc6a). If I do it with really unpatch kernel,
> I get same result as patched one.
>
> Sorry for noise.
>
symptoms we're seeing, but no luck so far. I'm eyeballing the
following patch which partially reverts kib's 84eaf2ccc6aa05 ("x86:
stop punishing VMs with low priority for TSC timecounter") and only
punishes VirtualBox guests.
diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c
index 68fc57e6ea7..6f25360a67c 100644
--- a/sys/x86/x86/tsc.c
+++ b/sys/x86/x86/tsc.c
@@ -501,7 +501,12 @@ test_tsc(int adj_max_count)
uint64_t *data, *tsc;
u_int i, size, adj;
- if ((!smp_tsc && !tsc_is_invariant) || vm_guest)
+ /*
+ * Misbehavior of TSC under VirtualBox has been observed. In
+ * particular, threads doing small (~1 second) sleeps may miss their
+ * wakeup and hang around in sleep state, causing hangs on shutdown.
+ */
+ if ((!smp_tsc && !tsc_is_invariant) || vm_guest == VM_GUEST_VBOX)
return (-100);
size = (mp_maxid + 1) * 3;
data = malloc(sizeof(*data) * size * N, M_TEMP, M_WAITOK);
Ed Maste
Mar 8, 2021, 2:32:02 PM3/8/21
to Kyle Evans, Yasuhiro Kimura, FreeBSD Current
On Mon, 8 Mar 2021 at 12:07, Kyle Evans <kev...@freebsd.org> wrote:
>
> diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c
> index 68fc57e6ea7..6f25360a67c 100644
> --- a/sys/x86/x86/tsc.c
> +++ b/sys/x86/x86/tsc.c
> @@ -501,7 +501,12 @@ test_tsc(int adj_max_count)
> uint64_t *data, *tsc;
> u_int i, size, adj;
>
> - if ((!smp_tsc && !tsc_is_invariant) || vm_guest)
> + /*
> + * Misbehavior of TSC under VirtualBox has been observed. In
> + * particular, threads doing small (~1 second) sleeps may miss their
> + * wakeup and hang around in sleep state, causing hangs on shutdown.
> + */
> + if ((!smp_tsc && !tsc_is_invariant) || vm_guest == VM_GUEST_VBOX)
> return (-100);
> size = (mp_maxid + 1) * 3;
> data = malloc(sizeof(*data) * size * N, M_TEMP, M_WAITOK);
This seems like a sensible change to me. To make it explicilty clear
>
> diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c
> index 68fc57e6ea7..6f25360a67c 100644
> --- a/sys/x86/x86/tsc.c
> +++ b/sys/x86/x86/tsc.c
> @@ -501,7 +501,12 @@ test_tsc(int adj_max_count)
> uint64_t *data, *tsc;
> u_int i, size, adj;
>
> - if ((!smp_tsc && !tsc_is_invariant) || vm_guest)
> + /*
> + * Misbehavior of TSC under VirtualBox has been observed. In
> + * particular, threads doing small (~1 second) sleeps may miss their
> + * wakeup and hang around in sleep state, causing hangs on shutdown.
> + */
> + if ((!smp_tsc && !tsc_is_invariant) || vm_guest == VM_GUEST_VBOX)
> return (-100);
> size = (mp_maxid + 1) * 3;
> data = malloc(sizeof(*data) * size * N, M_TEMP, M_WAITOK);
what the comment/workaround applies to, maybe rewrite it as:
if (!smp_tsc && !tsc_is_invariant)
return (-100);
/*
* Misbehavior of TSC under VirtualBox has been observed. In
* particular, threads doing small (~1 second) sleeps may miss their
* wakeup and hang around in sleep state, causing hangs on shutdown.
*/
if (vm_guest == VM_GUEST_VBOX)
return (-100);
Yasuhiro Kimura
Mar 9, 2021, 1:39:02 AM3/9/21
to freebsd...@freebsd.org
From: Kyle Evans <kev...@freebsd.org>
> I've tried tracking down exactly what the problem is that causes the
> symptoms we're seeing, but no luck so far. I'm eyeballing the
> following patch which partially reverts kib's 84eaf2ccc6aa05 ("x86:
> stop punishing VMs with low priority for TSC timecounter") and only
> punishes VirtualBox guests.
>
> diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c
> index 68fc57e6ea7..6f25360a67c 100644
> --- a/sys/x86/x86/tsc.c
> +++ b/sys/x86/x86/tsc.c
> @@ -501,7 +501,12 @@ test_tsc(int adj_max_count)
> uint64_t *data, *tsc;
> u_int i, size, adj;
>
> - if ((!smp_tsc && !tsc_is_invariant) || vm_guest)
> + /*
> + * Misbehavior of TSC under VirtualBox has been observed. In
> + * particular, threads doing small (~1 second) sleeps may miss their
> + * wakeup and hang around in sleep state, causing hangs on shutdown.
> + */
> + if ((!smp_tsc && !tsc_is_invariant) || vm_guest == VM_GUEST_VBOX)
> return (-100);
> size = (mp_maxid + 1) * 3;
> data = malloc(sizeof(*data) * size * N, M_TEMP, M_WAITOK);
After updating to 6ffdaa5f2d4, I confirmed timeout of bufdaemon
dosen't happen even if I don't set kern.timecounter.hardware at all in
loader.conf.
Thank you for fixing the problem.
---
Yasuhiro Kimura
Subject: Re: Waiting for bufdaemon
Date: Mon, 8 Mar 2021 11:07:23 -0600
> I've tried tracking down exactly what the problem is that causes the
> symptoms we're seeing, but no luck so far. I'm eyeballing the
> following patch which partially reverts kib's 84eaf2ccc6aa05 ("x86:
> stop punishing VMs with low priority for TSC timecounter") and only
> punishes VirtualBox guests.
>
> diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c
> index 68fc57e6ea7..6f25360a67c 100644
> --- a/sys/x86/x86/tsc.c
> +++ b/sys/x86/x86/tsc.c
> @@ -501,7 +501,12 @@ test_tsc(int adj_max_count)
> uint64_t *data, *tsc;
> u_int i, size, adj;
>
> - if ((!smp_tsc && !tsc_is_invariant) || vm_guest)
> + /*
> + * Misbehavior of TSC under VirtualBox has been observed. In
> + * particular, threads doing small (~1 second) sleeps may miss their
> + * wakeup and hang around in sleep state, causing hangs on shutdown.
> + */
> + if ((!smp_tsc && !tsc_is_invariant) || vm_guest == VM_GUEST_VBOX)
> return (-100);
> size = (mp_maxid + 1) * 3;
> data = malloc(sizeof(*data) * size * N, M_TEMP, M_WAITOK);
dosen't happen even if I don't set kern.timecounter.hardware at all in
loader.conf.
Thank you for fixing the problem.
---
Yasuhiro Kimura
Reply all
Reply to author
Forward
0 new messages