I have FreeBSD 6.2R running on Intel SE7501HG2 motherboard with two
Xeon installed.
System detects 2 CPUs with 2 cores in each of them (total 4 cores) but
it uses only one core in each Xeon (total 2 of 4 cores are used only).
I see it in the 'top'. I tried the same hardware on FreeBSD 7.1R and
this OS uses all of the detected cores.
What should I do FreeBSD 6.2R to use all of the available cores?
I cannot upgrade it to 7.x, must use 6.x
Regards,
Robert Heron
_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-smp
To unsubscribe, send any mail to "freebsd-smp...@freebsd.org"
On Tue, Mar 10, 2009 at 11:40 AM, Robert Heron <rob...@heron.pl> wrote:
> Hi,
>
> I have FreeBSD 6.2R running on Intel SE7501HG2 motherboard with two Xeon
> installed.
> System detects 2 CPUs with 2 cores in each of them (total 4 cores) but it
> uses only one core in each Xeon (total 2 of 4 cores are used only).
> I see it in the 'top'. I tried the same hardware on FreeBSD 7.1R and this
> OS uses all of the detected cores.
>
> What should I do FreeBSD 6.2R to use all of the available cores?
You should post your dmesg output (eventually from both 6.2 and 7.1, kernel
booted verbosely with -v in bootloader) and/or submit a ProblemReport
(send-pr).
Regards,
Adrian.
I also have boxes with several Intel Xeon dual-core processors, and also got same results.
With FreeBSD 6.2R:
#sysctl -a | grep pci.enable
hw.pci.enable_io_modes: 1
I think 6.2 uses legacy interrupt (INTx) which allows sharing of IRQs and is CPU bounded(?). It is also probable that even MSI/MSIX-supported NIC drivers are used here, 6.2R still used the traditional interrupt mechanism.
With FreeBSD 7.1:
#sysctl -a | grep pci.enable
hw.pci.enable_msix: 1
hw.pci.enable_msi: 1
hw.pci.enable_io_modes: 1
Google would tell us that MSI/MSI-X allows control over which processor is chosen as the interrupt target. Does this explain the result?
http://www.onlamp.com/pub/a/bsd/2008/02/26/whats-new-in-freebsd-70.html?page=last
Regards,
Won
--- On Tue, 3/10/09, Robert Heron <rob...@heron.pl> wrote:
> Hi,
>
> I have FreeBSD 6.2R running on Intel SE7501HG2 motherboard
> with two Xeon installed.
> System detects 2 CPUs with 2 cores in each of them (total 4
> cores) but it uses only one core in each Xeon (total 2 of 4
> cores are used only).
> I see it in the 'top'. I tried the same hardware on FreeBSD
> 7.1R and this OS uses all of the detected cores.
>
> What should I do FreeBSD 6.2R to use all of the available
> cores?
> I cannot upgrade it to 7.x, must use 6.x
>
> Regards,
> Robert Heron
>
_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-smp
To unsubscribe, send any mail to "freebsd-smp...@freebsd.org"
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
MSI won't determine which CPUs are used. Probably the submitter needs to try
6.4. My guess is that 6.2 thinks his extra cores are hyperthreads and thus
has them disabled by default. You can toggle machdep.hyperthreading_allowed
(sysctl) if that is the case, but upgrading to 6.4 is probably better.
--
John Baldwin
machdep.hyperthreading_allowed
is '0' by default in FreeBSD 6.2R :(((
I just changed it now to '1' and all cores in all CPUs started to
work :)))
Very simple change and so much power more :)
Thanks a lot for this hint!!
Regards,
Robert
On Mar 10, 2009, at 3:19 PM, Lucas Holt wrote:
> Have you tried the hyperthreading setting?
>
> machdep.hyperthreading_allowed=1
> (put that in /etc/sysctl.conf)
>> To unsubscribe, send any mail to "freebsd-smp-
>> unsub...@freebsd.org"
>
>
>
> Lucas Holt
> Lu...@FoolishGames.com
> ________________________________________________________
> MidnightBSD.org (Free OS)
> JustJournal.com (Free blogging)
well certainly more power used..
It would be a good idea to do benchmarks on your workload
to check if it actually helps. This is is because
Hyperthreading on some cpu models actually reduces throughput
on some workloads because it thrashes the shared cache more.
Remember that a secondary cache miss can take up as much
extra time as dozens of instructions.
That's true.
> It would be a good idea to do benchmarks on your workload
> to check if it actually helps. This is is because
> Hyperthreading on some cpu models actually reduces throughput
> on some workloads because it thrashes the shared cache more.
No need for benchmarks here, because Robert's processors
don't do hyperthreading at all. FreeBSD 6.2 recognized
the *cores* as logical HT processors, so he needs to
set hyperthreading_allowed in order to enable the cores.
Of course, the better solution is to update to a more
recent release, such as 6.4 or even 7.1. His 6.2 is
more than 2 years old and contains lots of known bugs,
of which the wrong recognition of cores is a relatively
harmless one.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart
FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd
"That's what I love about GUIs: They make simple tasks easier,
and complex tasks impossible."
-- John William Chambless
> No need for benchmarks here, because Robert's processors
> don't do hyperthreading at all. FreeBSD 6.2 recognized
> the *cores* as logical HT processors, so he needs to
> set hyperthreading_allowed in order to enable the cores.
>
> Of course, the better solution is to update to a more
> recent release, such as 6.4 or even 7.1. His 6.2 is
> more than 2 years old and contains lots of known bugs,
> of which the wrong recognition of cores is a relatively
> harmless one.
I also tried 6.4 with exactly the same wrong results as 6.2.
7.1 is OK, but is has hyperthreading_allowed=1 by default.
Robert
That's suprising. Actually I think that all fixes to the
CPU detection code have been backported to 6-stable.
> 7.1 is OK, but is has hyperthreading_allowed=1 by default.
What happens if you set hyperthreading_allowed=0 on 7.1?
Are you 100% sure that your processor are really multi-core
and not just hyperthreaded? Please post the relevant lines
from /var/run/dmesg.boot or output from the `dmesg` command
(as Adrian already suggested), preferably on a recent
FreeBSD system (i.e. 7.1). The top 25 lines should be
sufficient.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart
FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd
"Documentation is like sex; when it's good, it's very, very good,
and when it's bad, it's better than nothing."
-- Dick Brandon
>> I also tried 6.4 with exactly the same wrong results as 6.2.
>
> That's suprising. Actually I think that all fixes to the
> CPU detection code have been backported to 6-stable.
>
Cores are detected correctly, but not used all by default
(hyperthreading_allowed=0).
When I change hyperthreading_allowed to 1 on a running system it
starts to use all cores. When switch back to 0, uses only 1 core of
each cpu.
>> 7.1 is OK, but is has hyperthreading_allowed=1 by default.
>
> What happens if you set hyperthreading_allowed=0 on 7.1?
>
no change. 7.1 uses all cores regardless of value of
hyperthreading_allowed
> Are you 100% sure that your processor are really multi-core
> and not just hyperthreaded? Please post the relevant lines
> from /var/run/dmesg.boot or output from the `dmesg` command
> (as Adrian already suggested), preferably on a recent
> FreeBSD system (i.e. 7.1). The top 25 lines should be
> sufficient.
I have tried 6.2, 6.4 and 7.1 on two different servers. The first
server of them (older) is Intel SE7501HG2 + 2 x Xeon 2.44GHz (2 cores
in each Xeon)
The second one (newer) is Intel S5000VSA + 2 x Xeon 2.66GHz (4 cores
in each Xeon)
================= dmesg 6.2 on the newer hardware ===================
>
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.2-RELEASE #1: Sun Mar 8 20:02:38 CET 2009
root@[machine name removed]:/usr/src/sys/i386/compile/
HERON-62-2009-03-08
ACPI APIC Table: <INTEL S5000VSA>
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 2.66GHz (2660.02-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf64 Stepping = 4
Features
=
0xbfebfbff
<
FPU
,VME
,DE
,PSE
,TSC
,MSR
,PAE
,MCE
,CX8
,APIC
,SEP
,MTRR
,PGE
,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0xe4bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,CNTX-
ID,CX16,<b14>,<b15>>
AMD Features=0x20100000<NX,LM>
AMD Features2=0x1<LAHF>
Cores per package: 2
Logical CPUs per core: 2
real memory = 2680160256 (2556 MB)
avail memory = 2622218240 (2500 MB)
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3
cpu4 (AP): APIC ID: 4
cpu5 (AP): APIC ID: 5
cpu6 (AP): APIC ID: 6
cpu7 (AP): APIC ID: 7
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
lapic0: Forcing LINT1 to edge trigger
kbd1 at kbdmux0
acpi0: <INTEL S5000VSA> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
cpu1: <ACPI CPU> on acpi0
acpi_throttle1: <ACPI CPU Throttling> on cpu1
acpi_throttle1: failed to attach P_CNT
device_attach: acpi_throttle1 attach returned 6
cpu2: <ACPI CPU> on acpi0
acpi_throttle2: <ACPI CPU Throttling> on cpu2
acpi_throttle2: failed to attach P_CNT
device_attach: acpi_throttle2 attach returned 6
cpu3: <ACPI CPU> on acpi0
acpi_throttle3: <ACPI CPU Throttling> on cpu3
acpi_throttle3: failed to attach P_CNT
device_attach: acpi_throttle3 attach returned 6
cpu4: <ACPI CPU> on acpi0
acpi_throttle4: <ACPI CPU Throttling> on cpu4
acpi_throttle4: failed to attach P_CNT
device_attach: acpi_throttle4 attach returned 6
cpu5: <ACPI CPU> on acpi0
acpi_throttle5: <ACPI CPU Throttling> on cpu5
acpi_throttle5: failed to attach P_CNT
device_attach: acpi_throttle5 attach returned 6
cpu6: <ACPI CPU> on acpi0
acpi_throttle6: <ACPI CPU Throttling> on cpu6
acpi_throttle6: failed to attach P_CNT
device_attach: acpi_throttle6 attach returned 6
cpu7: <ACPI CPU> on acpi0
acpi_throttle7: <ACPI CPU Throttling> on cpu7
acpi_throttle7: failed to attach P_CNT
device_attach: acpi_throttle7 attach returned 6
================== dmesg 7.1 on the older hardware ====================
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.1-RELEASE #1: Wed Mar 11 10:38:08 CET 2009
root@[machine name removed]:/usr/src/sys/i386/compile/HERON
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2392.29-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf25 Stepping = 5
Features
=
0xbfebfbff
<
FPU
,VME
,DE
,PSE
,TSC
,MSR
,PAE
,MCE
,CX8
,APIC
,SEP
,MTRR
,PGE
,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x4400<CNXT-ID,xTPR>
Logical CPUs per core: 2
real memory = 4026466304 (3839 MB)
avail memory = 3942027264 (3759 MB)
ACPI APIC Table: <INTEL S7501HG0>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 6
cpu3 (AP): APIC ID: 7
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
lapic0: Forcing LINT1 to edge trigger
kbd1 at kbdmux0
acpi0: <INTEL S7501HG0> on motherboard
ACPI Warning (dswload-0794): Type override - [DEB_] had invalid type
(Integer) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [MLIB] had invalid type
(Integer) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [DATA] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [SIO_] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [LEDP] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [GPEN] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [GPST] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [GP1N] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [WUES] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [WUSE] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [SBID] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [SWCE] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
ACPI Warning (dswload-0794): Type override - [SMIR] had invalid type
(String) for Scope operator, changed to (Scope) [20070320]
========================================================================
I observed that hardware does not make any difference.
6.2 and 6.4 uses all cores when machdep.hyperthreading_allowed=1
7.1 uses all cores regardless of value of
machdep.hyperthreading_allowed variable.
Regards,
Robert
The older box does have hyperthreading and the new one does not, so 6.x is
working properly. The issue with 7.1 is that if you use the default
scheduler (ULE) it doesn't actually honor the hyperthreading_allowed
variable. 7.x after 7.1 has been changed to at least honor it as a tunable
in the ULE case (though the sysctl doesn't work). If you use 4BSD on 7.1
then the tunable and sysctl will work fine.
--
John Baldwin
I'm afraid that's wrong.
According to your dmesg output, the older machine has two
Xeon processors which support hyperthreading, so you have
a total of four logical CPUs (but only two physical).
These Xeons are *not* multi-core, they're single-core.
The newer machine has two Xeon processors with two cores
each (not four!), again with hyperthreading, so you have
a total of eight logical CPUs.
So everything is working as expected.
Here's how you can see the details in the demsg output:
The older machine:
> Features = 0xbfebfbff <...,HTT,...>
The "HTT" bit in the features bitmask indicates that the
processor supports hyperthreading.
> Logical CPUs per core: 2
This also refers to hyperthreading: You have two logical
(i.e. hyperthreaded) CPUs per core.
> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
That's the total number of logical CPUs: You have two
processor packages, one core per package, and two HTT-
CPUs per core: 2 * 1 * 2 == 4. But you only have two
physical cores (one per processor package).
Now the newer machine:
> Features = 0xbfebfbff <...,HTT,...>
Again: "HTT" == hyperthreading supported.
> Cores per package: 2
These are a multi-core processor: There are two cores
per processor package.
> Logical CPUs per core: 2
And again hyperthreading: Two logical CPUs per core.
> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
So you get a total of eight logical CPUs, including
hyperthreading (2 * 2 * 2 == 8). Note that there are
only four physical cores.
Whether hyperthreading will improve performance is
controversial. Most people seem to believe that it
depends on your kind of application, but one thing is
sure: hyperthreading will *not* double the performance,
not even nearly. In most cases it's a matter of a few
percent only, probably barely noticeable. And in some
cases hyperthreading will make things worse.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart
FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd
"The scanf() function is a large and complex beast that often does
something almost but not quite entirely unlike what you desired."
-- Chris Torek
No, that flag just means it supports a register where you can ask for the
total number of threads/cores per package. Multi-core CPUs that don't have
hyperthreads have 'HTT' set, but when you read the registers properly you see
that they have 1 thread per core.
--
John Baldwin
Does this mean that the # of logical CPUs per core is not enough to determine whether the machine supports hyperthreading or not? You've said earlier that the newer machine is not hyperthreaded, so I'm assuming that there is another part in the dmesg output that says so. more guidance pls.
Your new machine might be hyperthreaded. I'm not really sure as it's a P4
descendant and I'm not sure if Intel released some dual-core P4 Xeon that
had 2 threads per core. The current Core processors do not implement multiple
threads in their cores, but neither of your systems have those CPUs.
--
John Baldwin
I tried this, but I can't see the variable being displayed.
#uname -a
FreeBSD 7.1-RELEASE FreeBSD 7.1-RELEASE #1: Mon Mar 16 12:06:49 PHT 2009 ...
#sysctl -a | grep kern.sched.name
kern.sched.name: 4BSD
#sysctl -a | grep machdep
machdep.acpi_timer_freq: 3579545
machdep.enable_panic_key: 0
machdep.adjkerntz: -28800
machdep.wall_cmos_clock: 1
machdep.disable_rtc_set: 0
machdep.acpi_root: 1040336
machdep.disable_mtrrs: 0
machdep.cpu_idle_hlt: 1
machdep.hlt_cpus: 0
machdep.prot_fault_translation: 0
machdep.panic_on_nmi: 1
machdep.tsc_freq: 2992517568
machdep.i8254_freq: 1193182
machdep.conspeed: 9600
machdep.gdbspeed: 9600
machdep.conrclk: 1843200
machdep.hlt_logical_cpus: 0
machdep.logical_cpus_mask: 14
How should make the variable visible? I am using IBM x3650 with a Quad-core Xeon processor.
Thank you very much.
Won
Then your machine doesn't support hyperthreading.
The machdep.hyperthreading_allowed sysctl is only present
if your machine actually has multiple logical CPUs.
If you don't see that sysctl, then your processor doesn't
have multiple hyperthreaded CPUs.
If you're interested how this is done in the source, see the
function cpu_hlt_setup() in src/sys/amd64/amd64/mp_machdep.c
or src/sys/i386/i386/mp_machdep.c, respectively.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart
FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd
C++: "an octopus made by nailing extra legs onto a dog"
-- Steve Taylor, 1998
Thanks for that further clarification. The dmesg has also justified this.
#dmesg
...
Cores per package: 4
...
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
...
> If you're interested how this is done in the source, see
> the
> function cpu_hlt_setup() in
> src/sys/amd64/amd64/mp_machdep.c
> or src/sys/i386/i386/mp_machdep.c, respectively.
>
Yes. On machines with hyperthreaded processors, you would
also see the message "Logical CPUs per core: 2" or similar
in dmesg. This message appears only if you have hyper-
threaded processors.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart
FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd
"File names are infinite in length, where infinity is set to 255 characters."
-- Peter Collinson, "The Unix File System"