Problems with AMD64 and 8 GB RAM?
Greg 'groggy' Lehey
motherboard
(http://www.msi.com.tw/program/products/server/svr/pro_svr_detail.php?UID=484).
See below for more details). I find it very unstable running with 8
GB memory, though 4 GB are not a problem. At first I thought it was
the onboard peripherals, but after disabling them it still persisted.
What's unstable? I only once got it through the boot process.
Running a 5.3-RELEASE i386 kernel it panics, though I haven't
investigated the panic (yet), since I'm not interested in the i386
kernel. The amd64 5.4-PRERELEASE kernel just hangs/freezes. When the
peripherals are enabled, it's after probing the onboard NIC (bge) and
before probing SATA (no drives present). I've done a verbose boot, of
course, but no additional information is present. The NIC is
recognized, and that's all.
Without the peripherals, but with a 3Com 3c905 PCI NIC, it continues
beyond this point, but doesn't enable the NIC. I don't have dmesg
output for these attempts, so I can't produce the exact message, and I
suspect it's not important. It continues until trying to mount NFS
file systems, where it hangs for obvious reasons. Pressing ^C causes
the system to either panic (and be unable to dump because I don't have
that much swap) or just hang.
None of these problems occur when I use 4 GB memory. About the only
strangeness, which seems to come from the BIOS, is that it recognizes
only 3.5 GB. If I put all DIMMS in, it recognizes the full 8 GB
memory.
I realize that this isn't enough to diagnose the problem. The reason
for this message now is to ask:
1. Has anybody else seen this problem?
2. Has anybody else used this hardware configuration and *not* seen
this problem?
3. Where should I look next?
I'm attaching the (non-verbose) dmesg from a successful boot.
Greg
--
See complete headers for address and phone numbers.
Mar 30 14:17:16 obelix kernel: FreeBSD 5.4-PRERELEASE #0: Tue Mar 22 04:02:17 UTC 2005
Mar 30 14:17:16 obelix kernel: root@obelix:/usr/obj/src/FreeBSD/OBELIX/src/sys/OBELIX
Mar 30 14:17:16 obelix kernel: Timecounter "i8254" frequency 1193182 Hz quality 0
Mar 30 14:17:16 obelix kernel: CPU: AMD Opteron(tm) Processor 242 (1603.65-MHz K8-class CPU)
Mar 30 14:17:16 obelix kernel: Origin = "AuthenticAMD" Id = 0xf5a Stepping = 10
Mar 30 14:17:16 obelix kernel: Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE3
6,CLFLUSH,MMX,FXSR,SSE,SSE2>
Mar 30 14:17:16 obelix kernel: AMD Features=0xe0500800<SYSCALL,NX,MMX+,LM,3DNow+,3DNow>
Mar 30 14:17:16 obelix kernel: real memory = 3756916736 (3582 MB)
Mar 30 14:17:16 obelix kernel: avail memory = 3623907328 (3456 MB)
Mar 30 14:17:16 obelix kernel: ACPI APIC Table: <VIAK8 AWRDACPI>
Mar 30 14:17:16 obelix kernel: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
Mar 30 14:17:16 obelix kernel: cpu0 (BSP): APIC ID: 0
Mar 30 14:17:16 obelix kernel: cpu1 (AP): APIC ID: 1
Mar 30 14:17:16 obelix kernel: ioapic0: Changing APIC ID to 2
Mar 30 14:17:16 obelix kernel: ioapic0 <Version 0.3> irqs 0-23 on motherboard
Mar 30 14:17:16 obelix kernel: acpi0: <VIAK8 AWRDACPI> on motherboard
Mar 30 14:17:16 obelix kernel: acpi0: Power Button (fixed)
Mar 30 14:17:16 obelix kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
Mar 30 14:17:16 obelix kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
Mar 30 14:17:16 obelix kernel: cpu0: <ACPI CPU> on acpi0
Mar 30 14:17:16 obelix kernel: cpu1: <ACPI CPU> on acpi0
Mar 30 14:17:16 obelix kernel: acpi_button0: <Power Button> on acpi0
Mar 30 14:17:16 obelix kernel: pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
Mar 30 14:17:16 obelix kernel: pci0: <ACPI PCI bus> on pcib0
Mar 30 14:17:16 obelix kernel: pcib1: <PCI-PCI bridge> at device 1.0 on pci0
Mar 30 14:17:16 obelix kernel: pci1: <PCI bus> on pcib1
Mar 30 14:17:16 obelix kernel: pci1: <display, VGA> at device 0.0 (no driver attached)
Mar 30 14:17:16 obelix kernel: xl0: <3Com 3c905C-TX Fast Etherlink XL> port 0xd000-0xd07f mem 0xfb000000-0xfb00007f irq
18 at device 7.0 on pci0
Mar 30 14:17:16 obelix kernel: miibus0: <MII bus> on xl0
Mar 30 14:17:16 obelix kernel: xlphy0: <3c905C 10/100 internal PHY> on miibus0
Mar 30 14:17:16 obelix kernel: xlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
Mar 30 14:17:16 obelix kernel: xl0: Ethernet address: 00:50:da:cf:17:d3
Mar 30 14:17:16 obelix kernel: atapci0: <VIA 8237 UDMA133 controller> port 0xd400-0xd40f,0x376,0x170-0x177,0x3f6,0x1f0-0
x1f7 at device 15.0 on pci0
Mar 30 14:17:16 obelix kernel: ata0: channel #0 on atapci0
Mar 30 14:17:16 obelix kernel: ata1: channel #1 on atapci0
Mar 30 14:17:16 obelix kernel: uhci0: <VIA 83C572 USB controller> port 0xd800-0xd81f irq 21 at device 16.0 on pci0
Mar 30 14:17:16 obelix kernel: usb0: <VIA 83C572 USB controller> on uhci0
Mar 30 14:17:16 obelix kernel: usb0: USB revision 1.0
Mar 30 14:17:16 obelix kernel: uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
Mar 30 14:17:16 obelix kernel: uhub0: 2 ports with 2 removable, self powered
Mar 30 14:17:16 obelix kernel: uhci1: <VIA 83C572 USB controller> port 0xdc00-0xdc1f irq 21 at device 16.1 on pci0
Mar 30 14:17:16 obelix kernel: usb1: <VIA 83C572 USB controller> on uhci1
Mar 30 14:17:16 obelix kernel: usb1: USB revision 1.0
Mar 30 14:17:16 obelix kernel: uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
Mar 30 14:17:16 obelix kernel: uhub1: 2 ports with 2 removable, self powered
Mar 30 14:17:16 obelix kernel: uhci2: <VIA 83C572 USB controller> port 0xe000-0xe01f irq 21 at device 16.2 on pci0
Mar 30 14:17:16 obelix kernel: usb2: <VIA 83C572 USB controller> on uhci2
Mar 30 14:17:16 obelix kernel: usb2: USB revision 1.0
Mar 30 14:17:16 obelix kernel: uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
Mar 30 14:17:16 obelix kernel: uhub2: 2 ports with 2 removable, self powered
Mar 30 14:17:16 obelix kernel: pci0: <serial bus, USB> at device 16.4 (no driver attached)
Mar 30 14:17:16 obelix kernel: isab0: <PCI-ISA bridge> at device 17.0 on pci0
Mar 30 14:17:16 obelix kernel: isa0: <ISA bus> on isab0
Mar 30 14:17:16 obelix kernel: pci0: <multimedia, audio> at device 17.5 (no driver attached)
Mar 30 14:17:16 obelix kernel: acpi_tz0: <Thermal Zone> on acpi0
Mar 30 14:17:16 obelix kernel: fdc0: <floppy drive controller> port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0
Mar 30 14:17:16 obelix kernel: sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
Mar 30 14:17:16 obelix kernel: sio0: type 16550A
Mar 30 14:17:16 obelix kernel: sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
Mar 30 14:17:16 obelix kernel: sio1: type 16550A
Mar 30 14:17:16 obelix kernel: ppc0: <Standard parallel printer port> port 0x778-0x77b,0x378-0x37f irq 7 on acpi0
Mar 30 14:17:16 obelix kernel: ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
Mar 30 14:17:16 obelix kernel: ppbus0: <Parallel port bus> on ppc0
Mar 30 14:17:16 obelix kernel: plip0: <PLIP network interface> on ppbus0
Mar 30 14:17:16 obelix kernel: lpt0: <Printer> on ppbus0
Mar 30 14:17:16 obelix kernel: lpt0: Interrupt-driven port
Mar 30 14:17:16 obelix kernel: ppi0: <Parallel I/O> on ppbus0
Mar 30 14:17:16 obelix kernel: atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
Mar 30 14:17:16 obelix kernel: atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
Mar 30 14:17:16 obelix kernel: kbd0 at atkbd0
Mar 30 14:17:16 obelix kernel: orm0: <ISA Option ROMs> at iomem 0xd0000-0xd07ff,0xc0000-0xcffff on isa0
Mar 30 14:17:16 obelix kernel: sc0: <System console> at flags 0x100 on isa0
Mar 30 14:17:16 obelix kernel: sc0: VGA <16 virtual consoles, flags=0x300>
Mar 30 14:17:16 obelix kernel: vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Mar 30 14:17:16 obelix kernel: Timecounters tick every 1.000 msec
Mar 30 14:17:16 obelix kernel: ad0: 190782MB <ST3200826A/3.01> [387621/16/63] at ata0-master UDMA100
Mar 30 14:17:16 obelix kernel: ad1: 190782MB <ST3200826A/3.01> [387621/16/63] at ata0-slave UDMA100
Mar 30 14:17:16 obelix kernel: acd0: DVDR <PIONEER DVD-RW DVR-108/1.04> at ata1-master UDMA66
Mar 30 14:17:16 obelix kernel: SMP: AP CPU #1 Launched!
Mar 30 14:17:16 obelix kernel: Mounting root from ufs:/dev/ad0s1a
Scott Long
5.3-RELEASE has a lot of problems with >4GB due to busdma issues. Those
should no longer be an issue in RELENG_5, including 5.4-PRE. You'll
need to dig in and provide some more details, I guess. I have an HDAMA
dual Opteron system that behaves fine now with 8GB of RAM, so your
problem might lie with particular hardware and/or drivers.
Scott
Steve Kargl
> None of these problems occur when I use 4 GB memory. About the only
> strangeness, which seems to come from the BIOS, is that it recognizes
> only 3.5 GB. If I put all DIMMS in, it recognizes the full 8 GB
> memory.
>
> I realize that this isn't enough to diagnose the problem. The reason
> for this message now is to ask:
>
> 1. Has anybody else seen this problem?
> 2. Has anybody else used this hardware configuration and *not* seen
> this problem?
> 3. Where should I look next?
>
Have you run sysutils/memtest86 with the 8 GB? I had
4 bad out of 12 tested where the DIMMs were Crucial
PC2700 2GB Reg. ECC DIMMs.
--
Steve
Greg 'groggy' Lehey
> On Thu, Mar 31, 2005 at 07:54:39AM +0930, Greg 'groggy' Lehey wrote:
>> None of these problems occur when I use 4 GB memory. About the only
>> strangeness, which seems to come from the BIOS, is that it recognizes
>> only 3.5 GB. If I put all DIMMS in, it recognizes the full 8 GB
>> memory.
>>
>> I realize that this isn't enough to diagnose the problem. The reason
>> for this message now is to ask:
>>
>> 1. Has anybody else seen this problem?
>> 2. Has anybody else used this hardware configuration and *not* seen
>> this problem?
>> 3. Where should I look next?
>
> Have you run sysutils/memtest86 with the 8 GB?
Heh. Difficult when the system doesn't run.
> I had 4 bad out of 12 tested where the DIMMs were Crucial PC2700 2GB
> Reg. ECC DIMMs.
OK, this makes sense. It might also explain why the 4 GB
configuration only recognizes 3.5 GB.
Greg 'groggy' Lehey
> Greg 'groggy' Lehey wrote:
>>
>> What's unstable? ... The amd64 5.4-PRERELEASE kernel just
>> hangs/freezes.
>
> 5.3-RELEASE has a lot of problems with >4GB due to busdma issues.
> Those should no longer be an issue in RELENG_5, including 5.4-PRE.
They appear to be.
>> I realize that this isn't enough to diagnose the problem. The reason
>> for this message now is to ask:
>>
>> 3. Where should I look next?
>
> You'll need to dig in and provide some more details, I guess.
Yes, my guess too.
> I have an HDAMA dual Opteron system that behaves fine now with 8GB
> of RAM, so your problem might lie with particular hardware and/or
> drivers.
As I described, it doesn't appear to be the drivers.
Steve Kargl
> On Wednesday, 30 March 2005 at 14:35:46 -0800, Steve Kargl wrote:
> > On Thu, Mar 31, 2005 at 07:54:39AM +0930, Greg 'groggy' Lehey wrote:
> >> None of these problems occur when I use 4 GB memory. About the only
> >> strangeness, which seems to come from the BIOS, is that it recognizes
> >> only 3.5 GB. If I put all DIMMS in, it recognizes the full 8 GB
> >> memory.
> >>
> >> I realize that this isn't enough to diagnose the problem. The reason
> >> for this message now is to ask:
> >>
> >
> > Have you run sysutils/memtest86 with the 8 GB?
>
> Heh. Difficult when the system doesn't run.
That's what happens when 1 of 8 (1 of 4?) DIMM is bad :-)
> > I had 4 bad out of 12 tested where the DIMMs were Crucial PC2700 2GB
> > Reg. ECC DIMMs.
>
> OK, this makes sense. It might also explain why the 4 GB
> configuration only recognizes 3.5 GB.
Search amd64 mailing list. The missing memory is reserved for
something which escapes me at the moment. Similar to the
infamous ISA memory hole.
--
Steve
Greg 'groggy' Lehey
> Greg 'groggy' Lehey wrote:
>> On Wednesday, 30 March 2005 at 15:30:37 -0700, Scott Long wrote:
>>
>>> Greg 'groggy' Lehey wrote:
>>>
>>>> I've recently acquired an AMD64 box ...
>>>>
>>>> What's unstable? ... The amd64 5.4-PRERELEASE kernel just
>>>> hangs/freezes.
>>>
>>> 5.3-RELEASE has a lot of problems with >4GB due to busdma issues.
>>> Those should no longer be an issue in RELENG_5, including 5.4-PRE.
>>
>> They appear to be.
>
As I said above (and trimmed for convenience), this problem occurs on
5.4-PRERELEASE as of yesterday morning. The dmesg shows that too.
>> As I described, it doesn't appear to be the drivers.
>
> I don't see how you proved or disproved this.
Shall I resend the original message? It seems independent of any
particular driver. That's not proof, of course, but I didn't claim it
was.
Greg 'groggy' Lehey
> On Thu, Mar 31, 2005 at 08:14:45AM +0930, Greg 'groggy' Lehey wrote:
>> On Wednesday, 30 March 2005 at 14:35:46 -0800, Steve Kargl wrote:
>>> On Thu, Mar 31, 2005 at 07:54:39AM +0930, Greg 'groggy' Lehey wrote:
>>>> None of these problems occur when I use 4 GB memory. About the only
>>>> strangeness, which seems to come from the BIOS, is that it recognizes
>>>> only 3.5 GB. If I put all DIMMS in, it recognizes the full 8 GB
>>>> memory.
>>>>
>>>> I realize that this isn't enough to diagnose the problem. The reason
>>>> for this message now is to ask:
>>>>
>>>> 3. Where should I look next?
>>>
>>> Have you run sysutils/memtest86 with the 8 GB?
>>
>> Heh. Difficult when the system doesn't run.
>
> That's what happens when 1 of 8 (1 of 4?) DIMM is bad :-)
I've booted with the other 2 DIMMs now (I have 4 2 GB DIMMs, all the
MB will hold). No problems. See my last reply to Scott: I'm
wondering if the system is ignoring the PCI hole.
Peter Wemm
Greg: The busdma problems from 5.3-RELEASE are fixed. That doesn't
mean that there are no *other* problems. Scott is saying "the old
busdma bug shouldn't be affecting 5.4-PRE", and he's correct.
Most likely, something else is happening, eg: you're running out of KVM
or something silly like that. I know we're right on the brink at 8GB.
The layout of the devices may be just enough to tip it over the edge.
--
Peter Wemm - pe...@wemm.org; pe...@FreeBSD.org; pe...@yahoo-inc.com
"All of this is for nothing if we don't go to the stars" - JMS/B5
Greg 'groggy' Lehey
> Greg 'groggy' Lehey wrote:
>>
>>> On Thu, Mar 31, 2005 at 07:54:39AM +0930, Greg 'groggy' Lehey wrote:
>>>
>>>> None of these problems occur when I use 4 GB memory. About the only
>>>> strangeness, which seems to come from the BIOS, is that it recognizes
>>>> only 3.5 GB. If I put all DIMMS in, it recognizes the full 8 GB
>>>> memory.
>>>
>>> Reg. ECC DIMMs.
>>
>> OK, this makes sense. It might also explain why the 4 GB
>> configuration only recognizes 3.5 GB.
>
> place, since 4+ GB is so easy to get and people don't seem to understand
> the PC architecture very well.
That's not easy to understand when it's barely documented. Thanks for
the info: it helps a lot.
This may still be a hint, though: that memory hole doesn't show up
during a boot with 8 GB RAM. How come? Is the system trying to map
RAM over the PCI hole?
It looks as if I should get a verbose boot listing with 8 GB. It'll
be a couple of hours before I find time to reboot this machine. In
the meantime, there's a verbose boot with 4 GB at
http://www.lemis.com/grog/Images/20050331/obelix-dmesg. I'm told it
shows a number of strange things, including incorrect reporting of
on-chip cache sizes.
Scott Long
The SMAP will show the hole. It's well documented in most PC
archtitecure books.
Scott
Scott Long
> On Wednesday, 30 March 2005 at 16:04:44 -0700, Scott Long wrote:
>
>>Greg 'groggy' Lehey wrote:
>>
>>>On Wednesday, 30 March 2005 at 15:30:37 -0700, Scott Long wrote:
>>>
>>>
>>>>Greg 'groggy' Lehey wrote:
>>>>
>>>>
>>>>>I've recently acquired an AMD64 box ...
>>>>>
>>>>>What's unstable? ... The amd64 5.4-PRERELEASE kernel just
>>>>>hangs/freezes.
>>>>
>>>>5.3-RELEASE has a lot of problems with >4GB due to busdma issues.
>>>>Those should no longer be an issue in RELENG_5, including 5.4-PRE.
>>>
>>>They appear to be.
>>
>>I don't understand what you mean here.
>
>
> As I said above (and trimmed for convenience), this problem occurs on
> 5.4-PRERELEASE as of yesterday morning. The dmesg shows that too.
>
And you're certain that it's due to the same busdma issues that I was
describing? I must have missed the evidence that you use to support
this.
>
>>>As I described, it doesn't appear to be the drivers.
>>
>>I don't see how you proved or disproved this.
>
>
> Shall I resend the original message? It seems independent of any
> particular driver. That's not proof, of course, but I didn't claim it
> was.
Again, I must have missed the part where you investigated the drivers
that apply to your particular system. I highly doubt that they apply to
every 8GB Opteron system available on the market.
Scott
Scott Long
> On Wednesday, 30 March 2005 at 15:30:37 -0700, Scott Long wrote:
>
>>Greg 'groggy' Lehey wrote:
>>
>>>I've recently acquired an AMD64 box ...
>>>
>>>What's unstable? ... The amd64 5.4-PRERELEASE kernel just
>>>hangs/freezes.
>>
>>5.3-RELEASE has a lot of problems with >4GB due to busdma issues.
>>Those should no longer be an issue in RELENG_5, including 5.4-PRE.
>
>
> They appear to be.
>
I don't understand what you mean here.
>
>>>I realize that this isn't enough to diagnose the problem. The reason
>>>for this message now is to ask:
>>>
>>>3. Where should I look next?
>>
>>You'll need to dig in and provide some more details, I guess.
>
>
> Yes, my guess too.
>
>
>>I have an HDAMA dual Opteron system that behaves fine now with 8GB
>>of RAM, so your problem might lie with particular hardware and/or
>>drivers.
>
>
> As I described, it doesn't appear to be the drivers.
I don't see how you proved or disproved this.
Scott
Ask Bjørn Hansen
On Thu, 31 Mar 2005 08:14:45 +0930 "Greg 'groggy' Lehey" <gr...@FreeBSD.org>
wrote:
>> Have you run sysutils/memtest86 with the 8 GB?
>
>Heh. Difficult when the system doesn't run.
There is a bootable ISO version of memtest86 that you could try.
- ask
Scott Long
No, and I'm going to make this an FAQ and post it in a very obvious
place, since 4+ GB is so easy to get and people don't seem to understand
the PC architecture very well.
Almost all systems put the PCI Memory Mapped IO window into the 3.75-4GB
region of the physical memory map. The registers for the APICs and
other system resources are also typically in this region. Now with
PCI-Express, the Memory Mapped PCI config registers are typically being
mapped in the 3.5-3.75GB range. The memory controllers, host bridges,
north-bridges, and/or whatever else glues the memory to the bus to the
CPU decode these addresses into PCI cycles, not RAM cycles. Some
systems are smart and re-map the RAM that is hidden by these holes into
a region >4GB. Some systems are dumb, though, and just deny you access
to the RAM that is covered up. It's very much like the old days of the
XT/AT architecture when you had 1MB of RAM but everything above 640k was
hidden by the VGA framebuffer, ISA option ROMs, and system BIOS, but
some systems where smart enough to relocate the hidden RAM.
So, your missing .5GB is almost certainly not due to defective RAM, it's
just due to The Way Things Are. It's a lot harder for Opteron systems
to be smart about this than Xeon systems since all of the remapping
magic can happen in the hostbridge on the Xeon, while the Opertons need
to have their built-in memory controllers programmed specially for it.
Scott
Peter Wemm
> On Wednesday, 30 March 2005 at 16:01:14 -0700, Scott Long wrote:
> > Greg 'groggy' Lehey wrote:
> >> On Wednesday, 30 March 2005 at 14:35:46 -0800, Steve Kargl wrote:
> >>> On Thu, Mar 31, 2005 at 07:54:39AM +0930, Greg 'groggy' Lehey
wrote:
> >>>> None of these problems occur when I use 4 GB memory. About the
> >>>> only strangeness, which seems to come from the BIOS, is that it
> >>>> recognizes only 3.5 GB. If I put all DIMMS in, it recognizes
> >>>> the full 8 GB memory.
> >>>
> >>> I had 4 bad out of 12 tested where the DIMMs were Crucial PC2700
> >>> 2GB Reg. ECC DIMMs.
> >>
> >> OK, this makes sense. It might also explain why the 4 GB
> >> configuration only recognizes 3.5 GB.
> >
> > No, and I'm going to make this an FAQ and post it in a very obvious
> > place, since 4+ GB is so easy to get and people don't seem to
> > understand the PC architecture very well.
>
> That's not easy to understand when it's barely documented. Thanks
> for the info: it helps a lot.
>
> This may still be a hint, though: that memory hole doesn't show up
> during a boot with 8 GB RAM. How come? Is the system trying to map
> RAM over the PCI hole?
Nope, its still there. When you boot -v, you'll see the hole in the
"Physical memory chunk(s)" list.
However, I suspect that some of the bioses will set the 4GB hole
partition in the physical ram lower so that there will be 4.5GB of ram
above the 4GB mark. I haven't looked too closely to see for sure.
> It looks as if I should get a verbose boot listing with 8 GB. It'll
> be a couple of hours before I find time to reboot this machine. In
> the meantime, there's a verbose boot with 4 GB at
> http://www.lemis.com/grog/Images/20050331/obelix-dmesg. I'm told it
> shows a number of strange things, including incorrect reporting of
> on-chip cache sizes.
Nope, it is correct. You have 1MB of L2 cache.
L1 data cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative
L1 instruction cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way
associative
L2 unified cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way
associative
> Greg
> --
> See complete headers for address and phone numbers.
--
Peter Wemm
Thats what the port does.. It produces a bootable floppy or ISO.
Greg 'groggy' Lehey
> On Wednesday 30 March 2005 03:09 pm, Greg 'groggy' Lehey wrote:
>> On Wednesday, 30 March 2005 at 16:04:44 -0700, Scott Long wrote:
>>> Greg 'groggy' Lehey wrote:
>>>> As I described, it doesn't appear to be the drivers.
>>>
>>> I don't see how you proved or disproved this.
>>
>> Shall I resend the original message? It seems independent of any
>> particular driver. That's not proof, of course, but I didn't claim
>> it was.
>
> Greg: The busdma problems from 5.3-RELEASE are fixed. That doesn't
> mean that there are no *other* problems. Scott is saying "the old
> busdma bug shouldn't be affecting 5.4-PRE", and he's correct.
Yes, now I understand.
> Most likely, something else is happening, eg: you're running out of KVM
> or something silly like that. I know we're right on the brink at 8GB.
> The layout of the devices may be just enough to tip it over the edge.
Yes, this seems reasonable. Where should I look next? I'm currently
rebuilding world and will attempt a verbose boot via serial console
when it's done. Anything else I should try?
Mike Hunter
> On Wednesday 30 March 2005 03:22 pm, Ask Bjørn Hansen wrote:
> > ...... Original Message .......
> > On Thu, 31 Mar 2005 08:14:45 +0930 "Greg 'groggy' Lehey"
> > <gr...@FreeBSD.org>
> >
> > wrote:
> > >> Have you run sysutils/memtest86 with the 8 GB?
> > >
> > >Heh. Difficult when the system doesn't run.
> >
> > There is a bootable ISO version of memtest86 that you could try.
>
> Thats what the port does.. It produces a bootable floppy or ISO.
This reminds me, I noticed that gentoo includes a memtest86 "kernel" in
their install ISO. Would this be a hard feature to include in FreeBSD?
Mike
M. Warner Losh
"Greg 'groggy' Lehey" <gr...@freebsd.org> writes:
: I've booted with the other 2 DIMMs now (I have 4 2 GB DIMMs, all the
: MB will hold). No problems. See my last reply to Scott: I'm
: wondering if the system is ignoring the PCI hole.
Unlikely. If it was, you'd not have enough of a system to complain
about.
Warner
Greg 'groggy' Lehey
[gratuitous empty lines removed]
On Wednesday, 30 March 2005 at 16:23:34 -0700, Scott Long wrote:
> Greg 'groggy' Lehey wrote:
>> On Wednesday, 30 March 2005 at 16:04:44 -0700, Scott Long wrote:
>>> Greg 'groggy' Lehey wrote:
>>>> On Wednesday, 30 March 2005 at 15:30:37 -0700, Scott Long wrote:
>>>>> Greg 'groggy' Lehey wrote:
>>>>>> I've recently acquired an AMD64 box ...
>>>>>>
>>>>>> What's unstable? ... The amd64 5.4-PRERELEASE kernel just
>>>>>> hangs/freezes.
>>>>>
>>>>> 5.3-RELEASE has a lot of problems with >4GB due to busdma issues.
>>>>> Those should no longer be an issue in RELENG_5, including 5.4-PRE.
>>>>
>>>> They appear to be.
>>>
>>> I don't understand what you mean here.
>>
>> As I said above (and trimmed for convenience), this problem occurs on
>> 5.4-PRERELEASE as of yesterday morning. The dmesg shows that too.
>
> And you're certain that it's due to the same busdma issues that I
> was describing?
No.
> I must have missed the evidence that you use to support this.
I didn't give any. It appears that I misunderstood what you were
saying.
>>>> As I described, it doesn't appear to be the drivers.
>>>
>>> I don't see how you proved or disproved this.
>>
>> Shall I resend the original message? It seems independent of any
>> particular driver. That's not proof, of course, but I didn't claim it
>> was.
>
> Again, I must have missed the part where you investigated the drivers
> that apply to your particular system.
The description is still there.
> I highly doubt that they apply to every 8GB Opteron system available
> on the market.
I never suggested that they did. There's every reason to believe that
it's something to do with this particular motherboard, but that
doesn't mean that FreeBSD is blameless.
Greg
--
When replying to this message, please take care not to mutilate the
original text.
For more information, see http://www.lemis.com/email.html
Daniel O'Connor
> On Thu, Mar 31, 2005 at 10:32:33AM +0930, Daniel O'Connor wrote:
> > On Thu, 31 Mar 2005 08:14, Greg 'groggy' Lehey wrote:
> > > > Have you run sysutils/memtest86 with the 8 GB?
> > >
> > > Heh. Difficult when the system doesn't run.
> >
>
> http://www.memtest.org/
Ahh well there you go :)
Thanks!
--
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
-- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
Daniel O'Connor
>
> Heh. Difficult when the system doesn't run.
You could try http://www.memtest86.com although that doesn't do >4Gb :(
--
Steve Kargl
> > > Have you run sysutils/memtest86 with the 8 GB?
> >
> > Heh. Difficult when the system doesn't run.
>
> You could try http://www.memtest86.com although that doesn't do >4Gb :(
>
--
Steve
Daniel O'Connor
> I'm pretty sure it's not the memory. I've tried each pair
> individually, and it's only when they're both in there together that
> it's a problem. And yes, I've tried them in each pair of slots.
Could be a marginal timing issue.. You could try winding out the RAM timing
slightly.
Greg 'groggy' Lehey
> On Thu, 31 Mar 2005 08:14, Greg 'groggy' Lehey wrote:
>>> Have you run sysutils/memtest86 with the 8 GB?
>>
>> Heh. Difficult when the system doesn't run.
>
> You could try http://www.memtest86.com although that doesn't do >4Gb
:(
I'm pretty sure it's not the memory. I've tried each pair
individually, and it's only when they're both in there together that
it's a problem. And yes, I've tried them in each pair of slots.
I now have dmesg output for verbose boots with both 4 GB and 8 GB
memory. The complete dmesg output is at
http://www.lemis.com/grog/Images/20050331/dmesg.4GB and
http://www.lemis.com/grog/Images/20050331/dmesg.8GB. The diffs are at
http://www.lemis.com/grog/Images/20050331/dmesg.diff. Here's a
truncated summary:
> --- dmesg.4GB Thu Mar 31 10:47:16 2005
> +++ dmesg.8GB Thu Mar 31 10:52:32 2005
> @@ -64,6 +10,7 @@
> SMAP type=01 base=0000000000100000 len=00000000dfde0000
> SMAP type=03 base=00000000dfee3000 len=000000000000d000
> SMAP type=04 base=00000000dfee0000 len=0000000000003000
> +SMAP type=01 base=0000000100000000 len=0000000100000000
> Copyright (c) 1992-2005 The FreeBSD Project.
> @@ -75,7 +22,7 @@
> Calibrating clock(s) ... i8254 clock: 1193283 Hz
> CLK_USE_I8254_CALIBRATION not specified - using default frequency
> Timecounter "i8254" frequency 1193182 Hz quality 0
> -Calibrating TSC clock ... TSC clock: 1603647337 Hz
> +Calibrating TSC clock ... TSC clock: 1603647241 Hz
> CPU: AMD Opteron(tm) Processor 242 (1603.65-MHz K8-class CPU)
> Origin = "AuthenticAMD" Id = 0xf5a Stepping = 10
> Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
> @@ -90,11 +37,12 @@
> L2 4KB data TLB: 512 entries, 4-way associative
> L2 4KB instruction TLB: 512 entries, 4-way associative
> L2 unified cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative
> -real memory = 3756916736 (3582 MB)
> +real memory = 8589934592 (8192 MB)
This is interesting in that it has gained 4.5 GB.
> Physical memory chunk(s):
> 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages)
> -0x0000000000a05000 - 0x00000000d95b7fff, 3636146176 bytes (887731 pages)
> -avail memory = 3623817216 (3455 MB)
> +0x0000000000a09000 - 0x00000000dfedffff, 3746394112 bytes (914647 pages)
> +0x0000000100000000 - 0x00000001f0fcffff, 4043112448 bytes (987088 pages)
> +avail memory = 7777177600 (7416 MB)
> ACPI APIC Table: <VIAK8 AWRDACPI>
> APIC ID: physical 0, logical 0:0
> APIC ID: physical 1, logical 0:1
> @@ -138,41 +86,12 @@
> ioapic0: intpin 9 trigger: level
> ioapic0: intpin 9 polarity: low
> lapic0: Routing NMI -> LINT1
> -A IRQ 3 (edge, high)
> -ioapic0: intpin 4 -> ISA IRQ 4 (edge, high)
> -ioapic0: intpin 5 -> ISA IRQ 5 (edge, high)
> -ioapic0: intpin 6 -> ISA IRQ 6 (edge, high)
> -ioapic0: intpin 7 -> ISA IRQ 7 (edge, high)
> -ioapic0: intpin 8 -> ISA IRQ 8 (edge, high)
> -ioapic0: intpin 9 -> ISA IRQ 9 (edge, high)
> -ioapic0: intpin 10 -> ISA IRQ 10 (edge, high)
> -ioapic0: intpin 11 -> ISA IRQ 11 (edge, high)
> -ioapic0: intpin 12 -> ISA IRQ 12 (edge, high)
> -ioapic0: intpin 13 -> ISA IRQ 13 (edge, high)
> -ioapic0: intpin 14 -> ISA IRQ 14 (edge, high)
> -ioapic0: intpin 15 -> ISA IRQ 15 (edge, high)
> -ioapic0: intpin 16 -> PCI IRQ 16 (level, low)
> -ioapic0: intpin 17 -> PCI IRQ 17 (level, low)
> -ioapic0: intpin 18 -> PCI IRQ 18 (level, low)
> -ioapic0: intpin 19 -> PCI IRQ 19 (level, low)
> -ioapic0: intpin 20 -> PCI IRQ 20 (level, low)
> -ioapic0: intpin 21 -> PCI IRQ 21 (level, low)
> -ioapic0: intpin 22 -> PCI IRQ 22 (level, low)
> -ioapic0: intpin 23 -> PCI IRQ 23 (level, low)
> -MADT: Interrupt override: source 0, irq 2
> -ioapic0: Routing IRQ 0 -> intpin 2
> -ioapic0: intpin 2 trigger: edge
> -ioapic0: intpin 2 polarity: high
> -MADT: Interrupt override: source 9, irq 9
> -ioapic0: intpin 9 trigger: level
> -ioapic0: intpin 9 polarity: low
> -lapic0: Routing NMI -> LINT1
This stuff is puzzling. I suppose it could be related. Does anybody
have any ideas?
> lapic0: LINT1 trigger: edge
> lapic0: LINT1 polarity: high
> lapic1: Routing NMI -> LINT1
> lapic1: LINT1 trigger: edge
> lapic1: LINT1 polarity: high
> -ioapic0 <Version 0.3> irqs 0-23 on motherboard
> +ioapic0 <Version 0.0> irqs 0-23 on motherboard
> cpu0 BSP:
> ID: 0x00000000 VER: 0x00040010 LDR: 0x01000000 DFR: 0x0fffffff
> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
The last lines in the 8 GB dmesg are:
> bge0: <Broadcom BCM5705 Gigabit Ethernet, ASIC rev. 0x3003> mem 0xfa000000-0xfa00ffff irq 16 at device 11.0 on pci0
> bge0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xfa000000
They're identical in each probe.
Greg 'groggy' Lehey
> Greg 'groggy' Lehey wrote:
>
>> individually, and it's only when they're both in there together that
>> it's a problem. And yes, I've tried them in each pair of slots.
>
> they aren't different pairs? Like one pair is single-sided and the
> other double-sided (had some nasty and obscure problems with such
> a combination myself)?
No, they're all the same.
Matthias Buelow
>I'm pretty sure it's not the memory. I've tried each pair
>individually, and it's only when they're both in there together that
>it's a problem. And yes, I've tried them in each pair of slots.
I'm sure you have checked this aswell but just for completeness,
they aren't different pairs? Like one pair is single-sided and the
other double-sided (had some nasty and obscure problems with such
a combination myself)?
mkb.
John Baldwin
On Mar 30, 2005, at 8:54 PM, Greg 'groggy' Lehey wrote:
>> lapic0: LINT1 trigger: edge
>> lapic0: LINT1 polarity: high
>> lapic1: Routing NMI -> LINT1
>> lapic1: LINT1 trigger: edge
>> lapic1: LINT1 polarity: high
>> -ioapic0 <Version 0.3> irqs 0-23 on motherboard
>> +ioapic0 <Version 0.0> irqs 0-23 on motherboard
>> cpu0 BSP:
>> ID: 0x00000000 VER: 0x00040010 LDR: 0x01000000 DFR: 0x0fffffff
>> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
This shows that in the - case the APIC is broken somehow (0.0 isn't a
valid I/O APIC version). It would seem that the system has mapped RAM
over top of the I/O APIC perhaps? It would be interesting to see the
contents of your MADT to see if it's trying to use a 64-bit PA for your
APIC. The local APIC portion seems ok though.
--
John Baldwin <j...@FreeBSD.org> <>< http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve" = http://www.FreeBSD.org
Greg 'groggy' Lehey
>
> On Mar 30, 2005, at 8:54 PM, Greg 'groggy' Lehey wrote:
>>> lapic0: LINT1 trigger: edge
>>> lapic0: LINT1 polarity: high
>>> lapic1: Routing NMI -> LINT1
>>> lapic1: LINT1 trigger: edge
>>> lapic1: LINT1 polarity: high
>>> -ioapic0 <Version 0.3> irqs 0-23 on motherboard
>>> +ioapic0 <Version 0.0> irqs 0-23 on motherboard
>>> cpu0 BSP:
>>> ID: 0x00000000 VER: 0x00040010 LDR: 0x01000000 DFR: 0x0fffffff
>>> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
>
> This shows that in the - case the APIC is broken somehow (0.0 isn't a
> valid I/O APIC version).
You mean the + case, I suppose. Yes, that's what I suspected.
> It would seem that the system has mapped RAM over top of the I/O
> APIC perhaps?
That's what I suspected too, but imp doesn't think so.
> It would be interesting to see the contents of your MADT to see if
> it's trying to use a 64-bit PA for your APIC.
Any suggestions about how to do so?
Greg 'groggy' Lehey
On Wednesday, 30 March 2005 at 21:28:36 -0700, Scott Long wrote:
> Greg 'groggy' Lehey wrote:
>> On Wednesday, 30 March 2005 at 23:01:03 -0500, John Baldwin wrote:
>>
>>> On Mar 30, 2005, at 8:54 PM, Greg 'groggy' Lehey wrote:
>>>
>>>>> lapic0: LINT1 trigger: edge
>>>>> lapic0: LINT1 polarity: high
>>>>> lapic1: Routing NMI -> LINT1
>>>>> lapic1: LINT1 trigger: edge
>>>>> lapic1: LINT1 polarity: high
>>>>> -ioapic0 <Version 0.3> irqs 0-23 on motherboard
>>>>> +ioapic0 <Version 0.0> irqs 0-23 on motherboard
>>>>> cpu0 BSP:
>>>>> ID: 0x00000000 VER: 0x00040010 LDR: 0x01000000 DFR: 0x0fffffff
>>>>> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
>>>
>>> This shows that in the - case the APIC is broken somehow (0.0 isn't a
>>> valid I/O APIC version).
>>
>> You mean the + case, I suppose. Yes, that's what I suspected.
>>
>>> It would seem that the system has mapped RAM over top of the I/O
>>> APIC perhaps?
>>
>> That's what I suspected too, but imp doesn't think so.
>
> I'd be more inclined to believe that there is an erroneous mapping
> by the OS, not that things are fundamentally broken in hardware.
Agreed. This has been my favourite hypothesis all along. But isn't
that what jhb is saying?
> Your SMAP table shows everything correctly. It's becoming hard to
> break through your pre-concieved notions here and explain how things
> actually work.
No, there's nothing to break through. I think you're just having
problems
1. expressing yourself, and
2. understanding what I'm saying.
I have no preconceived notions. All I can see here is an antagonistic
attitude on your part. What's the problem? You'll recall from my
first message that I asked for suggestions about how to approach the
issue. jhb provided some; you haven't so far. From what you've
written, it's unclear whether you disagree with jhb or not. If you
do, why? If you don't, what's your point here?
>>> It would be interesting to see the contents of your MADT to see if
>>> it's trying to use a 64-bit PA for your APIC.
>>
>> Any suggestions about how to do so?
>
> man acpidump
How do you run that on a system that won't boot?
Scott Long
> On Wednesday, 30 March 2005 at 23:01:03 -0500, John Baldwin wrote:
>
>>On Mar 30, 2005, at 8:54 PM, Greg 'groggy' Lehey wrote:
>>
>>>>lapic0: LINT1 trigger: edge
>>>>lapic0: LINT1 polarity: high
>>>>lapic1: Routing NMI -> LINT1
>>>>lapic1: LINT1 trigger: edge
>>>>lapic1: LINT1 polarity: high
>>>>-ioapic0 <Version 0.3> irqs 0-23 on motherboard
>>>>+ioapic0 <Version 0.0> irqs 0-23 on motherboard
>>>>cpu0 BSP:
>>>> ID: 0x00000000 VER: 0x00040010 LDR: 0x01000000 DFR: 0x0fffffff
>>>> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
>>
>>This shows that in the - case the APIC is broken somehow (0.0 isn't a
>>valid I/O APIC version).
>
>
> You mean the + case, I suppose. Yes, that's what I suspected.
>
>
>>It would seem that the system has mapped RAM over top of the I/O
>>APIC perhaps?
>
>
> That's what I suspected too, but imp doesn't think so.
>
I'd be more inclined to believe that there is an erroneous mapping by
the OS, not that things are fundamentally broken in hardware. Your SMAP
table shows everything correctly. It's becoming hard to break through
your pre-concieved notions here and explain how things actually work.
>
>>It would be interesting to see the contents of your MADT to see if
>>it's trying to use a 64-bit PA for your APIC.
>
>
> Any suggestions about how to do so?
>
man acpidump
Damian Gerow
: This reminds me, I noticed that gentoo includes a memtest86 "kernel" in
: their install ISO. Would this be a hard feature to include in FreeBSD?
In the case of AMD64, it'd be almost pointless. I've not yet been able to
run a copy of memtest86 on my machine (though it could have been due to
memory problems at the time).
memtest86+[1] works great, though. It's a fork from memtest86.
Jon Noack
You said the system worked with 4 GB (albeit detecting only 3.5 GB). My
perception of this whole ACPI thing is that it is fixed in your BIOS
(although it can be overridden by the OS). As such, the amount of RAM
you have in the machine shouldn't change acpidump results. Is that not
correct?
Jon
Jon Noack
> On Wednesday, 30 March 2005 at 22:27:43 -0700, Scott Long wrote:
> detected 3.5 GB in this configuration.
>
>>>My perception of this whole ACPI thing is that it is fixed in your
>>>BIOS (although it can be overridden by the OS). As such, the
>>>amount of RAM you have in the machine shouldn't change acpidump
>>>results. Is that not correct?
>>
>
> Ah, so you meant to say that the output from the system running with 4
> GB memory is useful? That wasn't in the man page you pointed to.
> What it does say is:
>
>>When invoked with the -t flag, the acpidump utility dumps contents of
>>the following tables:
>>
>>... MADT
>
> This may be the case, but between man page and output some terminology
> must have changed. I can't see any reference to anything like an MADT
> there. Does that mean that there isn't one, or that ACPI can't find
> it, or does the section APIC refer to/dump the MADT? Here's the
> complete output of acpidump -t, anyway:
>
> <snip acpidump output>
>
> Since I don't know anything about ACPI, this doesn't say too much to
> me. Suggestions welcome. If the APIC section is the MADT, it looks
> as if we should update the docco.
My limited research (as in, Google) shows that the MADT was defined as
part of ACPI 2.0:
http://www.microsoft.com/whdc/system/platform/64bit/IA64_ACPI.mspx
According to your previous link the motherboard specs, it supports both
ACPI 1.0A and 2.0. Perhaps there is a BIOS knob to toggle between the two?
Jon
Greg 'groggy' Lehey
> Jon Noack wrote:
Yes, this is correct. A number of people have explained why it only
detected 3.5 GB in this configuration.
>> My perception of this whole ACPI thing is that it is fixed in your
>> BIOS (although it can be overridden by the OS). As such, the
>> amount of RAM you have in the machine shouldn't change acpidump
>> results. Is that not correct?
>
> This is absolutely correct.
Ah, so you meant to say that the output from the system running with 4
GB memory is useful? That wasn't in the man page you pointed to.
What it does say is:
> When invoked with the -t flag, the acpidump utility dumps contents of
> the following tables:
>
> ... MADT
This may be the case, but between man page and output some terminology
must have changed. I can't see any reference to anything like an MADT
there. Does that mean that there isn't one, or that ACPI can't find
it, or does the section APIC refer to/dump the MADT? Here's the
complete output of acpidump -t, anyway:
/*
RSD PTR: OEM=VIAK8, ACPI_Rev=1.0x (0)
RSDT=0xdfee3000, cksum=97
*/
/*
RSDT: Length=44, Revision=1, Checksum=4,
OEMID=VIAK8, OEM Table ID=AWRDACPI, OEM Revision=0x42302e31,
Creator ID=AWRD, Creator Revision=0x0
Entries={ 0xdfee3040, 0xdfee7b40 }
*/
/*
FACP: Length=116, Revision=1, Checksum=255,
OEMID=VIAK8, OEM Table ID=AWRDACPI, OEM Revision=0x42302e31,
Creator ID=AWRD, Creator Revision=0x0
FACS=0xdfee0000, DSDT=0xdfee30c0
INT_MODEL=PIC
Preferred_PM_Profile=Unspecified (0)
SCI_INT=9
SMI_CMD=0x402f, ACPI_ENABLE=0xa1, ACPI_DISABLE=0xa0, S4BIOS_REQ=0x0
PSTATE_CNT=0x0
PM1a_EVT_BLK=0x4000-0x4003
PM1a_CNT_BLK=0x4004-0x4005
PM_TMR_BLK=0x4008-0x400b
GPE0_BLK=0x4020-0x4023
P_LVL2_LAT=101 us, P_LVL3_LAT=1001 us
FLUSH_SIZE=0, FLUSH_STRIDE=0
DUTY_OFFSET=0, DUTY_WIDTH=1
DAY_ALRM=125, MON_ALRM=126, CENTURY=50
IAPC_BOOT_ARCH=
Flags={WBINVD,PROC_C1,SLP_BUTTON,RTC_S4,RESET_REG}
RESET_REG=0x00000000:0[0] (Memory), RESET_VALUE=0x44
*/
/*
FACS: Length=64, HwSig=0x00000000, Firm_Wake_Vec=0x00000000
Global_Lock=
Flags=
Version=0
*/
/*
DSDT: Length=19020, Revision=1, Checksum=28,
OEMID=VIAK8, OEM Table ID=AWRDACPI, OEM Revision=0x1000,
Creator ID=MSFT, Creator Revision=0x100000e
*/
/*
APIC: Length=104, Revision=1, Checksum=145,
OEMID=VIAK8, OEM Table ID=AWRDACPI, OEM Revision=0x42302e31,
Creator ID=AWRD, Creator Revision=0x0
Local APIC ADDR=0xfee00000
Flags={PC-AT}
Type=Local APIC
ACPI CPU=0
Flags={ENABLED}
APIC ID=0
Type=Local APIC
ACPI CPU=1
Flags={ENABLED}
APIC ID=1
Type=IO APIC
APIC ID=2
INT BASE=0
ADDR=0x00000000fec00000
Type=INT Override
BUS=0
IRQ=0
INTR=2
Flags={Polarity=conforming, Trigger=conforming}
Type=INT Override
BUS=0
IRQ=9
INTR=9
Flags={Polarity=active-lo, Trigger=level}
Type=Local NMI
ACPI CPU=0
LINT Pin=1
Flags={Polarity=active-hi, Trigger=edge}
Type=Local NMI
ACPI CPU=1
LINT Pin=1
Flags={Polarity=active-hi, Trigger=edge}
*/
Since I don't know anything about ACPI, this doesn't say too much to
me. Suggestions welcome. If the APIC section is the MADT, it looks
as if we should update the docco.
Greg
Scott Long
This is absolutely correct.
Scott
Greg 'groggy' Lehey
Thanks for the link.
> According to your previous link the motherboard specs, it supports
> both ACPI 1.0A and 2.0. Perhaps there is a BIOS knob to toggle
> between the two?
I've taken a look, but I can't find anything.
Peter Wemm
> On Wednesday, 30 March 2005 at 22:27:43 -0700, Scott Long wrote:
> > Jon Noack wrote:
> >> On 03/30/05 23:14, Greg 'groggy' Lehey wrote:
> >>> On Wednesday, 30 March 2005 at 21:28:36 -0700, Scott Long wrote:
> >>>> Greg 'groggy' Lehey wrote:
> >>>>> On Wednesday, 30 March 2005 at 23:01:03 -0500, John Baldwin wrote:
> >>>>>> It would be interesting to see the contents of your MADT to see if
> >>>>>> it's trying to use a 64-bit PA for your APIC.
> >>>>>
> >>>>> Any suggestions about how to do so?
> >>>>
> >>>> man acpidump
> >>>
> >>> How do you run that on a system that won't boot?
> >>
> >> You said the system worked with 4 GB (albeit detecting only 3.5
> >> GB).
>
> Yes, this is correct. A number of people have explained why it only
> detected 3.5 GB in this configuration.
>
You're also being confused by the implementation of the 'real memory' report.
If you take a 30 second glance at the code, you'll see that it is reporting
the same units that the hw.maxmem tunable uses. ie: it is the LIMIT or
Highest Address that the system has, not the sum total of all the parts.
eg: see the machdep.c comment next to the printf
* Maxmem isn't the "maximum memory", it's one larger than the
* highest page of the physical address space. It should be
* called something like "Maxphyspage". We may adjust this
* based on ``hw.physmem'' and the results of the memory test.
The SMAP lines are what you need to pay attention to. In the output you
posted with 8G, you can see the 4GB going from the 4->8GB range, exactly.
SMAP type 1 is "usable memory".
-Peter
John Baldwin
On Mar 30, 2005, at 11:08 PM, Greg 'groggy' Lehey wrote:
> On Wednesday, 30 March 2005 at 23:01:03 -0500, John Baldwin wrote:
>>
>> On Mar 30, 2005, at 8:54 PM, Greg 'groggy' Lehey wrote:
>>>> lapic0: LINT1 trigger: edge
>>>> lapic0: LINT1 polarity: high
>>>> lapic1: Routing NMI -> LINT1
>>>> lapic1: LINT1 trigger: edge
>>>> lapic1: LINT1 polarity: high
>>>> -ioapic0 <Version 0.3> irqs 0-23 on motherboard
>>>> +ioapic0 <Version 0.0> irqs 0-23 on motherboard
>>>> cpu0 BSP:
>>>> ID: 0x00000000 VER: 0x00040010 LDR: 0x01000000 DFR:
>>>> 0x0fffffff
>>>> lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR:
>>>> 0x000001ff
>>
>> This shows that in the - case the APIC is broken somehow (0.0 isn't a
>> valid I/O APIC version).
>
> You mean the + case, I suppose. Yes, that's what I suspected.
>
>> It would seem that the system has mapped RAM over top of the I/O
>> APIC perhaps?
>
> That's what I suspected too, but imp doesn't think so.
Actually, if the full version register were zero, it would not have had
24 IRQs (irqs 0-23 part), so I'm not sure what it is doing. 0.3 isn't
really a valid APIC version AFAIK either, though I'm more familiar with
the versions used in Intel APICs (usually 1.1, 1.2, or 2.0).
>> It would be interesting to see the contents of your MADT to see if
>> it's trying to use a 64-bit PA for your APIC.
>
> Any suggestions about how to do so?
Boot with 4g or boot an i386 version and get acpidump -t output.
John Baldwin
It might though. Notice the change in APIC version with 4GB of RAM vs
8GB. The APIC hardware is the same, so that's already indicative of
something fishy going on. I think that his APIC address is correct
though as otherwise no interrupts at all would work and it wouldn't
claim to have 24 IRQs on the APIC in both cases. One can always boot
an i386 non-PAE kernel with 8GB in the machine and get an acpidump
John Baldwin
On Mar 31, 2005, at 12:49 AM, Greg 'groggy' Lehey wrote:
> This may be the case, but between man page and output some terminology
> must have changed. I can't see any reference to anything like an MADT
> there. Does that mean that there isn't one, or that ACPI can't find
> it, or does the section APIC refer to/dump the MADT? Here's the
> complete output of acpidump -t, anyway:
MADT is the name of the table (Multiple APIC Descriptor Table or some
such), but "APIC" is the 4 character signature of the MADT, hence
seeing 'APIC' output from acpidump -t when looking at the MADT.
Similarly, the MP Table is known as the MP Table, but the signature for
the table that you search for in the BIOS is "_MP_".
Nothing strange here, and it is giving a 64-bit PA for the I/O APIC,
albeit one that is < 4GB. One thing to verify is that the physical
addresses listed here for the APICs (0xfec00000 and 0xfee00000) aren't
included in the SMAP as valid RAM addresses in both cases. It might be
useful to boot an i386 CD with 8GB in the machine to see if the MADT
looks any different in that case.
Vivek Khera
I just ran acpidump -t on one of my dual opterons, and it produced no
MADT info, and I *know* I enabled ACPI 2.0 on it in the BIOS... Tyan
K8SR.
Vivek Khera
On Mar 31, 2005, at 1:15 AM, Damian Gerow wrote:
> In the case of AMD64, it'd be almost pointless. I've not yet been
> able to
> run a copy of memtest86 on my machine (though it could have been due to
> memory problems at the time).
>
I found that memtest86 runs on a dual opteron, but doesn't discover my
entire 4Gb of RAM. It found 4032Mb only. However, FreeBSD found the
whole 4GB. I shall try the memtest86+ next time.
John Baldwin
On Mar 31, 2005, at 1:00 AM, Jon Noack wrote:
> My limited research (as in, Google) shows that the MADT was defined as
> part of ACPI 2.0:
> http://www.microsoft.com/whdc/system/platform/64bit/IA64_ACPI.mspx
>
> According to your previous link the motherboard specs, it supports
> both ACPI 1.0A and 2.0. Perhaps there is a BIOS knob to toggle
> between the two?
It's part if ACPI 1.0 as well. Trust me, I have machines built before
ACPI 2.0 was defined that have MADTs. :)
David O'Brien
> Thus spake Mike Hunter (mhu...@ack.Berkeley.EDU) [30/03/05 18:46]:
> : This reminds me, I noticed that gentoo includes a memtest86 "kernel" in
> : their install ISO. Would this be a hard feature to include in FreeBSD?
>
> In the case of AMD64, it'd be almost pointless. I've not yet been able to
> run a copy of memtest86 on my machine (though it could have been due to
> memory problems at the time).
>
> from memtest86.
FEH! memtest86+ can be crap. Version 1.30 had a very bad bug where it
would report all memory above 4GB on Opteron systems as bad. I replaced
a motherboard and a complete set of 8x1GB DIMM's because of it. Very
unfortunately for me 1.30 was the latest version when FreeBSD/amd64 had
the busdma bug that would corrumpt data. (thus why I was bothering to
test my memory). Anyone using memtest86+ on AMD64 should make sure they
have version 1.40.
memtest86 (the original) versions below 3.2 had fatal bugs on AMD64 CPU's
also.
--
-- David (obr...@FreeBSD.org)
David O'Brien
> On Thursday, 31 March 2005 at 10:32:33 +0930, Daniel O'Connor wrote:
> > On Thu, 31 Mar 2005 08:14, Greg 'groggy' Lehey wrote:
> >>> Have you run sysutils/memtest86 with the 8 GB?
> >>
> >> Heh. Difficult when the system doesn't run.
> >
> > You could try http://www.memtest86.com although that doesn't do >4Gb
> :(
>
> I'm pretty sure it's not the memory. I've tried each pair
> individually, and it's only when they're both in there together that
> it's a problem. And yes, I've tried them in each pair of slots.
You have a dual-channel memory controller. If you insert one DIMM you
perform 64-bit data accesses. If you install DIMM's in pairs (making
sure you're using the right "paired" sockets), you perform 128-bit data
accesses. Thus your access pattern is different between these two
situations. I'm highly suspious that you can us 4x2GB DIMM's with out
knowing the exact part number. Don't forget 2GB DIMM's are
double-stacked and thus look like double the electrical bus loads. The
same is true for older 1GB DIMM's.
Install all the memory you would like to use into your motherboard,
download memtest86+ version 1.40 from http://www.memtest.org, dd to
floppy or burn the ISO, and report back your findings from running it.
Also what version of the BIOS are you using?
--
-- David (obr...@FreeBSD.org)
David O'Brien
> I've recently acquired an AMD64 box (dual Opteron 242, SiS Master@-FAR
> motherboard
> (http://www.msi.com.tw/program/products/server/svr/pro_svr_detail.php?UID=484).
> See below for more details). I find it very unstable running with 8
> GB memory, though 4 GB are not a problem.
Exactly what is the OPN (first line on the CPU's that includes the model
number)? Exactly what DIMM's are you using? Make + part number.
--
-- David (obr...@FreeBSD.org)
David O'Brien
> Greg: The busdma problems from 5.3-RELEASE are fixed. That doesn't
> mean that there are no *other* problems. Scott is saying "the old
> busdma bug shouldn't be affecting 5.4-PRE", and he's correct.
>
> Most likely, something else is happening, eg: you're running out of KVM
> or something silly like that. I know we're right on the brink at 8GB.
> The layout of the devices may be just enough to tip it over the edge.
Grog's motherboard is a 4+0 configuration -- which would mean he is using
(trying to) 2GB DIMM's. There are memory bus loading specifictions he
may be out of spec of.
--
-- David (obr...@FreeBSD.org)
David O'Brien
> > I had 4 bad out of 12 tested where the DIMMs were Crucial PC2700 2GB
> > Reg. ECC DIMMs.
>
> OK, this makes sense. It might also explain why the 4 GB
> configuration only recognizes 3.5 GB.
No. This is due to the 3.5-4.0GB PA address range that the PeeCee
architecture reserves for the PCI config space, AGP GART, memory mapped
I/O, etc... Many Opteron BIOS's don't bother to hoist the "covered"
memory above 4GB.
Please see the freebsd-amd64 archives -- this has been discussed many
times.
--
-- David (obr...@FreeBSD.org)
David O'Brien
> On Thu, 31 Mar 2005 08:14, Greg 'groggy' Lehey wrote:
> > > Have you run sysutils/memtest86 with the 8 GB?
> >
> > Heh. Difficult when the system doesn't run.
>
> You could try http://www.memtest86.com although that doesn't do >4Gb :(
Are you sure version 3.2 does not?
--
-- David (obr...@FreeBSD.org)
James R. Van Artsdalen
>I've recently acquired an AMD64 box (dual Opteron 242, SiS Master@-FAR
>motherboard I find it very unstable running with 8
>GB memory, though 4 GB are not a problem.
>
I installed 5.4 beta 1 amd64 on a 8 GB Tyan Thunder K8W with two 244
CPUs. It's been looping on buildworld for about 60 hours, no problems.
Vivek Khera
On Apr 1, 2005, at 11:22 PM, James R. Van Artsdalen wrote:
> I installed 5.4 beta 1 amd64 on a 8 GB Tyan Thunder K8W with two 244
> CPUs. It's been looping on buildworld for about 60 hours, no
> problems.
>
can you see what happens when you run very heavy network I/O through
it? I assume it has a bge network interface. I experience severe
lockups and/or timeouts every few days on a K8SR board.
Vivek Khera, Ph.D.
+1-301-869-4449 x806
Scott Long
What network interfaces do you use?
Scott
Vivek Khera
Right now I have the on-board Broadcom:
bge0: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2003> mem
0xfc9b0000-0xfc9bffff,0xfc9c0000-0xfc9cffff irq 24 at device 9.0 on
pci3
miibus0: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
However, my vendor is shipping me an Intel NIC card to try to see if
that is really the problem. but given another post on freebsd-amd64
list, I am now suspecting a PCI-x bus timing fault on the S2881
motherboard.
Greg 'groggy' Lehey
> On Thu, Mar 31, 2005 at 11:24:29AM +0930, Greg 'groggy' Lehey wrote:
>> On Thursday, 31 March 2005 at 10:32:33 +0930, Daniel O'Connor wrote:
>>> On Thu, 31 Mar 2005 08:14, Greg 'groggy' Lehey wrote:
>>>>> Have you run sysutils/memtest86 with the 8 GB?
>>>>
>>>> Heh. Difficult when the system doesn't run.
>>>
>>> You could try http://www.memtest86.com although that doesn't do >4Gb
>> :(
>>
>> I'm pretty sure it's not the memory. I've tried each pair
>> individually, and it's only when they're both in there together that
>> it's a problem. And yes, I've tried them in each pair of slots.
>
> You have a dual-channel memory controller. If you insert one DIMM you
> perform 64-bit data accesses. If you install DIMM's in pairs (making
> sure you're using the right "paired" sockets), you perform 128-bit data
> accesses. Thus your access pattern is different between these two
> situations. I'm highly suspious that you can us 4x2GB DIMM's with out
> knowing the exact part number. Don't forget 2GB DIMM's are
> double-stacked and thus look like double the electrical bus loads. The
> same is true for older 1GB DIMM's.
This looks like it's the issue.
> Install all the memory you would like to use into your motherboard,
> download memtest86+ version 1.40 from http://www.memtest.org, dd to
> floppy or burn the ISO, and report back your findings from running
> it.
Done that. It was quite revealing. This particular motherboard and
BIOS supply either "ECC off" or "ECC in chip kill mode". Mine was
off, and I got many errors. With 4 GB (any two chips) and chip kill
mode enabled, I ran memtest86+ for about 18 hours and got about 1 ECC
error per second. With 8 GB, with or without chip kill, neither
FreeBSD, memtest86+ nor Linux run reliably: memtest86+ spontaneously
reboots every 5 minutes or so.
I borrowed this memory to test the motherboard, and it's going back
this week anyway. I now have my own memory :-(all 1 GB of it), and it
tests perfectly. The memory I had the trouble with works perfectly in
the machine for which it was purchased, so it does indeed look like an
electrical loading problem.
The moral of the story is, I suppose, "don't buy the MSI K8T
Master2-FAR". I was warned about the motherboard before I bought it,
but at the time it was the only game in town. Now I know of other
places with (hopefully) better boards.
Damian Gerow
: The moral of the story is, I suppose, "don't buy the MSI K8T
: Master2-FAR". I was warned about the motherboard before I bought it,
: but at the time it was the only game in town. Now I know of other
: places with (hopefully) better boards.
I'd like to second this. I have it, and it's caused me *no* end of memory
issues. I've gone through ~6GB of 512MB and 1GB kits from OCZ which *all*
test fine when they do them, but I've only gotten one current 1GB kit to
work properly on my board. And I can't use ECC; FreeBSD and memtest86+ both
spontaneously reboot when it's enabled.
Too bad the purchase cost broke me, otherwise I'd buy a new board.
Willem Jan Withagen
> I've recently acquired an AMD64 box (dual Opteron 242, SiS Master@-FAR
> motherboard
> See below for more details). I find it very unstable running with 8
> GB memory, though 4 GB are not a problem. At first I thought it was
> the onboard peripherals, but after disabling them it still persisted.
>
> What's unstable? I only once got it through the boot process.
> Running a 5.3-RELEASE i386 kernel it panics, though I haven't
> investigated the panic (yet), since I'm not interested in the i386
> kernel. The amd64 5.4-PRERELEASE kernel just hangs/freezes. When the
> peripherals are enabled, it's after probing the onboard NIC (bge) and
> before probing SATA (no drives present). I've done a verbose boot, of
> course, but no additional information is present. The NIC is
> recognized, and that's all.
>
> Without the peripherals, but with a 3Com 3c905 PCI NIC, it continues
> beyond this point, but doesn't enable the NIC. I don't have dmesg
> output for these attempts, so I can't produce the exact message, and I
> suspect it's not important. It continues until trying to mount NFS
> file systems, where it hangs for obvious reasons. Pressing ^C causes
> the system to either panic (and be unable to dump because I don't have
> that much swap) or just hang.
>
> None of these problems occur when I use 4 GB memory. About the only
> strangeness, which seems to come from the BIOS, is that it recognizes
> only 3.5 GB. If I put all DIMMS in, it recognizes the full 8 GB
> memory.
>
> I realize that this isn't enough to diagnose the problem. The reason
> for this message now is to ask:
>
> 1. Has anybody else seen this problem?
Hi Greg,
[Currently little time so I'll dig the archives later for more details]
I'm sorry to come into this discussion after 58 messages, but this board has
been extensively discussed about 1 year ago, because it gave me trouble to no
end (even with 2Gb). One of the early amd64 developers (not David or Scott)
had the same board but could not get it stable under amd64 (i386 was fine with
2Gb). He tossed it, and suggested me to do the same. Which I did, and went to
a Tyan board S278. After that there where no more problems at all. At the time
I think things we're at 5.1 so now with 5.3 some features might have made the
board act more stable.
--WjW
Willem Jan Withagen
> I've recently acquired an AMD64 box (dual Opteron 242, SiS Master@-FAR
> motherboard
> (http://www.msi.com.tw/program/products/server/svr/pro_svr_detail.php?UID=484).
> See below for more details). I find it very unstable running with 8
> GB memory, though 4 GB are not a problem. At first I thought it was
> the onboard peripherals, but after disabling them it still persisted.
>
> What's unstable? I only once got it through the boot process.
> Running a 5.3-RELEASE i386 kernel it panics, though I haven't
> 1. Has anybody else seen this problem?
> 2. Has anybody else used this hardware configuration and *not* seen
> this problem?
[Posted something like thisearlier, but did not see it on the list]
Little late to the discussion, but none the less.
I bought this board over a year ago to run amd64 (you're running i386).
But in the end I trashed it for running amd64 since one of then involved
developers also tried to use the board without much success.
It was running fine with amd64, 1 CPU, 2Gb, but as soon as I added the
2nd CPU the slightest load crashed the system somewhere in IPI-areas.
After long discussions I came to the point that it was easier to get a new
motherboard, so I got a Tyan Tiger S275. Which as not yet failed on me.
So if ever you'd like to run amd64 on this system, even now you've determined
the problem to be the load on the memory-bus, be warned that odd things could
be happening.
--WjW
David O'Brien
> The moral of the story is, I suppose, "don't buy the MSI K8T
> Master2-FAR". I was warned about the motherboard before I bought it,
WHY?? There is nothing wrong with that motherboard -- I have three of
them. You are at fault for trying to treat a consumer desktop 2P mobo as
a high-end server one. People do not run 2GB DIMM's in an MSI K8T
Master2-FAR. If you want >4GB RAM get an eATX 4+4 DIMM configuration
motherboard from Tyan or Iwill.
The MSI K8T Master2-FAR works fine with 1GB DIMM's. Where on MSI's
website did they state 2GB double-stacked DIMM's were supported?
--
-- David (obr...@FreeBSD.org)
David O'Brien
> I'm sorry to come into this discussion after 58 messages, but this board
> has been extensively discussed about 1 year ago, because it gave me trouble
> to no end (even with 2Gb). One of the early amd64 developers (not David or
> Scott) had the same board but could not get it stable under amd64 (i386 was
> fine with 2Gb). He tossed it, and suggested me to do the same.
Hogwash. It was Peter Wemm and he was talking about the Asus SK8N, not
the MSI K8T Master2-FAR.
--
-- David (obr...@FreeBSD.org)
Willem Jan Withagen
Not true....
unfortunatley I cannot seem to find the old mails on this. Probably got lost
when upgrading my WinBox. But I'm very shure about this, why else would I burn
a nice board and get me an expensive new one??? It is still somewhere in my
storage....
I had several people look at my kerneldumps. (I even needed to fix a small bug
for that in doadump().) Nobody seemed to be able to really explain what was
wrong, and everything magically went away when I bought the Tyan board. So
please don't tell me I was smoking something illegal.
--WjW
Ketrien I. Saihr-Kesenchedra
> The MSI K8T Master2-FAR works fine with 1GB DIMM's. Where on MSI's
> website did they state 2GB double-stacked DIMM's were supported?
http://www.msicomputer.com/product/p_spec.asp?model=K8T_Master2_FAR&class=mb
[QUOTE]
Main Memory
- 144-bit DDR at 200, 266, 333MHz, 400MHz.
- Supports DIMM sizes from 64MB (128Mb x 16 DRAMs) to 2GB on each DIMM slot.
- Supports 4 DDR DIMMs upto 8GB (Registered Memory only)
[/QUOTE]
And the number of people having nothing but trouble from these boards far
outweighs the success stories. You need only look at AMD's support forums to
confirm that. (There's _how_ many threads complaining of constant problems
at this poitn?)
-ksaihr
Joel Dahl
> The MSI K8T Master2-FAR works fine with 1GB DIMM's. Where on MSI's
> website did they state 2GB double-stacked DIMM's were supported?
I haven't been following the discussion, but if we're talking about the
MSI K8T Master2-FAR, one can read the following from MSI's website:
"
• Supports DIMM sizes from 64MB (128Mb x 16 DRAMs) to 2GB.
• Supports 4 DDR DIMMs up to 8GB (Registered Memory only)
"
http://www.msi.com.tw/program/products/server/svr/pro_svr_detail.php?UID=484
:-)
--
Joel
Greg 'groggy' Lehey
> On Tue, Apr 05, 2005 at 10:09:11AM +0930, Greg 'groggy' Lehey wrote:
>> The moral of the story is, I suppose, "don't buy the MSI K8T
>> Master2-FAR". I was warned about the motherboard before I bought it,
>
> WHY?? There is nothing wrong with that motherboard -- I have three of
> them. You are at fault for trying to treat a consumer desktop 2P mobo as
> a high-end server one.
"You are at fault for trying to used a mobo in an advertised
configuration".
Sorry, David. If a board doesn't perform as advertised, it's
inferior. If you can find a way to work around, fine. I can too, and
I have done. But the ability to find a workaround for a deficiency is
no reason to say "There is nothing wrong with that motherboard".
> People do not run 2GB DIMM's in an MSI K8T Master2-FAR.
Indeed. You can't.
> The MSI K8T Master2-FAR works fine with 1GB DIMM's. Where on MSI's
> website did they state 2GB double-stacked DIMM's were supported?
I can't be bothered to search their web site. It's on page 2-8 of the
instructions supplied with the motherboard (document G52-S9130X3,
version 1.2).