Memory Scrub detects ECC errors in main memory...
Hamish Marson
the memory scrub, it logs hundreds of errors (Like several a second) about
correcting errors in ECC memory. Now the error log is adamant that there
is a memory problem, but the diag program doesn't report anything wrong
with the memory. Therefore IBM won't replace it.... Now the test that
diag runs on the memory finishes in about 3 seconds, which sounds a little
short to me for an exhaustive memory test of 512MB. Does diag actually do
any testing of memory or does it just verify that I have 2 512MB cards
in there. If so is there any program I can run that will tell me if my
memory is OK or not? If it is actually testing my memory can anyone
convince me how this machine is managing to do an exhaustive test of
512MB in 3 seconds? (I know its supposed to be fast but this is a little
rediculous)....
Oh yeah... Second question... HACMP/6000 2.1 won't run on this machine
although it will run in another R24 configured identically except that
the one that fails has an ethernet card ans the one that works has an
ATM card in it.... Could it be that HACMP picks up memopry problems
without memory scrub turned on? (I turned it off and cleared the error
logs with errclean 0)...
--
======================================================================
| Hamish Marson |
| Systems Programmer | |
| Computer Services | INTERNET h.ma...@waikato.ac.nz |
| University of Waikato | PHONE +64 7 8562889 xt 8181 |
| New Zealand | FAX +64 7 8384066 |
===========Disclaimer :- Remember. You heard it here first.===========
Susan Sterngold
already existing medical office management program. This is a risc6000
system running on AIX. AT the moment, when a patient's bill is paid, they
are deleted from the present program. We need a comprehensive patient
database running alongside and automatically updated when a patient is
seen and noted in the other program. Various consultants have offered to
do it for exorbitant amounts of money. Please give suggestions....thanks
Eliot Wilson
|> I have an RS6k model R24 with 2x 256MB cards in it, and whenever I enable
|> the memory scrub, it logs hundreds of errors (Like several a second) about
|> correcting errors in ECC memory. Now the error log is adamant that there
|> is a memory problem, but the diag program doesn't report anything wrong
|> with the memory. Therefore IBM won't replace it.... Now the test that
|> diag runs on the memory finishes in about 3 seconds, which sounds a little
|> short to me for an exhaustive memory test of 512MB. Does diag actually do
I think it tests memory as part of the power-on tests.
|> any testing of memory or does it just verify that I have 2 512MB cards
|> in there. If so is there any program I can run that will tell me if my
|> memory is OK or not? If it is actually testing my memory can anyone
This is a bug and I'm getting rather pissed at IBM for not being more
responsive about it. We upgraded 2 980B's to R24's and see the same problem
you do with the memory scrubbing errors. I opened IBM Support Line problem
number AD2141 and PMR# 1E616,082 on Nov 2,1994, and haven't had much luck
getting someone to admit it is a problem.
Another nice side effect is that if you have scrubbing turned on and
do a "shutdown -Fr", as it reboots you get "888"'s flashing away in your
face. Nice huh!?! Please open a problem w/ IBM and tell them it is a duplicate
of the one above.
Anyone else seeing trouble with R24's and memory scrubbing? Gee I wonder
if IBM even tested it?
-Eliot
|> convince me how this machine is managing to do an exhaustive test of
|> 512MB in 3 seconds? (I know its supposed to be fast but this is a little
|> rediculous)....
|>
|>
|>
|> --
|> ======================================================================
|> | Hamish Marson |
|> | Systems Programmer | |
|> | Computer Services | INTERNET h.ma...@waikato.ac.nz |
|> | University of Waikato | PHONE +64 7 8562889 xt 8181 |
|> | New Zealand | FAX +64 7 8384066 |
|> ===========Disclaimer :- Remember. You heard it here first.===========
--
^ Eliot Wilson * I OS/2 it! * ^
^ SquareD CIS AIX is ^
^ uunet.uu.net!meaddata.com!cisdfl01!eliotw Red HOT & Blue ^
Hamish Marson
> In article <3bu1b8$13...@thebes.waikato.ac.nz>, ham...@thebes.waikato.ac.nz (Hamish Marson) writes:
> |> I have an RS6k model R24 with 2x 256MB cards in it, and whenever I enable
> |> the memory scrub, it logs hundreds of errors (Like several a second) about
> |> correcting errors in ECC memory. Now the error log is adamant that there
> |> is a memory problem, but the diag program doesn't report anything wrong
> |> with the memory. Therefore IBM won't replace it.... Now the test that
> |> diag runs on the memory finishes in about 3 seconds, which sounds a little
> |> short to me for an exhaustive memory test of 512MB. Does diag actually do
> I think it tests memory as part of the power-on tests.
Yes. But I'd like to have my machine sit there and test the memory
over and over again. Like for 24 hours. Maybe a whole weekend... Now
on a sparc or Alpha its easy... (Or at least easy to getit to test the
memory on command. The psarc even says which SIMM is bad!). The rs6000
is a real pain when it comes to this. Who was it that decided against
a ROM level monitor anyway? I'd like to have a word with them! Several
bad ones in fact! (I hate keys!)
> |> any testing of memory or does it just verify that I have 2 512MB cards
> |> in there. If so is there any program I can run that will tell me if my
> |> memory is OK or not? If it is actually testing my memory can anyone
> This is a bug and I'm getting rather pissed at IBM for not being more
> responsive about it. We upgraded 2 980B's to R24's and see the same problem
> you do with the memory scrubbing errors. I opened IBM Support Line problem
> number AD2141 and PMR# 1E616,082 on Nov 2,1994, and haven't had much luck
> getting someone to admit it is a problem.
OK. if it is a bug why does it affect only 1 of my R24's? They are
identical, except that the one that DOESN"T have the ECC scrub errors
has an ATM card in it.
> Another nice side effect is that if you have scrubbing turned on and
> do a "shutdown -Fr", as it reboots you get "888"'s flashing away in your
> face. Nice huh!?! Please open a problem w/ IBM and tell them it is a duplicate
> of the one above.
> Anyone else seeing trouble with R24's and memory scrubbing? Gee I wonder
> if IBM even tested it?
The even worse news is that the engineer came out today to swap the
memory to see if it would make the problem move or go away. Now neither
of them work properly anymore. (HACMP kills them both with what seems
like bad memory detection according to IBM whereas one of them use
to work fine! GGGRRRRRR!!!!)
Eric Keen
ERRORLOG ENTRIES' deal with this?
Milton D. Miller II
Eliot Wilson <uunet.uu.net!meaddata.com!cisdfl01!eliotw> wrote:
>In article <3bu1b8$13...@thebes.waikato.ac.nz>, ham...@thebes.waikato.ac.nz (Hamish Marson) writes:
>|> I have an RS6k model R24 with 2x 256MB cards in it, and whenever I enable
>|> the memory scrub, it logs hundreds of errors (Like several a second) about
>|> correcting errors in ECC memory. Now the error log is adamant that there
Sounds like the old memory scrub kernel bug whose APAR was mentioned, but
maybe not....
>|> is a memory problem, but the diag program doesn't report anything wrong
>|> with the memory. Therefore IBM won't replace it.... Now the test that
>|> diag runs on the memory finishes in about 3 seconds, which sounds a little
>|> short to me for an exhaustive memory test of 512MB. Does diag actually do
>
>I think it tests memory as part of the power-on tests.
From my understanding (and I am *NOT* privileged to look at the code),
diag looks at the bitmap that IPL ROS leaves behind from its memory test.
>
>|> any testing of memory or does it just verify that I have 2 512MB cards
>|> in there. If so is there any program I can run that will tell me if my
>|> memory is OK or not? If it is actually testing my memory can anyone
>
>This is a bug and I'm getting rather pissed at IBM for not being more
>responsive about it. We upgraded 2 980B's to R24's and see the same problem
>you do with the memory scrubbing errors. I opened IBM Support Line problem
>number AD2141 and PMR# 1E616,082 on Nov 2,1994, and haven't had much luck
>getting someone to admit it is a problem.
>
>Another nice side effect is that if you have scrubbing turned on and
>do a "shutdown -Fr", as it reboots you get "888"'s flashing away in your
>face. Nice huh!?! Please open a problem w/ IBM and tell them it is a duplicate
>of the one above.
Also sounds like the previous one, but maybe not. Have you tried getting
a crash dump and/or enabling the kernel debugger?
>
>Anyone else seeing trouble with R24's and memory scrubbing? Gee I wonder
>if IBM even tested it?
Yes, IBM tested scrubbing on a sample of R24s. IBM may not have tested it
on your specific cpu planar. You might also double check your memory
cards to make sure they are the latest EC level.
Scrubbing Mini-FAQ (from the memory of the Author, has not been reviewed):
What does memory scrubbing do?
It uses a feature of the SCU (aka memory controller) to initiate
a background read of the memory in your machine. If it encounters
an ECC error, the data failing syndrome and memory address is logged
in the hardware registers (as it would be even if scrubbing was
turned off), then the scrubbing logic re-writes the corrected data
to memory with new ECC. (I don't remember if it rereads the block
at this point or not).
What other memory tests are performed?
As a part of the normal boot sequence, IPL ROS performs standard
memory addressing tests, including the spare bits and the ECC bits.
If a single bad bit is detected with these bit patterns, the spare
bit (the ECC code uses 39 of the available 40 bits) steering logic
will be set to replace the bad bit. In the absence of other
information, information saved in nvram will control the setting of
the spare bit logic.
What does a scrubbing error indicate:
It indicates that an single-bit error that would normally have
been corrected while the reading the data was found and good ECC
was written back to memory. The syndrome indicates which bit was
determined to have the error (determining which simm and/or bitline
line involves several table lookups).
What advantage is there to turning on scrubbing?
If you have areas of your main memory that are infrequently
accessed, scrubbing will cause a read and ECC check that will detect
flipped bits (due to alpha particles, etc) before two bits flip
causing a machine check when the data is referenced.
I am getting repeated errors. Should I be concerned?
If you are getting repeated errors to the same address, then you
may have a marginal simm. I have been told that AIX will remember
repeated failing syndromes and tell IPL ROS (via nvram) to steer
the spare bit accordingly, assuming that the bit is not otherwise
used (due to a IPL ROS pattern test failing).
milton
--
Milton Miller KB5TKF mil...@bga.com mil...@austin.ibm.com
I never speak for IBM.
Eliot Wilson
In article <e_keen.786727602@access3>, e_k...@access3.digex.net (Eric Keen) writes:
|> Didn't APAR IX40116 titled 'MEMORY SCRUB ALGORITHM ERROR CAUSES EXTRA
|> ERRORLOG ENTRIES' deal with this?
|>
Any other ideas?
-Eliot