Checking for Bad Blocks with Alt-F

[michae...@googlemail.com]

Hi,
so I just received my new disk today and want to run a surface scan on it.
I would usually do this with the utility 'badblocks', which according to some googling is part of the 'e2fsprogs'  package. This in turn is part of the firmware according to the listed software here: https://sites.google.com/site/altfirmware/home/software

However I am unable to find 'badblocks' in /usr/sbin, which is where it should be(?). Any idea where I can find it? Is it not included in Alt-F?
If so, any idea on how I would do a surface scan of the disk?
Thanks in advance!

Edit: In my googling I came across this sourceforge page: http://code.google.com/p/alt-f/source/browse/trunk/alt-f/package/e2fsprogs/e2fsprogs.mk?r=2280
Which mentions badblocks and it seems to be built by default. It seems that that was when e2fsprogs was still a package, whereas with 0.1RC3 it appears to be part of the firmware.

[João Cardoso]

On Saturday, February 22, 2014 1:15:17 PM UTC, michae...@googlemail.com wrote:

Hi,
so I just received my new disk today and want to run a surface scan on it.
I would usually do this with the utility 'badblocks', which according to some googling is part of the 'e2fsprogs'  package. This in turn is part of the firmware according to the listed software here: https://sites.google.com/site/altfirmware/home/software

However I am unable to find 'badblocks' in /usr/sbin, which is where it should be(?).

Yes, but 'badblocks'  is not shipped with Alt-F, saving 20KB of flash memory

I don't use badblocks since 80MegaByte disks was the standard (yes, I'm that old, and I even remember dealing with 10 and 20MB disks :-)

Modern disks have a reserved area for badblocks and automatically remap badblocks to that area. You only run into trouble when that area exhausts.

The smartmoontools (SMART) package is included in Alt-F, and you can activate it under Services->System. It will (helpfully) warn you when your disk is about to dye.

SMART, by itself, will also scan the disk surface and other disk sensitive areas, use Disk->Utilities, health to run it and/or see the SMART report.

Any idea where I can find it? Is it not included in Alt-F?
If so, any idea on how I would do a surface scan of the disk?
Thanks in advance!

Edit: In my googling I came across this sourceforge page: http://code.google.com/p/alt-f/source/browse/trunk/alt-f/package/e2fsprogs/e2fsprogs.mk?r=2280
Which mentions badblocks and it seems to be built by default. It seems that that was when e2fsprogs was still a package, whereas with 0.1RC3 it appears to be part of the firmware.

No,

grep -i bad local/dns323/dns323.config

outputs

# BR2_PACKAGE_E2FSPROGS_BADBLOCKS is not set

[michae...@googlemail.com]

Thanks for the reply.
I was aware that modern disks do automatic remapping of bad blocks. Now I am wondering, if this effectively renders the badblocks tool useless, because the disk is "hiding" bad blocks from the badblocks tool. I always looked at using badblocks to run a new disk through its paces and see if it works - not sure if that makes sense then.

Using smartctl I assume the "long test" is what I would want to run then? (sorry, that these are not Alt-F related questions, but you appear to be in the know :) )

For the issue as to using badblocks with Alt-F:
I ended up installing Debian to an external NTFS drive connected to the USB port. Although the Debian install failed (I am guessing because of the NTFS), it *did* copy the e2fsprogs package from Debian. Doing a "mount --bind /dev" to the NTFS disk and using chroot I am now running badblocks on the new disk, which installed to a drive bay. Only question is, it makes sense to use badblocks (see above)...
I would be very grateful for an opinion. :)

[João Cardoso]

On Sunday, February 23, 2014 11:33:54 AM UTC, michae...@googlemail.com wrote:

Thanks for the reply.
I was aware that modern disks do automatic remapping of bad blocks. Now I am wondering, if this effectively renders the badblocks tool useless, because the disk is "hiding" bad blocks from the badblocks tool. I always looked at using badblocks to run a new disk through its paces and see if it works - not sure if that makes sense then.

I don't think that 'badblocks' will return any badblocks, as the surface disk is scanned at the factory to create the initial badblocks mapping.

When the first 2/3TB disks appeared, and the technology was still young, they contained a significant amount of badblocks, hidden from the user, of course.

So, in my opinion, the 'badblocks' program will not return any valuable data, is useless and redundant.

On an old or problematis disk, it might prove to be useful, as it will force the writing of pending badblocks (SMART Current_Pending_Sector), and will, eventually, exhaust the mapping table. That's an educated guess.

 

Using smartctl I assume the "long test" is what I would want to run then?

Yes, but remember to turn disk spindown timeout to zero first, or the test will not complete.

And regular tests should be performed, Services->System->SMART

 

(sorry, that these are not Alt-F related questions, but you appear to be in the know :) )

For the issue as to using badblocks with Alt-F:
I ended up installing Debian to an external NTFS drive connected to the USB port. Although the Debian install failed (I am guessing because of the NTFS),

good guess

 

it *did* copy the e2fsprogs package from Debian. Doing a "mount --bind /dev" to the NTFS disk and using chroot I am now running badblocks on the new disk,

As you know, you have to save the list of badblocks somewhere. Please tell us what the result is.

As you seem to use that often, you could also run it on your old disk, and tell us the result. You might want to get a SMART log first, so you can compare the output before and after running 'badblocks'.

 

which installed to a drive bay. Only question is, it makes sense to use badblocks (see above)...
I would be very grateful for an opinion. :)

You now have it, a fundamented opinion, not one from an expert in the field ;-)

As the mythbusters often say, "we was not expecting these results!, life is full of surprises"

[michae...@googlemail.com]

Hi.
Thanks for the helpful reply.
I ran a long smart test, which ended today - without errors, yay! (took like 18 hours). I am currently considering rerunning a whole badblocks scan. But the first pass of the first time I ran it did not report any errors.

Anyway.... I looked at the smart settings page of Alt-F (services->system->smart) after activating it, and I am a little confused about the options there:
There appear to be 4 locations, where I can set, when/how often to run a scan, which confuses me:
1) Do a short disk test every:
2) Do a long disk test every:
3) Scans the drive every four hours for disk defects
4) Check disks every XX minutes

The first 2 are obviously two different options. But how do the last two options differ from the first two?
I am guessing that option 4) is with respect to the Email notification, telling Alt-F how often to check if an error was reported and if so send a mail?
Not sure what option 3) controls though ...

[João Cardoso]

On Monday, February 24, 2014 9:07:45 AM UTC, michae...@googlemail.com wrote:

Hi.
Thanks for the helpful reply.
I ran a long smart test, which ended today - without errors, yay! (took like 18 hours). I am currently considering rerunning a whole badblocks scan. But the first pass of the first time I ran it did not report any errors.

Anyway.... I looked at the smart settings page of Alt-F (services->system->smart) after activating it, and I am a little confused about the options there:
There appear to be 4 locations, where I can set, when/how often to run a scan, which confuses me:
1) Do a short disk test every:
2) Do a long disk test every:

from the smartd.conf manual page:

        -s REGEXP

              Run  Self-Tests or Offline Immediate Tests, at scheduled times.  A Self- or Offline

              Immediate Test will be run at the end of periodic device polling, if all 12 charac-

              ters of the string T/MM/DD/d/HH match the extended regular expression REGEXP. Here:

3) Scans the drive every four hours for disk defects

from the smartd.conf manual page:

       -o VALUE

              [ATA only] Enables or disables SMART Automatic Offline Testing when  smartd  starts

              up  and  has  no  further effect.  The valid arguments to this Directive are on and

              off.

              The delay between tests is vendor-specific, but is typically four hours.

              Note that SMART Automatic Offline Testing is not part  of  the  ATA  Specification.

              Please  see  the smartctl -o command-line option documentation for further informa-

              tion about this feature.

from the smartctl manual page:

        -o VALUE, --offlineauto=VALUE

              [ATA  only] Enables or disables SMART automatic offline test, which scans the drive

              every four hours for disk defects. This command can be given during  normal  system

              operation.  The valid arguments to this option are on and off.

              (and it continues...)

             Note that the SMART automatic offline test command is listed as "Obsolete" in every

              version of the ATA and ATA/ATAPI Specifications. (...)

              You can tell if automatic offline testing  is  sup-

              ported by seeing if this command enables and disables it, as indicated by the ´Auto

              Offline Data Collection´ part of the  SMART  capabilities  report  (displayed  with

              ´-c´).

4) Check disks every XX minutes

from the smartd manual page:

       If you send a USR1 signal to smartd it will immediately check the status of the disks, and

       then return to polling the disks every 30 minutes. See the ´-i´  option  below  for  addi-

       tional details.

        -i N, --interval=N

              Sets the interval between disk checks to N seconds, where N is a  decimal  integer.

              The  minimum  allowed  value is ten and the maximum is the largest positive integer

              that can be represented on your system (often 2^31-1).  The default  is  1800  sec-

              onds.

 

The first 2 are obviously two different options. But how do the last two options differ from the first two?
I am guessing that option 4) is with respect to the Email notification, telling Alt-F how often to check if an error was reported and if so send a mail?
Not sure what option 3) controls though ...

Sorry for not giving a more clear explanation, but I wrote the code a couple of years ago, and now I can only read the code and see what it does.

If you dig deep into this and find that something is not reasonable or useful, please tell me.

Joao

[stainl...@gmail.com]

Badblocks is still useful as a way to stress a disk or to erase it.  I agree with João that the original use for the program is probably irrelevant due to current disk firmware (I can remember when people talked in hushed tones about the upcoming availability of 2 MB hard disks and wondering what anyone would ever do to fill up such a vast amount of space).

Also, badblocks will return badblocks for a disk which is failing--perhaps due to rough handling during shipping or weakness in the vendor's QA dept.  The value of running badblocks isn't mapping around badblocks, it's finding out if you have a sick disk.

I bought a whole bunch of new disk last winter (12 disks).  I read all around on the 'net about whether one should stress or "burn in" disks.  Not everyone agrees.  Being conservative, I burned in all of my important disks in, and then, because it's boring work, I didn't burn in some other disks (they were just going into USB enclosures).  Guess what?  I lost data on a disk that I hadn't stressed...  "Infant mortality" for new disks is a real thing... prepare for it.

So, my advice to anyone with new disks is to stress your disks for about 24 hrs somehow.  I really like the stressdisk program.  You can connect the disk to your PC and run it there (rather than on the DNS-323), or you could build it and run it on Alt-F.  I haven't done the latter, so I have no idea how involved that would become...  badblocks would be an alternative for stressing the disk.

And life is truly full of surprises...

[João Cardoso]

On Wednesday, February 26, 2014 7:01:02 AM UTC, stainl...@gmail.com wrote:

Badblocks is still useful as a way to stress a disk or to erase it.  I agree with João that the original use for the program is probably irrelevant due to current disk firmware (I can remember when people talked in hushed tones about the upcoming availability of 2 MB hard disks and wondering what anyone would ever do to fill up such a vast amount of space).

Also, badblocks will return badblocks for a disk which is failing--perhaps due to rough handling during shipping or weakness in the vendor's QA dept.  The value of running badblocks isn't mapping around badblocks, it's finding out if you have a sick disk.

I agree with that. And probably 'badblocks' will force SMART to remap bad sectors or force pending (read) sectors to be effectively remapped (which is also good for "senior" disks).

The end result will be that a sick disk will become a dead disk.

For RAID users it's enough to force a RAID check, available under the RAID webUI, which verifies all disk sectors of all disks involved. And a 'resync' or 'rebuild' will have the same effect.

But be warned, a RAID5 resync can lead to unrecoverable disk/RAID failure on huge disks, as the amount of data movement approaches the disk manufacturer specified error rate.

And as disks tend to be bought together at the same time/shop, they belong to the same manufacturing lot and share the same manufacturing characteristics, so when one of them fails the odds that the others will also fail increases.

[Norbert Horváth]

I was also searching for badblocks and that brought me to this thread. With all the respect for your opinion and your work, I have a few thoughts.
You said that badblocks kind of forces the disk to remap the bad area and will exhaust the free blocks, BUT if you do not use badblocks and your drive already has bad blocks in an unused area, you are sitting on a bomb. I think I would use badblocks to scan the entire disk, at least when I buy it (or later when I am getting worried about my data and that happens frequently). If it has bad blocks to remap, then do it. If remapping is impossible because the bad area is larger than the free blocks, then I want to know about it, even if the bad blocks would not be used now.

[João Cardoso]

On Sunday, 5 June 2016 23:03:37 UTC+1, Norbert Horváth wrote:

I was also searching for badblocks and that brought me to this thread. With all the respect for your opinion and your work, I have a few thoughts.
You said that badblocks kind of forces the disk to remap the bad area and will exhaust the free blocks, BUT if you do not use badblocks and your drive already has bad blocks in an unused area, you are sitting on a bomb. I think I would use badblocks to scan the entire disk, at least when I buy it (or later when I am getting worried about my data and that happens frequently). If it has bad blocks to remap, then do it. If remapping is impossible because the bad area is larger than the free blocks, then I want to know about it, even if the bad blocks would not be used now.

OK, use at your will (when available). The commit message:

e2fsprogs-badblocks: new package containing the badblocks utility. In my opinion, it is only useful as a stress-test for new drives, as it might provoke infant-mortality to occur, and its advantage on senior disks is dubious, where SMART should routinely be used instead.

Adding         ipkgfiles/e2fsprogs-badblocks.control

Adding         ipkgfiles/e2fsprogs-badblocks.lst

Transmitting file data ..

Committed revision 3463.

If meanwhile you need to really stress-test new drives, you can use 'srm' from the 'srm' package or other programs from the 'secure-delete' package:

srm DESCRIPTION

       srm  removes  each  specified  file by overwriting, renaming, and truncating it

       before unlinking.  This prevents other people from undeleting or recovering any

       information  about  the  file  from  the  command line.  By default srm uses 35

       passes to overwrite the file's contents.

...

ENTIRE HARD DISKS

       srm can write to block devices on Linux.  You can use srm to securely delete an

       entire  hard disk, however you should only do this for classic magnetic drives.