[Techtalk] finding bad files

[agoats at compuserve.com]

I've been through dying disks as well, and the terrabyte ones are rough.

I found ClamAV is able to find bad files and move them to another 
directory. It looks for them as an opportunity to create memory leaks 
and the like.  No virus or trojan, just not coming up with the right 
sizes et al. It also cleans up the other nonsense as well... ;D

I bought another drive the same size or larger (been through this a few 
times) and moved all files that could be moved to the new drive.

I used ddrescue to recover the other files and in some cases, to recover 
entire drives. For the drives with the recovered filesystem, I ran 
e2fsck to fix the transferred files with better success than trying to 
fix the failing drive. Fewer bad files to fix and less lockups.

I've tried badblocks, but it has limits. IF the drive has soft error 
patches, you may not fix all of the errors because they happened to pass 
when you ran badblocks but fail later (hot day, cold day shift in head 
to sector alignment, phase of the moon, all the bugaboos).

I've done the hdparm disk drive zeroing; not easy, but done a few times. 
Takes forever to do. If you aren't careful, it can brick your drive. I 
bricked one myself!

Use the DOS boot floppy disk versions of the OEM software like Western 
Digital's dlgdiag, or Seagate's 2 disk version. They zero much faster 
and less frustrating. Do the FULL zero, so it writes zeros over the 
entire disk. As it does so, it re-reads the disk and hardware locks out 
the bad areas. Once it can no longer re-allocate sectors, the drive 
starts shrinking in size. I've managed to run a few drives a couple more 
years this way, just have to make sure you don't store truly valuable 
items on those drives. If you have sufficient re-allocatable space left, 
it can fix the SMART drive's info so you don't have the constant nag 
errors. The one I just zeroed wouldn't complete the SMART tests, it 
would hang up until I rebooted the computer. Running dlgdiag and zeroing 
the drive fixed that issue completely.

I've just completed the above on a 1 TB drive. Now to put it back to 
use.... ;D

Alvin

On 04/13/2015 05:05 PM, Miriam English wrote:
> Lots of great suggestions. Thanks.
> Using tar was a terrific idea. Using find and cat to send to /dev/null 
> was the perfect solution.
>
> I should have waited longer before trying what I did. I ran e2fsck to 
> check for errors, then thought, hell, I might as well get it to fix 
> all the errors at the same time and then I could restore the files 
> from my backup because now the damaged files will be different from 
> the backups. But I didn't think long enough about this, because I also 
> have some new files that haven't yet been backed up from that drive. 
> And some will almost certainly be affected by the errors, with no 
> chance of recovering them now. Oops.
>
> The good news is that although many blocks had erroneous data, there 
> were none listed as "bad blocks". I assume that bad blocks are 
> physically faulty regions on the disk.
>
> The drive may be failing, though it isn't terribly old and being an 
> external drive I only connect and switch it on when I actually need to 
> read or write data from/to it. The data may have become affected 
> during one of the very frequent blackouts we have here. Or I wonder if 
> dust on the USB connectors could have damaged the data, though I think 
> that they use checksums to guard against that. Looks like I'll have to 
> save the money to buy another 2 terabyte (or larger) drive. (I wish 
> solid state drives would get cheaper more quickly.)
>
> I used to have an uninterruptible power supply, but it got destroyed a 
> while back when lightning struck while I was shutting everything down 
> because I heard a storm approaching. Since then I've become a big fan 
> of ultra-low-power computing and want to move entirely to solar panels 
> and batteries in the near future. I'm sick of having thousands of 
> dollars in computers and other equipment destroyed by the electrical 
> grid over the years.
>
> There is also the human-unfriendly problem of unmounting a drive 
> before disconnecting it. Everybody knows this must be done, but 
> everybody has lost data because of it. I have, on odd occasions, 
> usually when over-tired, made the mistake of pulling out the wrong USB 
> connector or switching off the wrong external drive.
>
> <rant>I'm certain the design of peripherals could have been made to 
> take account of the flawed nature of humans so that data was buffered 
> in drive electronics, and capacitors supplied just enough power to 
> ensure pointers were updated correctly. But USB, while having terrific 
> advantages over previous kinds of data connectors, was nevertheless a 
> good example of an over-complex, yet flawed, spec designed by 
> committee. The main USB document is more than 600 pages long!</rant>
>
> Cheers,
>
>     - Miriam
>
>