How to help users detect bit rot on Windows?

[Michael Kjörling]

If this is the wrong place to ask, then please point me at a better
forum.

On Windows, I need to find a way to allow end users, as easily and
ideally automated as possible, to determine:

1. If any files have been deleted
2. If any files have been altered or otherwise suffered from bit rot

while at least in some fashion _semi-automatically_ capturing and
monitoring any newly added files, and notifying the user if during a
check run anything on disk doesn't match previously stored fixity
data. It's okay if they need to start an application and initiate a
scan, but it's not okay if they have to point it manually at each file
that has been added under an existing directory structure.

I don't know the exact version of Windows that'll be involved, but I
suspect Windows 8.

The purpose here is to run this tool on a laptop not under my direct
control, in order to capture deterioration in data otherwise not
frequently accessed but kept for potentially several years or more,
before running a backup to safekeep newly added data. I want this to
happen in as much an automated way as possible to ensure that files
that are later corrupted (or deleted) are not backed up in their
corrupted state or deleted from backups, even when backups are
rotated, without some clear indication to the user that there is
something amiss with these specific files that they need to look into.

It would be great if I can just tell the user "put stuff you are
currently working on in _this_ folder, and then once you're done with
it and there are going to be no more changes to those files, move it
to _that_ folder instead and click _this_ button" and "to run a
backup, first start _this_ application and click _that_ button to
check everything, and if and only if it comes back saying everything
is okay, then start _that_ application and click _this_ button to
actually run the backup".

This _has_ to be doable, repeatedly, by someone who is non-technical
(but understands why it's important to do). The requirement for doing
it on Windows (and on a laptop, so without ECC RAM) rules out my
favorite file system ZFS, which could likely have made this fairly
easy using snapshots, regular scrubbing and a shell script or two that
could have been prepared ahead of time.

I suppose in a way, I'm basically looking for a variant of the virus
scanner technique sometimes used back in the MS-DOS days, with a
resident component calculating a checksum for any executable file
written (or on first access) and comparing the checksum on read
access, and a corresponding foreground scanner one could also use;
only geared toward fixity checking of non-executables instead. As long
as everything was unchanged it stayed out of the way (costing only
performance or some time) but if something had changed, it alerted the
user to the change.

Any ideas?

--
Michael Kjörling • https://michael.kjorling.se • mic...@kjorling.se
OpenPGP B501AC6429EF4514 https://michael.kjorling.se/public-keys/pgp
“People who think they know everything really annoy
those of us who know we don’t.” (Bjarne Stroustrup)

[Sarah Barsness]

Michael,

I have previously done some research on different fixity programs...you can see all the relevant stuff on the Digital Preservation Q&A.  You might find something useful in there.  

Based on the use case you described, though, I think Fixity by AVPreserve might be a good fit for you.  It's meant to work in a pre-scheduled, automated way, deals well with new/additional files, and has some nice reporting features that you might be able to use to create a pretty slick workflow (e.g. you can have it email you the reports about new items, which you could then use to initiate the backup piece of the equation).