On Mon, Nov 2, 2020 at 3:50 AM Patrick O'Callaghan
<pocallaghan(a)gmail.com> wrote:
On Sun, 2020-11-01 at 19:00 -0700, Chris Murphy wrote:
> On Sun, Nov 1, 2020 at 6:51 AM John Mellor <john.mellor(a)gmail.com> wrote:
> > On 2020-10-31 10:46 p.m., Tim via users wrote:
> > > On Sat, 2020-10-31 at 16:11 +0000, lancelassetter(a)gmail.com wrote:
> > > > Will NFS tell you data has been corrupted during the transfer and
> > > > write process?
> > > Does any filing system? In general, writes to storage are assumed to
> > > have worked unless something throws up an error message. Your hard
> > > drive could be silently corrupting data as it writes to the drive due
> > > to various reasons (defects in its media, bugs in its firmware,
> > > glitches from bad power supplies). You'd never know unless your
> > > filing system did a sanity check after writing. Some specialised ones
> > > might do that, but the average ones don't
> > >
> > You are correct for some very popular filesystems. EXT2/3/4, XFS, NTFS
> > etc. will not detect this situation. However, newer filesystems (<10
> > years old) do handle silent data glitches, bad RAM and cosmic ray hits
> > correctly.
> >
> > BTRFS has been the default filesystem on SUSE Linux for years, and is
> > now the default filesystem on Fedora-33. ZFS is an optional filesystem
> > on Ubuntu-20 and all the Berkeley-derived Unixen like FreeBSD, and
> > standard on Oracle Linux and Solaris. BTRFS and ZFS are both COW
> > filesystems using checksumming of both data and metadata. When you push
> > something to the disk(s) with some kind of RAM error or power glitch,
> > the first write will be stored with the error, and then the checksummed
> > metadata is simply redirected to reference the new stuff. This will
> > detect the checksum errors on the data on ZFS with the reread to verify
> > the checksum, but I believe that BTRFS will return a successful write
> > without one of the RAID configurations set on the pool. If you are
> > running one of the RAID configurations, the checksum error will be
> > detected before the write completes. To guard against on-disk
> > corruption (bit rot), both ZFS and BTRFS will also correct it on the
> > next read of that data if you are running the filesystem in one of the
> > RAID-z configurations (multiple copies stored), or upon running a
> > filesystem integrity check.
>
> Short story:
[...]
Thanks. That more or less matches what I thought. So BTRFS does not do
read-after-write verification and ZFS does, correct? Just trying to
clarify.
Btrfs doesn't do that, and I'm not aware of such an option in ZFS.
https://github.com/openzfs/zfs/issues/2526
https://www.illumos.org/issues/2008
Btrfs and ZFS both do checksum verification of every read, and also
have a scrub option.
--
Chris Murphy