Opinions on new Fedora Core 2 install with LVM 2 and snapshots?

[Bryan J. Smith]

On Tue, 2004-07-27 at 09:45, Bill Rugolsky Jr. wrote:
> I've used earlier 3ware products, and they are quite good.
> Now that there is a CLI configuration tool, even better.

Yep, now it's scriptable.

> (Though the fact that it is closed source is rather silly.)

Well, they probably have their reasons.

> There are some corruption issues with 3ware 9500's on Opteron boards.
> The IOMMU flush workaround is ugly.

Yeah, I noticed that.  But I'm running on P3 and Athlon MP.

> I'll wait to see where and how it gets resolved.  For now, I have
> 4 (somewhat slower) onboard SATA ports.

Well, I don't know if it's a matter of "slower."  It's just passing
around data on your interconnect redundantly (at least in the case
of RAID-1/0+1/3/4/5/6).

> Yes.  H. Peter Anvin implemented it a few months ago, and it is in the
> mainline 2.6 kernel.

Very cool.  MD then?

> XFS is in the mainline kernel, and available in the Fedora 2.6 kernels.
> xfsprogs is in FC2.

Yep.  I've followed XFS' entry into the kernel.  It was production
quality back in the 2.4 days, but it required a lot of patching to the
2.4 kernel.

> I don't like the idea of my data stored in a proprietary format.
> When the controller or machine dies (perhaps several years from now) or I
> have a simultaneous two-drive partial failure (which happened three years
> ago), I want to be able to attach the drives to standard ports and use
> "dd" and Device Mapper or MD to recover my data.

Well, I've always been able to move 3Ware volumes upwards to newer
controllers without issue over the past 4 years.

But I can see your point on the capability to read the drives "raw."
But you can always do that by putting them on regular ATA controllers.
You just don't know the format.

> It seems that many of the "ext3 filesystem is corrupt" reports on
> ext3-users and elsewhere have been IDE drives that don't flush writes on
> powerdown,

Never had that issue.  The 3Ware cards seem to handle flushing with
all the Maxtor, Seagate and WD ATA drives I've thrown at it without
issue.

> bad RAM, bad cables,

Always an issue there.

> overheating chipsets,

Yep, NIC ASICs seem to be my problem there.

> and buggy DMA engines. (Witness the 3ware/Opteron problem.)

Not sure it's a buggy DMA engine, but more of a maturity thing?
What have you heard?

> Add to that occasional driver, VM, and filesystem bugs (typically
> race conditions), as well as operator error when doing things like
> fscking one partition of a RAID1 pair (i.e., sda1 instead of md1), or
> using GRUB "savedefault".

Which is my #1 reason why I like to keep the RAID in hardware.  I
agree that it would be nice for 3Ware to support Linux approaches
and integration with VM, but I still like keeping it "abstract"
below Linux at the same time.

With that said, are there race conditions with LVM2, snapshots and
Ext3 in order-writes mode?  Assuming you are _not_ using _anything_
else -- especially not MD.

> Journaling prevents none of this.

Exactly.

> You've been lucky;

I haven't run LVM yet.  I've just done plain Ext3 atop of 3Ware
cards.  So I'm worried about such race conditions.

> better to be vigilant.

Which makes me wonder if I should hold off on LVM2.  I want to
go LVM2 and eventually enable snapshots.  But snapshots off-the-bat
are not a requirement.

> We compare md5sums longitudinally (inter-day) and latitudinally
> (across different hosts) every day via cron.  Snapshots allow the
> meta-data to be sanity checked too.

And what have been your results?

> Probably not for a few months.

I'm documenting what you've told me thus far.
Whatever else you can offer, I'd greatly appreciate it.

> Something like:
> modprobe dm-snapshot
> lvm lvcreate -L 1G -s -n lv00snap vg00/lv00

Er, doh, I meant lvcreate, yes.

Is there a good example of cronjobs for "last 2 days, last 2 weeks"?

-- 
     Linux Enthusiasts call me anti-Linux.
   Windows Enthusisats call me anti-Microsoft.
 They both must be correct because I have over a
decade of experience with both in mission critical
environments, resulting in a bigotry dedicated to
 mitigating risk and focusing on technologies ...
           not products or vendors
--------------------------------------------------
Bryan J. Smith, E.I.         b.j.smith at ieee.org