Advanced Format drives + RAID5 + LVM + EXT4 = migraine

[Jake Peavy]

Hey all,

I'm going a little off my rocker here trying to understand how to best
configure a 3-drive RAID5 array using the new Western Digital 4k sector
drives.

I've got 3xWD15EARS (1.5T 4k sector advanced format drives).

If I was using the drive as-is, my understanding is I should create a
partition starting at sector 64.  This would solve all my problems.

But I want to put them into a RAID5 array, so currently I didn't partition
at all and just passed mdadm the whole drive.  Not sure if this is the best
approach.  I used 512kB as chunk size.

Then I want to create LVM on the array.  I didn't pass any special parms to
{pv,vg,lv}create (should I?)

Lastly, I want to create an EXT4 filesystem on the array.  I used the stride
calculator at http://busybox.net/~aldot/mkfs_stride.html<http://busybox.net/%7Ealdot/mkfs_stride.html>
with
raid level = 5,  devices = 3, chunk size = 512 and filesystem blocks = 4 (?)
which suggested I use mkfs.ext4 -b 4096 -E stride=128,stripe-width=256
(which I did).

After doing all this, I get writes to my filesystem of only 38MB/s.  Reads
of 119MB/s.

[root at morbo opt]# hdparm  -Tt /dev/mapper/array_vg-lv_opt

/dev/mapper/array_vg-lv_opt:
 Timing cached reads:   1574 MB in  2.00 seconds = 786.85 MB/sec
 Timing buffered disk reads:  332 MB in  3.00 seconds = 110.57 MB/sec
[root at morbo opt]# time (dd if=/dev/zero of=zerofile bs=1M count=8192; sync)
8192+0 records in
8192+0 records out
8589934592 bytes (8.6 GB) copied, 226.646 s, 37.9 MB/s

real    3m51.062s
user    0m0.072s
sys    0m39.794s
[root at morbo opt]# dd if=zerofile of=/dev/null bs=1M
8192+0 records in
8192+0 records out
8589934592 bytes (8.6 GB) copied, 72.4094 s, 119 MB/s


I have a feeling my array should perform much better than this and it's
probably because I haven't taken the 4k sector size into account and I'm
bleeding performance.

Can anyone comment?  If I need to take the 4k sector size into account,
where should I do so?  With partitions?  In pvcreate?  Some other place?  Or
multiple places?

TIA,

-- 
-jp

If you ever go temporarily insane, don't shoot somebody, like a lot of
people do. Instead, try to get some weeding done, because you'd really be
surprised.

deepthoughtsbyjackhandey.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.fedoraproject.org/pipermail/users/attachments/20100829/98e2dea4/attachment.html