Odd rebalancing behavior

[Michel Alexandre Salim]

On Tue, Apr 5, 2011 at 9:20 AM, Arne Jansen <sensille at gmx.net> wrote:
> On 04.04.2011 12:14, Michel Alexandre Salim wrote:
>> I have an external 4-disk enclosure, connected through USB 2.0 (my
>> laptop does not have a USB 3.0 connector, and the eSATA connector
>> somehow does not work); it initially had a 2-disk btrfs soft-RAID1 file
>> system (both data and metadata are RAID1).
>>
>> I recently added two more disks and did a rebalance. To my surprise it
>> went past the point where all four disks have the same amount of disk
>> usage, and went all the way to the original disks being empty, and the
>> new disks having all the data!
>>
>> Label: 'media.store'  uuid: 4cfd3551-aa85-4399-b872-9238ddb14c97
>>       Total devices 4 FS bytes used 1.22TB
>>       devid    3 size 1.82TB used 1.24TB path /dev/sdb
>>       devid    4 size 1.82TB used 1.24TB path /dev/sdc
>>       devid    2 size 1.82TB used 8.00MB path /dev/sde
>>       devid    1 size 1.82TB used 12.00MB path /dev/sdd
>>
>> Is this to be expected? Would another rebalance fix it, or should I
>> force-stop it by shutting down when the disk usage is roughly balanced?
>
> Currently this is expected behavior, as the current allocator doesn't
> distribute the chunks evenly. As soon as the round-robin allocator patch
> is merged, rebalance will lead to a balanced distribution. It has been
> posted to the list with the subject 'quasi-round-robin for chunk
> allocation'.
> If you test it, some feedback would be nice.
>
I tested it, by applying Arne's patch directly on the Fedora kernel's
source, and then rebuilding the btrfs module using dkms as per the
instructions in the Btrfs wiki.

It just reached the point where all disks have roughly the same amount
of space, and I'm happy to report that, while rebalancing is still not
finished, disk usage is now evenly spread across all four disks.

Here are three runs, spaced with several minutes in between, of btrfs fi show.

It'd be great if this patch can be applied soon -- or people who
expand their arrays when disk utilization increases beyond a threshold
would be surprised when they try rebalancing.

Thanks,

-- 
Michel

[michel at hypatia Radio]$ sudo btrfs fi show media.store
[sudo] password for michel:
Label: 'media.store'  uuid: 4cfd3551-aa85-4399-b872-9238ddb14c97
        Total devices 4 FS bytes used 1.22TB
        devid    3 size 1.82TB used 638.00GB path /dev/sdc
        devid    4 size 1.82TB used 638.00GB path /dev/sdd
        devid    2 size 1.82TB used 637.26GB path /dev/sdf
        devid    1 size 1.82TB used 637.26GB path /dev/sde

Btrfs Btrfs v0.19
[michel at hypatia Radio]$ sudo btrfs fi show media.store
Label: 'media.store'  uuid: 4cfd3551-aa85-4399-b872-9238ddb14c97
        Total devices 4 FS bytes used 1.22TB
        devid    3 size 1.82TB used 637.00GB path /dev/sdc
        devid    4 size 1.82TB used 637.00GB path /dev/sdd
        devid    2 size 1.82TB used 637.26GB path /dev/sdf
        devid    1 size 1.82TB used 637.26GB path /dev/sde

Btrfs Btrfs v0.19
[michel at hypatia Radio]$ sudo btrfs fi show media.store
Label: 'media.store'  uuid: 4cfd3551-aa85-4399-b872-9238ddb14c97
        Total devices 4 FS bytes used 1.22TB
        devid    3 size 1.82TB used 638.00GB path /dev/sdc
        devid    4 size 1.82TB used 638.00GB path /dev/sdd
        devid    2 size 1.82TB used 637.26GB path /dev/sdf
        devid    1 size 1.82TB used 637.26GB path /dev/sde

Btrfs Btrfs v0.19
[michel at hypatia Radio]$ sudo btrfs fi show media.store
Label: 'media.store'  uuid: 4cfd3551-aa85-4399-b872-9238ddb14c97
        Total devices 4 FS bytes used 1.22TB
        devid    3 size 1.82TB used 638.00GB path /dev/sdc
        devid    4 size 1.82TB used 638.00GB path /dev/sdd
        devid    2 size 1.82TB used 637.26GB path /dev/sdf
        devid    1 size 1.82TB used 637.26GB path /dev/sde

Btrfs Btrfs v0.19

> -Arne
>
>>
>> This is on Fedora 15 pre-release, x86_64, fully updated, kernel
>> 2.6.38.2-9 and btrfs-progs 0.19-13
>>
>> Thanks,
>>
>
>



-- 
Michel Alexandre Salim
Fedora Project Contributor: http://fedoraproject.org/

Email:  salimma at fedoraproject.org  | GPG key ID: 78884778
Jabber: hircus at jabber.ccc.de       | IRC: hircus at irc.freenode.net

()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments