2:46 p.m.
I have been experimenting with putting /var/lib/libvirt/images on a
btrfs subvolume. What I found was ... interesting.
Just doing nothing else, a disk image file can en up with something like
50,000 extents or more. Now, you can go do a btrfs fi defrag and it
will reduce but when you use the system, it will increase again.
OK, you cannot turn off "cow" for a subvolume but you can turn it off
for a new file (or more important) for an new file in a directory:
chattr +C /var/lib/libvirt/images and any new disk image file will have
no-cow.
OK, install a system with a new disk file. The result: a file with
47,000 extents. The reason, I used virt-manager and the disk image file
allocated was a sparse file. Now in some respects sparse files are good
because they do not use space unless they need it. But, with BTRFS, the
result is large scale fragmentation. And, btrfs fi defrag does not
work on this file /directory.
OK, another try. Preallocate a file using dd if=/dev/zero
of=/var/lib/libvirt/images/xxx.img bs=1024 count=12587000 and the result
is a 12GB (more or less) virtual disk. Install a system using this as
the virtual disk. extents=9. install a second system subvol=root2 (it
is a virtual btrfs system). extents=9. And a third one. extents=9.
BTW, these installs were about the fastest I have ever seen!
So, is there a way to do this "sparse=false" disk allocation with
virt-manager, or do I need to do an RFE?
For a system where the disk images are going to be on BTRFS, a fully
allocated disk has definite benefits.
Gene
5:36 p.m.
On 11/14/2013 03:46 PM, Gene Czarcinski wrote:
I have been experimenting with putting /var/lib/libvirt/images on a
btrfs
subvolume. What I found was ... interesting.
Just doing nothing else, a disk image file can en up with something like
50,000 extents or more. Now, you can go do a btrfs fi defrag and it will
reduce but when you use the system, it will increase again.
OK, you cannot turn off "cow" for a subvolume but you can turn it off for a
new file (or more important) for an new file in a directory: chattr +C
/var/lib/libvirt/images and any new disk image file will have no-cow.
OK, install a system with a new disk file. The result: a file with 47,000
extents. The reason, I used virt-manager and the disk image file allocated
was a sparse file. Now in some respects sparse files are good because they do
not use space unless they need it. But, with BTRFS, the result is large scale
fragmentation. And, btrfs fi defrag does not
work on this file /directory.
OK, another try. Preallocate a file using dd if=/dev/zero
of=/var/lib/libvirt/images/xxx.img bs=1024 count=12587000 and the result is a
12GB (more or less) virtual disk. Install a system using this as the virtual
disk. extents=9. install a second system subvol=root2 (it is a virtual btrfs
system). extents=9. And a third one. extents=9. BTW, these installs were
about the fastest I have ever seen!
So, is there a way to do this "sparse=false" disk allocation with
virt-manager, or do I need to do an RFE?
For a system where the disk images are going to be on BTRFS, a fully allocated
disk has definite benefits.
virt-manager already has this option: in the 'new vm' wizard, there's a
checkbox for 'allocate entire disk now' or similar. it is checked by default.
But we just ask libvirt to handle this for us, and it is likely using
fallocate and not writing tons of zeros like dd will do.
Maybe libvirt should skip that fallocate step on btrfs. I've moved your bug
[1] to libvirt for further investigation.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1031303
- Cole
9:32 a.m.
On 11/16/2013 06:36 PM, Cole Robinson wrote:
On 11/14/2013 03:46 PM, Gene Czarcinski wrote:
> I have been experimenting with putting /var/lib/libvirt/images on a btrfs
> subvolume. What I found was ... interesting.
>
> Just doing nothing else, a disk image file can en up with something like
> 50,000 extents or more. Now, you can go do a btrfs fi defrag and it will
> reduce but when you use the system, it will increase again.
>
> OK, you cannot turn off "cow" for a subvolume but you can turn it off for
a
> new file (or more important) for an new file in a directory: chattr +C
> /var/lib/libvirt/images and any new disk image file will have no-cow.
>
> OK, install a system with a new disk file. The result: a file with 47,000
> extents. The reason, I used virt-manager and the disk image file allocated
> was a sparse file. Now in some respects sparse files are good because they do
> not use space unless they need it. But, with BTRFS, the result is large scale
> fragmentation. And, btrfs fi defrag does not
> work on this file /directory.
>
> OK, another try. Preallocate a file using dd if=/dev/zero
> of=/var/lib/libvirt/images/xxx.img bs=1024 count=12587000 and the result is a
> 12GB (more or less) virtual disk. Install a system using this as the virtual
> disk. extents=9. install a second system subvol=root2 (it is a virtual btrfs
> system). extents=9. And a third one. extents=9. BTW, these installs were
> about the fastest I have ever seen!
>
> So, is there a way to do this "sparse=false" disk allocation with
> virt-manager, or do I need to do an RFE?
>
> For a system where the disk images are going to be on BTRFS, a fully allocated
> disk has definite benefits.
>
virt-manager already has this option: in the 'new vm' wizard, there's a
checkbox for 'allocate entire disk now' or similar. it is checked by default.
But we just ask libvirt to handle this for us, and it is likely using
fallocate and not writing tons of zeros like dd will do.
Maybe libvirt should skip that fallocate step on btrfs. I've moved your bug
[1] to libvirt for further investigation.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1031303
I have been doing a lot more testing including running on Fedora 19 (my
all BTRFS system has both Fedora-20-Beta and Fedora 19 installed). With
Fedora-19 and virt-manager-0.10.0-4.fc19.noarch, the VM creation wizard
defaults the storage format defaults to raw. The result is the the disk
image file as a very low number of extents (59 under Fedora 19 but after
rebooting into Fedora-20-Beta, it was 12). Note: I am using "filefrag"
to determine the number of extents.
I also ran a test which left a very small qcow2 disk and added an 8GB
raw disk where Fedora 20 was installed.
I assume that the disk image formatting is being handed off to libvirt
for all disk images. In that case, libvirt is working properly. The
difference is qcow2 which, IIUC, is a sparse only format and raw which
can be a nonsparse file.
Nonsparse raw format files all have a very low number of extents: below
20 and most often below 10.
So, what is my problem? My "current" problem is that I cannot change
the storage format specified on the disk created by the vm creation
wizard. The current bugzilla report has become confused (my doing) and
I will be closing it and opening a new report with the problem as I now
understand it. To keep things simple, I will also be adding a new
bugzilla RFE report to ask for the ability to specify just what the
default storage format is.
Gene
9:51 a.m.
On 11/17/2013 10:32 AM, Gene Czarcinski wrote:
On 11/16/2013 06:36 PM, Cole Robinson wrote:
> On 11/14/2013 03:46 PM, Gene Czarcinski wrote:
>> I have been experimenting with putting /var/lib/libvirt/images on a
>> btrfs
>> subvolume. What I found was ... interesting.
>>
>> Just doing nothing else, a disk image file can en up with something
>> like
>> 50,000 extents or more. Now, you can go do a btrfs fi defrag and it
>> will
>> reduce but when you use the system, it will increase again.
>>
>> OK, you cannot turn off "cow" for a subvolume but you can turn it
>> off for a
>> new file (or more important) for an new file in a directory: chattr +C
>> /var/lib/libvirt/images and any new disk image file will have no-cow.
>>
>> OK, install a system with a new disk file. The result: a file with
>> 47,000
>> extents. The reason, I used virt-manager and the disk image file
>> allocated
>> was a sparse file. Now in some respects sparse files are good
>> because they do
>> not use space unless they need it. But, with BTRFS, the result is
>> large scale
>> fragmentation. And, btrfs fi defrag does not
>> work on this file /directory.
>>
>> OK, another try. Preallocate a file using dd if=/dev/zero
>> of=/var/lib/libvirt/images/xxx.img bs=1024 count=12587000 and the
>> result is a
>> 12GB (more or less) virtual disk. Install a system using this as
>> the virtual
>> disk. extents=9. install a second system subvol=root2 (it is a
>> virtual btrfs
>> system). extents=9. And a third one. extents=9. BTW, these
>> installs were
>> about the fastest I have ever seen!
>>
>> So, is there a way to do this "sparse=false" disk allocation with
>> virt-manager, or do I need to do an RFE?
>>
>> For a system where the disk images are going to be on BTRFS, a fully
>> allocated
>> disk has definite benefits.
>>
> virt-manager already has this option: in the 'new vm' wizard, there's a
> checkbox for 'allocate entire disk now' or similar. it is checked by
> default.
> But we just ask libvirt to handle this for us, and it is likely using
> fallocate and not writing tons of zeros like dd will do.
>
> Maybe libvirt should skip that fallocate step on btrfs. I've moved
> your bug
> [1] to libvirt for further investigation.
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1031303
I have been doing a lot more testing including running on Fedora 19
(my all BTRFS system has both Fedora-20-Beta and Fedora 19
installed). With Fedora-19 and virt-manager-0.10.0-4.fc19.noarch, the
VM creation wizard defaults the storage format defaults to raw. The
result is the the disk image file as a very low number of extents (59
under Fedora 19 but after rebooting into Fedora-20-Beta, it was 12).
Note: I am using "filefrag" to determine the number of extents.
I also ran a test which left a very small qcow2 disk and added an 8GB
raw disk where Fedora 20 was installed.
I assume that the disk image formatting is being handed off to libvirt
for all disk images. In that case, libvirt is working properly. The
difference is qcow2 which, IIUC, is a sparse only format and raw which
can be a nonsparse file.
Nonsparse raw format files all have a very low number of extents:
below 20 and most often below 10.
So, what is my problem? My "current" problem is that I cannot change
the storage format specified on the disk created by the vm creation
wizard. The current bugzilla report has become confused (my doing)
and I will be closing it and opening a new report with the problem as
I now understand it. To keep things simple, I will also be adding a
new bugzilla RFE report to ask for the ability to specify just what
the default storage format is.
Oh boy, this is one of those "never mind"
... "not a bug"
I found the preference where I can set the "default" for the "storage
format". Setting it to "raw", and things work as I want them to.
Gene
6:28 a.m.
On Thu, Nov 14, 2013 at 03:46:33PM -0500, Gene Czarcinski wrote:
OK, another try. Preallocate a file using dd if=/dev/zero
of=/var/lib/libvirt/images/xxx.img bs=1024 count=12587000 and the
result is a 12GB (more or less) virtual disk.
I would recommend using the 'fallocate' command, and not 'dd' for
this. fallocate uses a system call which tells the filesystem your
intentions. It's also much faster.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
3804
days inactive
3816
days old
4 comments
3 participants
participants (3)
-
Cole Robinson -
Gene Czarcinski -
Richard W.M. Jones