6:57 a.m.
Hi All,
Anyone know how to unsparsify a qcows2 file?
https://libguestfs.org/virt-sparsify.1.html
Many thanks,
-T
8:25 a.m.
On Mon, 12 Dec 2022 04:57:08 -0800
ToddAndMargo via users wrote:
Anyone know how to unsparsify a qcows2 file?
Boot up the virtual machine and write zeroes to a temp file on that disk
till you get out of space errors is one way. Basically stop after the
first step of sparsifying :-).
8:33 a.m.
On 12/12/22 06:25, Tom Horsley wrote:
On Mon, 12 Dec 2022 04:57:08 -0800
ToddAndMargo via users wrote:
> Anyone know how to unsparsify a qcows2 file?
Boot up the virtual machine and write zeroes to a temp file on that disk
till you get out of space errors is one way. Basically stop after the
first step of sparsifying :-).
This it?
dd if=/dev/zero of=/sometempfile
8:43 a.m.
On Mon, 12 Dec 2022 06:33:15 -0800
ToddAndMargo via users wrote:
This it?
dd if=/dev/zero of=/sometempfile
Seems like that would work. Might need to do it as root in case the
kernel doesn't allow an ordinary user to use up all the free space on
a disk. And don't forget "rm /sometempfile" once the dd finishes
with out of space errors.
Of course you could take a completely different tack and use
qemu-img convert to change the qcow2 format file to a raw
disk image and modify the virtual machine to use that new file.
8:49 a.m.
On Mon, Dec 12, 2022 at 9:44 AM Tom Horsley <horsley1953(a)gmail.com> wrote:
On Mon, 12 Dec 2022 06:33:15 -0800
ToddAndMargo via users wrote:
> This it?
>
> dd if=/dev/zero of=/sometempfile
Seems like that would work. Might need to do it as root in case the
kernel doesn't allow an ordinary user to use up all the free space on
a disk. And don't forget "rm /sometempfile" once the dd finishes
with out of space errors.
Of course you could take a completely different tack and use
qemu-img convert to change the qcow2 format file to a raw
disk image and modify the virtual machine to use that new file.
I use logical volumes for the virtual machines I care about
9:32 a.m.
On Mon, Dec 12, 2022 at 9:56 AM ToddAndMargo via users
<users(a)lists.fedoraproject.org> wrote:
On 12/12/22 06:49, Mauricio Tavares wrote:
> I use logical volumes for the virtual machines I care about
What is that?
My desktop's (a VM guest) drive
[user@vmhost ~]# sudo lvs vmhost_vg1
LV VG Attr LSize Pool Origin
Data% Meta% Move Log Cpy%Sync Convert
desktop vmhost_vg1 -wi-a----- 25.00g
[...]
[user@vmhost ~]#
It just has two partitions -- boot and the root -- because I am lazy.
[user@vmhost ~]# sudo fdisk -l /dev/vmhost_vg1/desktop
Disk /dev/vmhost_vg1/desktop: 26.8 GB, 26843545600 bytes
255 heads, 63 sectors/track, 3263 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a805c
Device Boot Start End Blocks Id System
/dev/vmhost_vg1/desktop1 * 1 32 248832 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/vmhost_vg1/desktop2 32 3264 25964544 83 Linux
[user@vmhost ~]#
Inside its KVM config it looks like this
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'
io='native'/>
<source file='/dev/vmhost_vg0/desktop'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x04'
function='0x0'/>
</disk>
> _______________________________________________
> users mailing list -- users(a)lists.fedoraproject.org
> To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
8:55 a.m.
On 12/12/22 06:43, Tom Horsley wrote:
On Mon, 12 Dec 2022 06:33:15 -0800
ToddAndMargo via users wrote:
> This it?
>
> dd if=/dev/zero of=/sometempfile
Seems like that would work. Might need to do it as root in case the
kernel doesn't allow an ordinary user to use up all the free space on
a disk. And don't forget "rm /sometempfile" once the dd finishes
with out of space errors.
Of course you could take a completely different tack and use
qemu-img convert to change the qcow2 format file to a raw
disk image and modify the virtual machine to use that new file.
I wonder if qemu-ing convert will convert from
qcows2 to qcows2? I will see!
Thank you for the help!
9:23 a.m.
On Mon, 12 Dec 2022 06:55:15 -0800
ToddAndMargo via users wrote:
I wonder if qemu-ing convert will convert from
qcows2 to qcows2? I will see!
Yep, that's how I always sparsified my files before the virt-sparsify
tool existed. Write zeroes till you run out of space (thus making all
the free space on the disk blocks of zeroes), shut down the VM, and
qemu-img convert from qcow2 to qcow2. The destination file is much
smaller (usually) but has the same data in it. Rename the file and
now the VM takes less space.
2:08 p.m.
On 12/12/22 06:43, Tom Horsley wrote:
On Mon, 12 Dec 2022 06:33:15 -0800
ToddAndMargo via users wrote:
> This it?
>
> dd if=/dev/zero of=/sometempfile
Seems like that would work. Might need to do it as root in case the
kernel doesn't allow an ordinary user to use up all the free space on
a disk. And don't forget "rm /sometempfile" once the dd finishes
with out of space errors.
Of course you could take a completely different tack and use
qemu-img convert to change the qcow2 format file to a raw
disk image and modify the virtual machine to use that new file.
I qemu-img to a raw and then back to a qcows2.
Dump/resore still did not restore right.
Now I converted to a raw and am about to see
if dump/restore will restore back correct.
If that does not work, I am going to dd /dev/zero
3:54 p.m.
On 12/12/22 12:08, ToddAndMargo via users wrote:
On 12/12/22 06:43, Tom Horsley wrote:
> On Mon, 12 Dec 2022 06:33:15 -0800
> ToddAndMargo via users wrote:
>
>> This it?
>>
>> dd if=/dev/zero of=/sometempfile
>
> Seems like that would work. Might need to do it as root in case the
> kernel doesn't allow an ordinary user to use up all the free space on
> a disk. And don't forget "rm /sometempfile" once the dd finishes
> with out of space errors.
>
> Of course you could take a completely different tack and use
> qemu-img convert to change the qcow2 format file to a raw
> disk image and modify the virtual machine to use that new file.
I qemu-img to a raw and then back to a qcows2.
Dump/resore still did not restore right.
Now I converted to a raw and am about to see
if dump/restore will restore back correct.
If that does not work, I am going to dd /dev/zero
dump/restore restored a "raw" file perfectly.
Interesting. sha256sum came back different for
before and after (.000 is before)
# sha256sum KVM-W11.raw KVM-W11.raw.000
cbc480f889a9e337ab8b41b1e761da5f7f27ad255ceb29d891f60eba8f9903de
KVM-W11.raw
6dbf2d0d893f3fc0b5e529a94aad71770bc971ff567881073c8b8a05d9254fca
KVM-W11.raw.000
4:13 p.m.
You might want to think about setting a size on that file. dd can be
pretty destructive as well as useful. I like to randomize file with it
before truncating the contents. For example, if a file is 4096 bytes, I
would use, 'dd if=/dev/zero of=/file bs=4096 count=1 conv=notrunc. Just
so there is no truncation of the file. dd isn't to be played with
haphazardly. Good luck.
B
On 12/12/2022 4:54 PM, ToddAndMargo via users wrote:
> On 12/12/22 12:08, ToddAndMargo via users wrote:
>> On 12/12/22 06:43, Tom Horsley wrote:
>>> On Mon, 12 Dec 2022 06:33:15 -0800
>>> ToddAndMargo via users wrote:
>>>
>>>> This it?
>>>>
>>>> dd if=/dev/zero of=/sometempfile
>>>
>>> Seems like that would work. Might need to do it as root in case the
>>> kernel doesn't allow an ordinary user to use up all the free space on
>>> a disk. And don't forget "rm /sometempfile" once the dd
finishes
>>> with out of space errors.
>>>
>>> Of course you could take a completely different tack and use
>>> qemu-img convert to change the qcow2 format file to a raw
>>> disk image and modify the virtual machine to use that new file.
>>
>>
>> I qemu-img to a raw and then back to a qcows2.
>> Dump/resore still did not restore right.
>>
>> Now I converted to a raw and am about to see
>> if dump/restore will restore back correct.
>>
>> If that does not work, I am going to dd /dev/zero
>
> dump/restore restored a "raw" file perfectly.
>
> Interesting. sha256sum came back different for
> before and after (.000 is before)
>
> # sha256sum KVM-W11.raw KVM-W11.raw.000
>
> cbc480f889a9e337ab8b41b1e761da5f7f27ad255ceb29d891f60eba8f9903de
> KVM-W11.raw
>
> 6dbf2d0d893f3fc0b5e529a94aad71770bc971ff567881073c8b8a05d9254fca
> KVM-W11.raw.000
>
> _______________________________________________
> users mailing list -- users(a)lists.fedoraproject.org
> To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
4:17 p.m.
On Mon, 2022-12-12 at 13:54 -0800, ToddAndMargo via users wrote:
dump/restore restored a "raw" file perfectly.
Interesting. sha256sum came back different for
before and after (.000 is before)
# sha256sum KVM-W11.raw KVM-W11.raw.000
cbc480f889a9e337ab8b41b1e761da5f7f27ad255ceb29d891f60eba8f9903de
KVM-W11.raw
6dbf2d0d893f3fc0b5e529a94aad71770bc971ff567881073c8b8a05d9254fca
KVM-W11.raw.000
Why do you say it restored perfectly if the checksums don't match?
poc
5:27 p.m.
On 12/12/22 14:17, Patrick O'Callaghan wrote:
On Mon, 2022-12-12 at 13:54 -0800, ToddAndMargo via users wrote:
> dump/restore restored a "raw" file perfectly.
>
> Interesting. sha256sum came back different for
> before and after (.000 is before)
>
> # sha256sum KVM-W11.raw KVM-W11.raw.000
>
> cbc480f889a9e337ab8b41b1e761da5f7f27ad255ceb29d891f60eba8f9903de
> KVM-W11.raw
>
> 6dbf2d0d893f3fc0b5e529a94aad71770bc971ff567881073c8b8a05d9254fca
> KVM-W11.raw.000
Why do you say it restored perfectly if the checksums don't match?
poc
Because I can run the VM without issue. Before,
I could not.
7:20 p.m.
On 12/12/22 13:54, ToddAndMargo via users wrote:
On 12/12/22 12:08, ToddAndMargo via users wrote:
> On 12/12/22 06:43, Tom Horsley wrote:
>> On Mon, 12 Dec 2022 06:33:15 -0800
>> ToddAndMargo via users wrote:
>>
>>> This it?
>>>
>>> dd if=/dev/zero of=/sometempfile
>>
>> Seems like that would work. Might need to do it as root in case the
>> kernel doesn't allow an ordinary user to use up all the free space on
>> a disk. And don't forget "rm /sometempfile" once the dd finishes
>> with out of space errors.
>>
>> Of course you could take a completely different tack and use
>> qemu-img convert to change the qcow2 format file to a raw
>> disk image and modify the virtual machine to use that new file.
>
>
> I qemu-img to a raw and then back to a qcows2.
> Dump/resore still did not restore right.
>
> Now I converted to a raw and am about to see
> if dump/restore will restore back correct.
>
> If that does not work, I am going to dd /dev/zero
dump/restore restored a "raw" file perfectly.
Interesting. sha256sum came back different for
before and after (.000 is before)
# sha256sum KVM-W11.raw KVM-W11.raw.000
cbc480f889a9e337ab8b41b1e761da5f7f27ad255ceb29d891f60eba8f9903de
KVM-W11.raw
6dbf2d0d893f3fc0b5e529a94aad71770bc971ff567881073c8b8a05d9254fca
KVM-W11.raw.000
Dump and restore only interact with used blocks. So if there is old
data in unused blocks in the original disk, the dump won't save those
and the restore won't put them back.
8:16 p.m.
On 12/12/22 17:20, Samuel Sieb wrote:
On 12/12/22 13:54, ToddAndMargo via users wrote:
> On 12/12/22 12:08, ToddAndMargo via users wrote:
>> On 12/12/22 06:43, Tom Horsley wrote:
>>> On Mon, 12 Dec 2022 06:33:15 -0800
>>> ToddAndMargo via users wrote:
>>>
>>>> This it?
>>>>
>>>> dd if=/dev/zero of=/sometempfile
>>>
>>> Seems like that would work. Might need to do it as root in case the
>>> kernel doesn't allow an ordinary user to use up all the free space on
>>> a disk. And don't forget "rm /sometempfile" once the dd
finishes
>>> with out of space errors.
>>>
>>> Of course you could take a completely different tack and use
>>> qemu-img convert to change the qcow2 format file to a raw
>>> disk image and modify the virtual machine to use that new file.
>>
>>
>> I qemu-img to a raw and then back to a qcows2.
>> Dump/resore still did not restore right.
>>
>> Now I converted to a raw and am about to see
>> if dump/restore will restore back correct.
>>
>> If that does not work, I am going to dd /dev/zero
>
> dump/restore restored a "raw" file perfectly.
>
> Interesting. sha256sum came back different for
> before and after (.000 is before)
>
> # sha256sum KVM-W11.raw KVM-W11.raw.000
>
> cbc480f889a9e337ab8b41b1e761da5f7f27ad255ceb29d891f60eba8f9903de
> KVM-W11.raw
>
> 6dbf2d0d893f3fc0b5e529a94aad71770bc971ff567881073c8b8a05d9254fca
> KVM-W11.raw.000
Dump and restore only interact with used blocks. So if there is old
data in unused blocks in the original disk, the dump won't save those
and the restore won't put them back.
That explains why the check sums are different,
but the restored files work. Thank you!
7:22 p.m.
On 12/12/22 12:08, ToddAndMargo via users wrote:
On 12/12/22 06:43, Tom Horsley wrote:
> On Mon, 12 Dec 2022 06:33:15 -0800
> ToddAndMargo via users wrote:
>
>> This it?
>>
>> dd if=/dev/zero of=/sometempfile
>
> Seems like that would work. Might need to do it as root in case the
> kernel doesn't allow an ordinary user to use up all the free space on
> a disk. And don't forget "rm /sometempfile" once the dd finishes
> with out of space errors.
>
> Of course you could take a completely different tack and use
> qemu-img convert to change the qcow2 format file to a raw
> disk image and modify the virtual machine to use that new file.
I qemu-img to a raw and then back to a qcows2.
Dump/resore still did not restore right.
You can't dump a qcow file because the blocks are not in order and there
is extra metadata. You can only dump from something that looks like a
raw partition. You might be able to use something like nbd to create a
device from a qcow that looks raw.
8:15 p.m.
On 12/12/22 17:22, Samuel Sieb wrote:
On 12/12/22 12:08, ToddAndMargo via users wrote:
> On 12/12/22 06:43, Tom Horsley wrote:
>> On Mon, 12 Dec 2022 06:33:15 -0800
>> ToddAndMargo via users wrote:
>>
>>> This it?
>>>
>>> dd if=/dev/zero of=/sometempfile
>>
>> Seems like that would work. Might need to do it as root in case the
>> kernel doesn't allow an ordinary user to use up all the free space on
>> a disk. And don't forget "rm /sometempfile" once the dd finishes
>> with out of space errors.
>>
>> Of course you could take a completely different tack and use
>> qemu-img convert to change the qcow2 format file to a raw
>> disk image and modify the virtual machine to use that new file.
>
>
> I qemu-img to a raw and then back to a qcows2.
> Dump/resore still did not restore right.
You can't dump a qcow file because the blocks are not in order and there
is extra metadata. You can only dump from something that looks like a
raw partition. You might be able to use something like nbd to create a
device from a qcow that looks raw.
I am switching to ".raw" files. They seems to come
back working
506
days inactive
507
days old
17 comments
6 participants
participants (6)
-
Bill Cunningham -
Mauricio Tavares -
Patrick O'Callaghan -
Samuel Sieb -
ToddAndMargo -
Tom Horsley