Hi All,
Fedora 41 borgbackup-1.4.1-1.fc41.x86_64
In my testing an hot to write up I made myself of borg backup, I found something confusing.
I backed up three file and restored them (somewhere else). The sha256sum matched the originals but the du's did not. What am I missing?
Many thanks, -T
Originals: $ dd count=0 bs=1M seek=100 of=sparseFile
$ du --bytes KVM-W10.raw KVM-W11.raw sparseFile 64_424_509_440 KVM-W10.raw 133_143_986_176 KVM-W11.raw 104_857_600 sparseFile
$ du --block-size=1 KVM-W10.raw KVM-W11.raw sparseFile 48_891_629_568 KVM-W10.raw 51_850_604_544 KVM-W11.raw 0 sparseFile
$ sha256sum KVM-W10.raw KVM-W11.raw sparseFile 67965ba41959f8e43df5710bfb9f8e95f741b89ead48c94a2206d786e80d23cc /home/kvm/KVM-W10.raw 0b25189cdd77f9b9bbe663e06b73916ff9cce27f2c430358910977c062dfa8ba KVM-W11.raw 20492a4d0d84f8beb1767f6616229f85d44c2827b64bdbfb260ee12fa1109e0e sparseFile
Recovered files: # du --block-size=1 /home/temp/borg/* 64_424_521_728 /home/temp/borg/KVM-W10.raw 133_144_117_248 /home/temp/borg/KVM-W11.raw 104_857_600 /home/temp/borg/sparseFile
# sha256sum /home/temp/borg/* 67965ba41959f8e43df5710bfb9f8e95f741b89ead48c94a2206d786e80d23cc /home/temp/borg/KVM-W10.raw 0b25189cdd77f9b9bbe663e06b73916ff9cce27f2c430358910977c062dfa8ba /home/temp/borg/KVM-W11.raw 20492a4d0d84f8beb1767f6616229f85d44c2827b64bdbfb260ee12fa1109e0e /home/temp/borg/sparseFile
On 5/8/25 4:54 PM, ToddAndMargo via users wrote:
Fedora 41 borgbackup-1.4.1-1.fc41.x86_64
In my testing an hot to write up I made myself of borg backup, I found something confusing.
I backed up three file and restored them (somewhere else). The sha256sum matched the originals but the du's did not. What am I missing?
I'm guessing that the backup program doesn't handle sparse files.
The checksum will match because the empty space is zeroed either way.
On 5/8/25 4:57 PM, Samuel Sieb wrote:
On 5/8/25 4:54 PM, ToddAndMargo via users wrote:
Fedora 41 borgbackup-1.4.1-1.fc41.x86_64
In my testing an hot to write up I made myself of borg backup, I found something confusing.
I backed up three file and restored them (somewhere else). The sha256sum matched the originals but the du's did not. What am I missing?
I'm guessing that the backup program doesn't handle sparse files.
The checksum will match because the empty space is zeroed either way.
Thank you!
My read is that if the check sum matches, the file is perfect. But, I wanted to double check in case I was missing something.
On 5/8/25 5:22 PM, ToddAndMargo via users wrote:
On 5/8/25 4:57 PM, Samuel Sieb wrote:
On 5/8/25 4:54 PM, ToddAndMargo via users wrote:
Fedora 41 borgbackup-1.4.1-1.fc41.x86_64
In my testing an hot to write up I made myself of borg backup, I found something confusing.
I backed up three file and restored them (somewhere else). The sha256sum matched the originals but the du's did not. What am I missing?
I'm guessing that the backup program doesn't handle sparse files.
The checksum will match because the empty space is zeroed either way.
Thank you!
My read is that if the check sum matches, the file is perfect. But, I wanted to double check in case I was missing something.
The file is equivalent. It just takes up more disk space. :-)
On 5/8/25 5:43 PM, Samuel Sieb wrote:
On 5/8/25 5:22 PM, ToddAndMargo via users wrote:
On 5/8/25 4:57 PM, Samuel Sieb wrote:
On 5/8/25 4:54 PM, ToddAndMargo via users wrote:
Fedora 41 borgbackup-1.4.1-1.fc41.x86_64
In my testing an hot to write up I made myself of borg backup, I found something confusing.
I backed up three file and restored them (somewhere else). The sha256sum matched the originals but the du's did not. What am I missing?
I'm guessing that the backup program doesn't handle sparse files.
The checksum will match because the empty space is zeroed either way.
Thank you!
My read is that if the check sum matches, the file is perfect. But, I wanted to double check in case I was missing something.
The file is equivalent. It just takes up more disk space. :-)
As a test, now that dump/restore is working again, I backed up a sparse file and restored:
Sparse file backups and restored with dump/restore: # du --bytes * 104857600 sparseFile 104857600 sparseFile.orig
# du --block-size=1 * 0 sparseFile 0 sparseFile.orig
# sha256sum *
20492a4d0d84f8beb1767f6616229f85d44c2827b64bdbfb260ee12fa1109e0e sparseFile
20492a4d0d84f8beb1767f6616229f85d44c2827b64bdbfb260ee12fa1109e0e sparseFile.orig
Borg is incapable of the same result. And I am backing up a lot of qamu-kvm virtual drives. They are sparse files and large. If borg can not deal with sparse files, then it is not a good candidate for replacing dump/restore. Bord will give back equivalent files that are much larger.
On Thu, 2025-05-08 at 16:57 -0700, Samuel Sieb wrote:
On 5/8/25 4:54 PM, ToddAndMargo via users wrote:
Fedora 41 borgbackup-1.4.1-1.fc41.x86_64
In my testing an hot to write up I made myself of borg backup, I found something confusing.
I backed up three file and restored them (somewhere else). The sha256sum matched the originals but the du's did not. What am I missing?
I'm guessing that the backup program doesn't handle sparse files.
The checksum will match because the empty space is zeroed either way.
Borg does handle sparse files on backing up, but may not on restoring (i.e. may just write null blocks rather than creating holes). This is a common feature of many backup programs. However there appears to be some work being done on this, though the last update is several years ago. It's not clear from the docs what the current situation is:
https://github.com/borgbackup/borg/issues/14
On BTRFS filesystems I think this makes little if any difference. AFAIK BTRFS would handle it at a lower level.
poc
On Thu, 2025-05-08 at 22:22 -0700, ToddAndMargo via users wrote:
On 5/8/25 5:43 PM, Samuel Sieb wrote:
On 5/8/25 5:22 PM, ToddAndMargo via users wrote:
On 5/8/25 4:57 PM, Samuel Sieb wrote:
On 5/8/25 4:54 PM, ToddAndMargo via users wrote:
Fedora 41 borgbackup-1.4.1-1.fc41.x86_64
In my testing an hot to write up I made myself of borg backup, I found something confusing.
I backed up three file and restored them (somewhere else). The sha256sum matched the originals but the du's did not. What am I missing?
I'm guessing that the backup program doesn't handle sparse files.
The checksum will match because the empty space is zeroed either way.
Thank you!
My read is that if the check sum matches, the file is perfect. But, I wanted to double check in case I was missing something.
The file is equivalent. It just takes up more disk space. :-)
As a test, now that dump/restore is working again, I backed up a sparse file and restored:
Sparse file backups and restored with dump/restore: # du --bytes * 104857600 sparseFile 104857600 sparseFile.orig # du --block-size=1 * 0 sparseFile 0 sparseFile.orig # sha256sum *20492a4d0d84f8beb1767f6616229f85d44c2827b64bdbfb260ee12fa1109e0e sparseFile
20492a4d0d84f8beb1767f6616229f85d44c2827b64bdbfb260ee12fa1109e0e sparseFile.orig
Borg is incapable of the same result. And I am backing up a lot of qamu-kvm virtual drives. They are sparse files and large. If borg can not deal with sparse files, then it is not a good candidate for replacing dump/restore. Bord will give back equivalent files that are much larger.
I don't run Borg on my VM images. If I want to back up a VM I'll use the VM's own backup system internally. However this might be of interest:
https://borgbackup.readthedocs.io/en/stable/deployment/image-backup.html
poc
-
Patrick O'Callaghan -
Samuel Sieb -
ToddAndMargo