Suggestion: bmap files and bmaptool

[Artem Bityutskiy]

On Fri, 2013-08-16 at 09:51 -0500, Eric Sandeen wrote:
> by single file I meant a _single_ file, not the original file and a mapping
> file.  :)

Oh, sorry, OK.

> I realize that you have a fully-fledged set of tools, and you're not looking
> for new directions, but I was thinking about encoding mapping info & file data
> into a single file, with a tool to extract it again.  That way there's nothing
> to get out of sync.

Well, I do wish to know about alternatives, or the better ways to do it,
of course.

> Actually, now that I think about it qemu-img can do that already:
> (sorry, I'm getting a little off topic here, bear with me)
> 
> # truncate --size=1g fsfile
> # mkfs.ext4 fsfile
> # cp fsfile --sparse=never fsfile.copy
> 
> // fsfile is sparse; the copy is not.
> 
> # du -hc fsfile*
> 49M	fsfile
> 1.0G	fsfile.copy
> 
> # qemu-img convert -f raw -O qcow fsfile.copy fsfile.qcow
> 
> // the qcow image now contains only data+mapping info,
> // no zero ranges:
> # du -h fsfile.qcow
> 832K	fsfile.qcow
> 
> // and can be re-extracted into a sparse file
> 
> # qemu-img convert -O raw fsfile.qcow fsfile.copy2
> # du -hc fsfile.copy2 
> 352K	fsfile.copy2

So from 49M down to 352K, sparseness increased by 48M? Where it came
from? Must be that this command turned zero blocks into gaps. Like the
zeroed out inode tables, etc.

> Ok, sorry for that diversion, but that's cool - the tool I want
> already exists, and I hadn't realized it.  :)

Exactly this usage is not good enough for flashing purposes, because
when flashing to a block device you have to flash the zeroes, you cannot
skip them as you do for holes.

But I am sure the qcow tools can save sparseness without turning zeroes
into gaps, or at least this should not be too difficult to implement
this.

But in case of bmaptool I chose to keep the sparseness information in a
separate file because I wanted to make sure the bmap is optional. Those
who use Windows/Mac or do not want to install any additional tools could
still use the old method of throwing the entire image to the target
block device.

Another goal I had is to make the additional bmap file to be a
human-readable text file.

But yes, having all in one file does have its own advantages.

-- 
Best Regards,
Artem Bityutskiy