Suggestion: bmap files and bmaptool

[Eric Sandeen]

On 8/16/13 3:46 AM, Artem Bityutskiy wrote:
>> Another approach which might (?) be more robust, is to somehow encode
>> > that sparseness in a single file format that can be
>> > transported/compressed/copied w/o losing the sparseness information,
>> > and another tool to operate efficiently on that format at the
>> > destination, either by unpacking it to a normal sparse file or piping
>> > it to some other process.
> Err, not sure I fully understand, but it sounds like what bmap-tools
> project actually does.

by single file I meant a _single_ file, not the original file and a mapping
file.  :)

I realize that you have a fully-fledged set of tools, and you're not looking
for new directions, but I was thinking about encoding mapping info & file data
into a single file, with a tool to extract it again.  That way there's nothing
to get out of sync.

Actually, now that I think about it qemu-img can do that already:
(sorry, I'm getting a little off topic here, bear with me)

# truncate --size=1g fsfile
# mkfs.ext4 fsfile
# cp fsfile --sparse=never fsfile.copy

// fsfile is sparse; the copy is not.

# du -hc fsfile*
49M	fsfile
1.0G	fsfile.copy

# qemu-img convert -f raw -O qcow fsfile.copy fsfile.qcow

// the qcow image now contains only data+mapping info,
// no zero ranges:
# du -h fsfile.qcow
832K	fsfile.qcow

// and can be re-extracted into a sparse file

# qemu-img convert -O raw fsfile.qcow fsfile.copy2
# du -hc fsfile.copy2 
352K	fsfile.copy2

Ok, sorry for that diversion, but that's cool - the tool I want
already exists, and I hadn't realized it.  :)

So that's a decent option for encoding sparse files for efficient
transfer, too.

> Piping is not implemented, because sparseness cannot be easily passed
> though a pipe.

Err, right ;)

-Eric

>> > Just some thoughts...
>> > 
> Thanks a lot for the feed-back!