an interface for filesystem preallocation that doesn't suck?
Now that ext4 is the default filesystem, it would be nice to see more
programs taking advantage of its preallocation features.
But here's my question: what is the best way to actually do this?
In recent glibc, posix_fallocate() will first try to use fallocate(),
and then fall back to writing nulls to fill up the requested range if
fallocate() fails.
The problem is, posix_fallocate()'s fallback behavior effectively
results in writing the file twice. If we expect that the process of
writing the file's actual content will take a long time (e.g., I'm
using a BitTorrent client to download some Fedora DVD ISOs), then
that's not a big deal. But if I'm using a utility like rsync or cp,
the performance penalty of writing every single file twice is
unacceptable.
Additionally, the semantics that result from using fallocate() with
FALLOC_FL_KEEP_SIZE (which posix_fallocate() does NOT do) are arguably
more intuitive, because FALLOC_FL_KEEP_SIZE (essentially) creates a
sparse file:
$ fallocate-test foo $[1024*1024*512] && ls -lsa foo
fallocate-test: allocating 536870912 bytes for new file foo
524292 -rw-r--r-- 1 root root 0 2009-04-02 13:59 foo
If I'm (e.g.) downloading a large file via a P2P network, then I can
see the progress of the download by eyeballing the file's actual size
versus its allocated size.
I see that at one point, people were kicking around patches to use
ioctl() and/or fcntl() to perform preallocation. Do any of those
interfaces actually exist, or were they deprecated in favor of
fallocate()?
If the latter, is there an interface for fallocate() that doesn't have
quite as much FAIL as posix_fallocate()?