Does your application depend on, or report, free disk space? Re: F20 Self Contained Change: OS Installer Support for LVM Thin Provisioning

[Mike Snitzer]

On Wed, Jul 31 2013 at  5:52am -0400,
Zdenek Kabelac <zkabelac at redhat.com> wrote:

> Dne 31.7.2013 10:39, Florian Weimer napsal(a):
> >On 07/29/2013 08:38 PM, Ric Wheeler wrote:
> >
> >>If application A does a stat or statvfs() call, sees 1GB of space left
> >>and then does a write, we could easily lose that race to any other
> >>application.
> >>
> >>If you want to reserve space, you need to grab the space yourself
> >>(always works with a large "write()" but preallocation can also help
> >>without dm-thin).
> >
> >In order to have it work "always", you'll have to write unpredictable data.
> >If you write just zeros, the reservation isn't guaranteed if the file system
> >supports compression.
> >
> >I'm pretty sure we want a crass layering violation for this one (probably a
> >new mode flag for fallocate), to ensure proper storage reservation for things
> >like VM images.
> 
> 
> If someone wants to use preallocation - thus always allocate whole space,
> than there is no reason to use provisioned devices unless someone
> want's to use its snapshot feature (for this  there could be
> probably introduced something like creation of fully provisioned
> device) - but then you end-up
> with the same problem once you start to use snapshot.
> 
> For me this rather looks like misuse of thin provisioning.
> 
> ThinP should be configured in a way that admin is able to extend
> pool to honour promised space if really needed. It's not a good
> idea, to provision 1EB if you have at most just 1TB disk and then
> you expect you will have no problems when someone fallocate() 500TB.

fallocate doesn't allow you to reserve more physical space than you have
(or allowed via quota).
 
> I.e. if someone is using  iSCSI disc array with 'hw' thin
> provisioning support, there is no scsi command to provision space -
> it's admin's work to ensure there is enough disc space to keep up
> with user demands....
> 
> Maybe - just an idea - there could be a kernel bit-flag somewhere,
> which might tell if the device used by fs is 'fully provisioned' or
> 'thin provisioned' (something like rotational/non-rotational)  But
> there is no way to return information about free disc space - since
> it's highly subjective value and moreover very expensive to
> calculate.

If things like journald _need_ to have a sysfs flag that denotes the
volume it is writing to is thinp then I'd like to understand what it'd
do differently knowing that info.  Would it conditionally call
fallocate() -- assuming dm-thinp grows REQ_RESERVE support like I
mentioned in my previous post.

I see little value in exposing whether some portion of the storage stack
is thin or not.  What is an app to do with that info?  It'd have to do
things like: 1) determine the blockdevice the FS is layered on 2) lookup
sysfs file for that device.. a filesystem can span multiple
devices.. some time some not.  It is just a rat's nest.

Thinly provisioned storage this isn't exactly a new concept.  But the
Linux provided target obviously engages other parts of the OS to
properly support it (at a minimum the volume manager and the
installer).  If the fallocate() triggered REQ_RESERVE passdown to the
underlying storage provides a reasonable stop gap we can really explore
it -- at least we'd be piggybacking on an established interface that
returns success or failure.

Mike