tmp on tmpfs in the cloud images: good or bad?

[Troy Dawson]

On 12/13/2012 01:00 AM, Garrett Holmstrom wrote:
> On 2012-12-12 16:42, Matthew Miller wrote:
>> The /tmp filesystem is on tmpfs by default in F18:
>> http://fedoraproject.org/wiki/Features/tmp-on-tmpfs (and affirmed by
>> FESCO
>> here: https://fedorahosted.org/fesco/ticket/940#comment:45).
>>
>> I'm agnostic about this as a default for Fedora overall, but I can see
>> it as
>> problematic on cloud/virt guests where RAM is usually more
>> constrained. (In
>> fact, there's a bugzilla report to that effect here
>> https://bugzilla.redhat.com/show_bug.cgi?id=858265).
>>
>> The feature page and release notes explicitly call out tmpfs as
>> "Administrators can override this" so I don't think we would be going too
>> far off the path if we disable it by default in the cloud images.
>>
>> What do you think?
> 
> I can think of two trains of thought here:  swap requirements and the
> size of /tmp itself.
> 
> The cost of putting /tmp on tmpfs in the worst-case scenario (the system
> needs all of its RAM *and* /tmp is full) is half the amount of RAM's
> worth of swap space.  At least in my experience, once I'm forced to
> create *any* swap space it doesn't really matter how large it is -- it's
> trivial to bump the amount of swap a little higher, and the instance is
> already likely to end up thrashing at some point anyway.  So especially
> on an I/O-constrained cloud, for me it really boils down to, "Am I going
> to have to provision swap space or not?"
> 
> For example, there's already one case of this happening even without
> /tmp on tmpfs:  a yum update on a t1.micro instance in EC2 may not
> finish if the update happens to involve the system recompiling its
> SELinux policy -- there simply isn't enough RAM to do everything.  In
> this case I don't really mind having /tmp on tmpfs since the system is
> going to be unusable during that process anyway and I only have to add
> around 300M of extra swap to compensate for the change.  The pivotal
> question for a given cloud and workload, then, is, "To how many
> instances will tmp-on-tmpfs force me to add swap space?"
> 
> The other case is, of course, the obvious one:  how big does /tmp
> actually have to be to run a reasonable Fedora server?  Do we know what
> is likely to break when /tmp is significantly smaller than usual?  Most
> of the distro's testing occurs on desktop/laptop machines with at least
> 1 or 2 GB of memory, after all.
> 
> There are enough cloud- and workload-specific variables here that I'm
> not particularly enthusiastic about putting /tmp on tmpfs, so I'd
> support masking that by default.  Since doing so would be deviating from
> Fedora's new default we should make sure to document that fact and tell
> people how to turn it back on if we go that route, though.
> 
> Those are my thoughts, at least.  What do the rest of you think?
> 
> -- 
> Garrett Holmstrom

+1 /tmp NOT on tmpfs for default cloud image
+1 document it very well

Troy