tmp on tmpfs in the cloud images: good or bad?

[Garrett Holmstrom]

On 2012-12-12 16:42, Matthew Miller wrote:
> The /tmp filesystem is on tmpfs by default in F18:
> http://fedoraproject.org/wiki/Features/tmp-on-tmpfs (and affirmed by FESCO
> here: https://fedorahosted.org/fesco/ticket/940#comment:45).
>
> I'm agnostic about this as a default for Fedora overall, but I can see it as
> problematic on cloud/virt guests where RAM is usually more constrained. (In
> fact, there's a bugzilla report to that effect here
> https://bugzilla.redhat.com/show_bug.cgi?id=858265).
>
> The feature page and release notes explicitly call out tmpfs as
> "Administrators can override this" so I don't think we would be going too
> far off the path if we disable it by default in the cloud images.
>
> What do you think?

I can think of two trains of thought here:  swap requirements and the 
size of /tmp itself.

The cost of putting /tmp on tmpfs in the worst-case scenario (the system 
needs all of its RAM *and* /tmp is full) is half the amount of RAM's 
worth of swap space.  At least in my experience, once I'm forced to 
create *any* swap space it doesn't really matter how large it is -- it's 
trivial to bump the amount of swap a little higher, and the instance is 
already likely to end up thrashing at some point anyway.  So especially 
on an I/O-constrained cloud, for me it really boils down to, "Am I going 
to have to provision swap space or not?"

For example, there's already one case of this happening even without 
/tmp on tmpfs:  a yum update on a t1.micro instance in EC2 may not 
finish if the update happens to involve the system recompiling its 
SELinux policy -- there simply isn't enough RAM to do everything.  In 
this case I don't really mind having /tmp on tmpfs since the system is 
going to be unusable during that process anyway and I only have to add 
around 300M of extra swap to compensate for the change.  The pivotal 
question for a given cloud and workload, then, is, "To how many 
instances will tmp-on-tmpfs force me to add swap space?"

The other case is, of course, the obvious one:  how big does /tmp 
actually have to be to run a reasonable Fedora server?  Do we know what 
is likely to break when /tmp is significantly smaller than usual?  Most 
of the distro's testing occurs on desktop/laptop machines with at least 
1 or 2 GB of memory, after all.

There are enough cloud- and workload-specific variables here that I'm 
not particularly enthusiastic about putting /tmp on tmpfs, so I'd 
support masking that by default.  Since doing so would be deviating from 
Fedora's new default we should make sure to document that fact and tell 
people how to turn it back on if we go that route, though.

Those are my thoughts, at least.  What do the rest of you think?

--
Garrett Holmstrom