On Wed, Sep 4, 2019 at 12:01 PM Kyle Marek <psppsn96(a)gmail.com> wrote:
I'm finding that 32G is not necessarily sufficient for compiling clang
itself. Similarly I've had a hard time compiling UnrealEngine from
source. I usually see ld using up to 12G of memory to link each
artifact. Using -j$(nproc) on a 16 VCPU system amplifies the issue. I
rely on adding another 32G swap file to complete the job.
I setup swap on ZRAM device at 1.5X RAM. That was just as
intermittently unresponsive (seconds, and periods above 10 minutes)
GUI, to the degree using the laptop as a laptop was pointless. But,
webkitgtk build did complete with the default command, and in a
shorter time than with a swap partition on SSD (which became
sufficiently unresponsive even via ssh that I gave up and forced power
off).
But, I don't think I can recommend such a setup as a default
installation of Fedora Workstation. On a system with 32G RAM, that
would mean a 48G /dev/zram device for swap. If it completely filled
up, it would use ~24G of RAM, leaving 8GiB for compiling + OS +
workstation stuff. You are getting a huge swap device that's almost as
fast as RAM, but it's so fast that it means burning a huge amount of
CPU cycles on compression/decompression. And kswapd is not threaded,
it maxes out at 100% of a single core, and once it's there, that's it,
you can't scale beyond it.
I'm now using an NVME SSD for my swap file. No more hangs for the
most
part! Usually if I lock up, it's for maybe a minute before OOM steps in.
However, I'd definitely like to see a non-zram "solution" for use cases
like this. Ultimately I'd like to see the use of traditional swap files
to not hang a system even if it is placed on a md RAID array. I'm
guessing that the long term fix is for OOM to happen sooner, and/or
kernel schedulers to be improved, as mentioned elsewhere in this thread.
Ideally this wouldn't require the use of systemd or cgroups to make it
possible.
I'm skeptical of swapfiles for this use case, because it likely
increases the chance for file system disk contention, see section
11.17
https://www.kernel.org/doc/gorman/html/understand/understand014.html
And in heavy swap use and memory pressure, I already see evidence of
applications evicting their own executable code and constantly
re-reading it from disk. That's increasing the chance of IO pressure
resulting in stalls.
Incidental swap is fine, and I think it's possible to use either zram
or zswap to moderate the transition to a swap partition. But once this
grows to an aggressive swap dependency, the desktop is responsiveness
is lost.
--
Chris Murphy