This seems very reminiscent of what is often referred to as "swapping insanity", often in the context of MySQL.
The root cause there is NUMA with memory allocation only happening on the node that requests the memory, resulting in the possibility of there being plenty of free memory on one node but the allocating node deciding to swap out instead of allocating far memory. I don't think there was ever a solution fielded, but the workaround was to set the memory heavy process up with numa round-robin allocation. This results in even allocation across the nodes, thus avoiding swapping, but it capitulates to the fact that half of the memory is always going to be twice as far away in terms of latency.
I don't know if the underlying root cause is shared in this case (something less than clever in the way memory allocation is handled).

On Mon, Aug 19, 2019 at 9:55 AM Florian Weimer <fweimer@redhat.com> wrote:
* Gordan Bobic:

> That may be so, but this thread started off with memory pressure also
> being an issue for regular desktop x86 use.

I think the problem there is that the system has sufficient reclaimable
memory, but cannot make that memory available to applications in a
timely fashion.

Reducing compiled application footprint will only increase the amount of
reclaimable memory, probably not changing anything as far as the actual
problem is concerned.

Thanks,
Florian