On to, 28 tammi 2021, Chris Murphy wrote:
On Thu, Jan 28, 2021, 6:21 AM Alexander Bokovoy
<[1]abokovoy(a)redhat.com>
wrote:
With today's OpenQA tests I can point out that using zram on 2048MB RAM
VMs actually breaks FreeIPA deployment:
[
2]https://openqa.fedoraproject.org/tests/763006#step/role_deploy_domain_c...
OpenQA uses 2048MB RAM for QEMU VMs and this was typically OK for
FreeIPA deployment with integrated CA and DNS server. Not anymore with
zram activated:
Jan 27 21:17:47 fedora zram_generator::generator[25243]: Creating unit
dev-zram0.swap (/dev/zram0 with 1384MB)
Swap on zram isn't recently enabled in Fedora, so why are the tests
recently failing?
Also, the default fraction is 0.5 so the zram device size should be
1024MB. Why is it 1384MB?
I have no idea why. This is Rawhide of today, automatically provisioned
in OpenQA. All logs are available in 'Logs and Artifacts' tab on the
OpenQA page referenced above.
Tests started to fail because we raised the low memory limit in FreeIPA
from 0.7GB to 1.2GB after seeing real world issues with lower memory
pressures.
which ends up eating 2/3rds of the whole memory budget and
FreeIPA
installer fails:
That's not possible with default settings. The device size is not the
amount of memory used. The device size is virtual. The real amount used
depends on what's paged out to swap divided by the commission ratio.
If swap is being used at all it means the workload already used ~95% of
memory.
In the OpenQA test there is nothing running on the system yet. This
literally happens when a test runs 'ipa-server-install' and we haven't
yet gone to configure *anything*. This check is one of the earliest in
the installer.
2021-01-28T02:18:31Z DEBUG ipa-server-install was invoked with arguments
[] and options: {'unattended': True, 'ip_addresses': None,
'domain_name': '[3]test.openqa.fedoraproject.org',
'realm_name':
'[4]TEST.OPENQA.FEDORAPROJECT.ORG', 'host_name': None, 'ca_cert
2021-01-28T02:18:31Z DEBUG IPA version 4.9.1-1.fc34
2021-01-28T02:18:31Z DEBUG IPA platform fedora
2021-01-28T02:18:31Z DEBUG IPA os-release Fedora 34 (Server Edition
Prerelease)
2021-01-28T02:18:31Z DEBUG Available memory is 823529472B
...
2021-01-28T02:18:31Z DEBUG The ipa-server-install command failed,
exception: ScriptError: Less than the minimum 1.2GB of RAM is available,
0.77GB available.
We need more info. Something is consuming more memory than the
provisioning expects. If there was no swap, the problem would be worse.
Please look into OpenQA logs. There is a tarball with /var/log/* content
there (and few more things), including a full systemd journal which
might have some additional information.
--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland