On Tue, Jun 29, 2021 at 04:25:13PM -0400, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/MemoryConstraintsMacros
== Summary ==
Introduce macros, similar to openSUSE's
[
https://build.opensuse.org/package/show/openSUSE:Factory/memory-constraints
memory-constraints]), for optionally limiting build parallelism for
build-time memory-bound packages
== Owner ==
* Name: [[User:salimma|Michel Alexandre Salim]]
* Email: michel AT michel-slm DOT name
== Detailed Description ==
Some source packages have a memory usage per build thread higher than
the RAM:CPU ratio available in some of our builders. Further, this
ratio can be different for different build server on different
architectures.
At the moment, such packages
([
https://src.fedoraproject.org/rpms/ceph/blob/d7454e4e0a98208dc569553b901a...
ceph],
[
https://src.fedoraproject.org/rpms/chromium/blob/baaf27b384295d6288ef367d...
chromium],
[
https://src.fedoraproject.org/rpms/mcrouter/blob/a0f7ecad2ccc51c4214646b0...
mcrouter]) have to implement their own logic for determining the safe
amount of parallelism, and redefine `_smp_build_ncpus`.
When this proposal is implemented, they can instead declaratively
specify the amount of RAM needed per build thread, e.g.
%limit_build -m 8192
for declaring a build thread should be allocated 8GB of RAM.
This is now implemented on Rawhide; Fedora updates are in testing:
F34:
https://bodhi.fedoraproject.org/updates/FEDORA-2021-7c42e41506
F33:
https://bodhi.fedoraproject.org/updates/FEDORA-2021-a5cd9680ba
Works fine with `mcrouter`:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1812360
The EPEL changes (to `epel-rpm-macros`) are committed and built, but
not published to Bodhi yet since Miro's `%py3_check_import` updates are
still in flight.
Could the Ceph maintainers help test this too? I looked at the spec, and
changing it seems straightforward, *but* I don't know how much the spec
is meant to be kept in sync with openSUSE (there's some openSUSE centric
definitions there).
`%limit_build` could be used on both openSUSE and Fedora now, but the
SUSE variant is slightly different (it seems to be inspired by Ceph to
begin with): it considers as available memory both physical and swap,
whereas Ceph (and the version I added to `redhat-rpm-config` /
`epel-rpm-macros`) only considers physical memory, which seems more
reasonable.
Best regards,
--
Michel Alexandre Salim
profile:
https://keyoxide.org/michel@michel-slm.name