Thanks a lot for this Michel!
Ben Cotton <bcotton(a)redhat.com> writes:
>
https://fedoraproject.org/wiki/Changes/MemoryConstraintsMacros
>
> == Summary ==
> Introduce macros, similar to openSUSE's
> [
https://build.opensuse.org/package/show/openSUSE:Factory/memory-constraints
> memory-constraints]), for optionally limiting build parallelism for
> build-time memory-bound packages
>
> == Owner ==
> * Name: [[User:salimma|Michel Alexandre Salim]]
> * Email: michel AT michel-slm DOT name
>
>
> == Detailed Description ==
> Some source packages have a memory usage per build thread higher than
> the RAM:CPU ratio available in some of our builders. Further, this
> ratio can be different for different build server on different
> architectures.
>
> At the moment, such packages
>
([
https://src.fedoraproject.org/rpms/ceph/blob/d7454e4e0a98208dc569553b901a...
> ceph],
[
https://src.fedoraproject.org/rpms/chromium/blob/baaf27b384295d6288ef367d...
> chromium],
[
https://src.fedoraproject.org/rpms/mcrouter/blob/a0f7ecad2ccc51c4214646b0...
> mcrouter]) have to implement their own logic for determining the safe
> amount of parallelism, and redefine `_smp_build_ncpus`.
>
> When this proposal is implemented, they can instead declaratively
> specify the amount of RAM needed per build thread, e.g.
>
> %limit_build -m 8192
>
> for declaring a build thread should be allocated 8GB of RAM.
>
> Since Koji supports
> [
https://docs.pagure.org/koji/release_notes/release_notes_1.18/#system-cha...
> setting default values for macros], there will be a macro for the
> default memory limit (name TBD) that, if set, will be used to cap
> `_smp_build_ncpus` unless overridden by `%limit_build -m`.
> 0
> I'm proposing to tentatively call the macro package
> `build-constraints-rpm-macros` to allow the possibility of adding
> macros for related needs e.g. [
https://pagure.io/copr/copr/issue/1678
> build timeouts] to the same package.
>
>
> == Benefit to Fedora ==
> This change simplifies maintaining specs for software that are
> memory-bounded rather than CPU-bounded on our build servers
>
> It could potentially improve build reliability for these packages, by
> reducing the number of jobs failing because of OOM errors, and reduce
> the need for package maintainers to debug these failures.
>
> By keeping the user-facing API aligned with what openSUSE is using, we
> open up the possibility to collaborate with them and with the upstream
> RPM project to get such macros upstreamed into RPM itself (see
> [
https://github.com/rpm-software-management/rpm/pull/821 previous
> attempt]). **note** that is not in scope for this Change.
>
> == Scope ==
> * Proposal owners:
> ** Introduce new macros
> ** Update known packages to use the new macros, replacing their custom
> `_smp_build_ncpus` overrides
>
> * Other developers:
> ** The proposal owners might not catch all references of such logic.
> Individual package maintainers can try refactoring their packages
> using these new macros
>
> * Release engineering: [
https://pagure.io/releng/issue/10188 #10188]
> No mass rebuild needed. Affected packages should be rebuilt using the new macro
>
> * Policies and guidelines: Packaging guideline can be updated to
> recommend using these macros for build-time memory-bound packages
>
> * Trademark approval: N/A (not needed for this Change)
> * Alignment with Objectives: N/A
>
>
> == Upgrade/compatibility impact ==
> No impact, affects package building only. Also, the use of the new
> macros are optional.
>
>
>
> == How To Test ==
> 1. Install `build-constraints-rpm-macros`
> 2. Modify spec to set `%limit_build -n AMOUNT_IN_MB` in `%build`
> 3. Rebuild in koji and make sure it passes on all supported architectures
>
>
>
> == User Experience ==
>
>
> == Dependencies ==
> This can optionally be added as dependencies of `redhat-rpm-config`
> and `epel-rpm-macros`, depending on how many packages need this
>
>
>
> == Contingency Plan ==
>
> * Contingency mechanism: (What to do? Who will do it?)
> Revert changed packages to their previous way of capping the number of
> build jobs
>
> * Contingency deadline: beta
> * Blocks release? No
>
>
> == Documentation ==
> Previous discussion:
>
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
>
> openSUSE implementation:
>
https://build.opensuse.org/package/show/openSUSE:Factory/memory-constraints
>
>
> --
> Ben Cotton
> He / Him / His
> Fedora Program Manager
> Red Hat
> TZ=America/Indiana/Indianapolis
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure