= Proposed Self Contained Change: Module Build Service = https://fedoraproject.org/wiki/Changes/ModuleBuildService
Change owner(s): * Ralph Bean rbean@redhat.com
We will deploy an instance of the Module Build Service to production in Fedora Infrastructure. Other teams will use this service to produce some "modular" content for the Fedora 26 release.
== Detailed Description == We will deploy an instance of the Module Build Service (MBS) to production in Fedora Infrastructure. Other teams will use this service to produce some "modular" content for the Fedora 26 release.
In short, the MBS is a workflow orchestration service on top of koji. When a user submits a request for a new module build:
* A new tag is created in koji for that module build. * A module-build macros package is synthesized and built in the new buildroot. * The buildroot is linked with other module tags that it has declared dependencies on. * RPMs to be included in the module are rebuilt from source in the new tag. * Kojira generates a yum/dnf repo from the new tag.
The compose tooling (pungi) will then pick up this tag and possibly include it in variants for the compose.
For the Fedora 26 timeframe, we will lock down the users who can submit to the MBS to a small number of Modularity WG members. This is not ideal, but the thought is that we want to limit the amount of spam that the MBS will impose on the production koji instance - we don't want to interfere with the general release of Fedora 26.
In the Fedora 27 timeframe, we will open it up to all packagers after we're sure MBS is sophisticated enough to throttle itself appropriately.
== Scope == * Proposal owners: We are responsible for the development, deployment, and maintenance of the Module Build Service itself. We have a prototype already functioning. At the time of writing, we still need to harden it for production, and have it vetted for deployment by Release Engineering and Fedora Infrastructure.
* Other developers: We don't think that other developers will have to make changes in response to this effort. The Base Runtime team (a sub-team of the Modularity Working Group) will be working to prepare content to be built in the Module Build Service.
* Release engineering: In order to make use of the Module Build Service, we will need changes to pungi to pull rpms for a variant from the koji tags created by the Module Build Service, but that is a separate Change proposal [https://fedoraproject.org/wiki/Changes/ModularCompose]. The owners of this proposal intend to do that work ourselves in consultation with Release Engineering.
* List of deliverables: N/A (not a System Wide Change)
* Policies and guidelines: Note that we do not think there are any packaging guidelines that will need to be changed in the Fedora 26 timeline. We would like to change the branching structure in pkgdb and dist-git in the Fedora 27 release cycle (with a separate Change document). We furthermore will need to submit new Module Guidelines that describe best practices and requirements for writing and maintaining modulemd definitions - similar to how the Packaging Guidelines describe best practices and requirements for writing and maintaining .spec files. We would like to avoid writing those Module Guidelines for the F26 cycle and instead limit the number of trusted module maintainers to a small subset of the Modularity Working Group. Once they have experience building the base modules, we'll use that experience to inform the docs we write for F27, at which time we'll open up the Module Build Service to the rest of the community.
* Trademark approval: N/A (not needed for this Change)
On Mon, Nov 7, 2016 at 4:13 AM, Jan Kurik jkurik@redhat.com wrote:
= Proposed Self Contained Change: Module Build Service = https://fedoraproject.org/wiki/Changes/ModuleBuildService
Change owner(s):
- Ralph Bean rbean@redhat.com
We will deploy an instance of the Module Build Service to production in Fedora Infrastructure. Other teams will use this service to produce some "modular" content for the Fedora 26 release.
== Detailed Description == We will deploy an instance of the Module Build Service (MBS) to production in Fedora Infrastructure. Other teams will use this service to produce some "modular" content for the Fedora 26 release.
In short, the MBS is a workflow orchestration service on top of koji. When a user submits a request for a new module build:
- A new tag is created in koji for that module build.
- A module-build macros package is synthesized and built in the new buildroot.
- The buildroot is linked with other module tags that it has declared
dependencies on.
- RPMs to be included in the module are rebuilt from source in the new tag.
Can you explain this in more detail?
How do layered modules work here? Is that what you mean by "buildroot is linked with other module tags"?
Can modules built via MBS span releases, or are you strictly limiting it to the f26 repos for base packages? (E.g. can a module declare a dependency on an older package found in f25 and have it rebuilt into the module repo?)
If all packages in a module are rebuilt from source, and we have a variety of modules that all include the packages, doesn't this increase the storage requirements over what we have today significantly?
Are modules built on all architectures? (The answer should be yes, otherwise we will face significant boot strapping issues.)
- Kojira generates a yum/dnf repo from the new tag.
The compose tooling (pungi) will then pick up this tag and possibly include it in variants for the compose.
For the Fedora 26 timeframe, we will lock down the users who can submit to the MBS to a small number of Modularity WG members. This is not ideal, but the thought is that we want to limit the amount of spam that the MBS will impose on the production koji instance - we don't want to interfere with the general release of Fedora 26.
In the Fedora 27 timeframe, we will open it up to all packagers after we're sure MBS is sophisticated enough to throttle itself appropriately.
Throttle?
== Scope ==
- Proposal owners: We are responsible for the development, deployment,
and maintenance of the Module Build Service itself. We have a prototype already functioning. At the time of writing, we still need to harden it for production, and have it vetted for deployment by Release Engineering and Fedora Infrastructure.
- Other developers: We don't think that other developers will have to
make changes in response to this effort. The Base Runtime team (a sub-team of the Modularity Working Group) will be working to prepare content to be built in the Module Build Service.
- Release engineering: In order to make use of the Module Build
Service, we will need changes to pungi to pull rpms for a variant from the koji tags created by the Module Build Service, but that is a separate Change proposal [https://fedoraproject.org/wiki/Changes/ModularCompose]. The owners of this proposal intend to do that work ourselves in consultation with Release Engineering.
Is this being done in staging to start with?
List of deliverables: N/A (not a System Wide Change)
Policies and guidelines: Note that we do not think there are any
packaging guidelines that will need to be changed in the Fedora 26 timeline. We would like to change the branching structure in pkgdb and dist-git in the Fedora 27 release cycle (with a separate Change document). We furthermore will need to submit new Module Guidelines that describe best practices and requirements for writing and maintaining modulemd definitions - similar to how the Packaging Guidelines describe best practices and requirements for writing and maintaining .spec files. We would like to avoid writing those Module Guidelines for the F26 cycle and instead limit the number of trusted module maintainers to a small subset of the Modularity Working Group. Once they have experience building the base modules, we'll use that experience to inform the docs we write for F27, at which time we'll open up the Module Build Service to the rest of the community.
I think f27 is an aggressive target, but it's possible. The documentation and policies are key here, and those need to be done well in advance of general availability.
josh
On Mon, Nov 7, 2016 at 4:13 AM, Jan Kurik jkurik@redhat.com wrote:
For the Fedora 26 timeframe, we will lock down the users who can submit to the MBS to a small number of Modularity WG members. This is not ideal, but the thought is that we want to limit the amount of spam that the MBS will impose on the production koji instance - we don't want to interfere with the general release of Fedora 26.
Is the intention then that users will be able to install modules on their system and have things work within the Fedora 26 timeframe, or would that not be possible until Fedora 27, assuming nothing slips? It seems like that would require at least one other change proposal (probably a system-wide one).
I ask as someone who's aware that Modularity is being worked on but not too clear on what it's actually going to wind up looking like and how my system / our package collection is going to change as a result. By which I mean, I understand the goal and basic concept (split packages into higher level units), but I'm iffy on the implementation details, and if we're at the point where things are going to start getting deployed in upcoming releases I'd like to read more about them.
A lot of the wiki pages on modularity [1] seem to be focused on "why is modularity a good idea" or "how do I contribute to modularity projects", but neither of these is quite what I'm looking for. Is there a "Fedora Modularity for current developers" writeup somewhere? (by which I mean "Fedora developers" in general).
Ben Rosser
Jan Kurik wrote:
We will deploy an instance of the Module Build Service (MBS) to production in Fedora Infrastructure. Other teams will use this service to produce some "modular" content for the Fedora 26 release.
In short, the MBS is a workflow orchestration service on top of koji. When a user submits a request for a new module build:
- A new tag is created in koji for that module build.
- A module-build macros package is synthesized and built in the new
buildroot.
- The buildroot is linked with other module tags that it has declared
dependencies on.
- RPMs to be included in the module are rebuilt from source in the new
tag.
- Kojira generates a yum/dnf repo from the new tag.
The compose tooling (pungi) will then pick up this tag and possibly include it in variants for the compose.
So, in short, instead of having one consistent Fedora, we will have an uncoordinated patchwork of several isolated island repositories that may or may not be compatible with each other, bringing us back to the old repository hell ("RPM hell") that ruined the reputation of RPM in the old days. Why is this something we want at all?
Modular Fedora is a horrible idea from the ground up!
Kevin Kofler
Hey, sorry I didn't respond sooner. I failed to see this hit the list.
Responses follow inline, below.
On Mon, Nov 07, 2016 at 07:22:12AM -0500, Josh Boyer wrote:
On Mon, Nov 7, 2016 at 4:13 AM, Jan Kurik jkurik@redhat.com wrote:
= Proposed Self Contained Change: Module Build Service = https://fedoraproject.org/wiki/Changes/ModuleBuildService
Change owner(s):
- Ralph Bean rbean@redhat.com
We will deploy an instance of the Module Build Service to production in Fedora Infrastructure. Other teams will use this service to produce some "modular" content for the Fedora 26 release.
== Detailed Description == We will deploy an instance of the Module Build Service (MBS) to production in Fedora Infrastructure. Other teams will use this service to produce some "modular" content for the Fedora 26 release.
In short, the MBS is a workflow orchestration service on top of koji. When a user submits a request for a new module build:
- A new tag is created in koji for that module build.
- A module-build macros package is synthesized and built in the new buildroot.
- The buildroot is linked with other module tags that it has declared
dependencies on.
- RPMs to be included in the module are rebuilt from source in the new tag.
Can you explain this in more detail?
Yes. See below.
How do layered modules work here? Is that what you mean by "buildroot is linked with other module tags"?
Yes.
To be more clear, there are two types of module relationships:
There is a "depends" relationship where one module depends on another. What this means at build-time is that the buildroot of the second module is added to the buildroot of the first. In koji terms, the tag associated with the build of the second module is added to the tag inheritance hierarchy of the build tag of the first module.
There is an "includes" relationship (that we haven't implemented yet), where one module includes another. The implementation will not involve tag inheritance. The sources of the rpms in the second module will be rebuilt in the buildroot of the second module. We will end up using this second type of relationship to build what are being called "module stacks". For F26, we will be looking primarilly at building the "generational core" stack. https://communityblog.fedoraproject.org/base-runtime-generational-core/
Can modules built via MBS span releases, or are you strictly limiting it to the f26 repos for base packages? (E.g. can a module declare a dependency on an older package found in f25 and have it rebuilt into the module repo?)
For F26, no. In the future, yes.
For starters, we're trying to *not* use f26 repos for the base packages. Instead, we're manually bootstrapping an initial base-runtime module, and future builds of the base-runtime module will declare a dependency on previous iterations.
For F27, we'll fork the f26 generational-core stack, and future modules will be built against both of those two generational-core stacks, the aim being portability.
If all packages in a module are rebuilt from source, and we have a variety of modules that all include the packages, doesn't this increase the storage requirements over what we have today significantly?
Without policy or restriction, yes.
For F26, the base set of modules we're building will have a very small number of packages (well, as small as we can get them. As I understand it, the current base-runtime has 3000 packages in it, but they think they can get it down to 400. Ask them! Better, help them.)
Note that, for distribution, I expect that for F26 we won't impose any load on the distribution network (on the mirrors). We're just producing one compose of a subset of our total package universe, and it's not getting updates. This is tiny.
In the F27 timeframe, if we're not careful about the way we carve up modules, we could get an unsustainable increase in demands on buildsystem resources (storage, cpu, etc..). If instead we are careful about that, then I think we'll still see an increase, but we can keep it in the sustainable ballpark. To help keep us honest about this, please engage with the Modularity Working Group (that team is going to be looking at guidelines and stuff like that, again, in the F27 timeframe).
Are modules built on all architectures? (The answer should be yes, otherwise we will face significant boot strapping issues.)
Yes.
(FWIW, our dev instance is only doing x86_64 at the moment, but koji is our primary backend and we'll be inheriting whatever architecture policy is in place in production there, once we get there).
- Kojira generates a yum/dnf repo from the new tag.
The compose tooling (pungi) will then pick up this tag and possibly include it in variants for the compose.
For the Fedora 26 timeframe, we will lock down the users who can submit to the MBS to a small number of Modularity WG members. This is not ideal, but the thought is that we want to limit the amount of spam that the MBS will impose on the production koji instance - we don't want to interfere with the general release of Fedora 26.
In the Fedora 27 timeframe, we will open it up to all packagers after we're sure MBS is sophisticated enough to throttle itself appropriately.
Throttle?
Merged into our codebase earlier this week. (Cheers to Matt Prahl and Jan Kaluza!)
Seriously though, we'll have to keep on top of details like this.
== Scope ==
- Proposal owners: We are responsible for the development, deployment,
and maintenance of the Module Build Service itself. We have a prototype already functioning. At the time of writing, we still need to harden it for production, and have it vetted for deployment by Release Engineering and Fedora Infrastructure.
- Other developers: We don't think that other developers will have to
make changes in response to this effort. The Base Runtime team (a sub-team of the Modularity Working Group) will be working to prepare content to be built in the Module Build Service.
- Release engineering: In order to make use of the Module Build
Service, we will need changes to pungi to pull rpms for a variant from the koji tags created by the Module Build Service, but that is a separate Change proposal [https://fedoraproject.org/wiki/Changes/ModularCompose]. The owners of this proposal intend to do that work ourselves in consultation with Release Engineering.
Is this being done in staging to start with?
It is.
List of deliverables: N/A (not a System Wide Change)
Policies and guidelines: Note that we do not think there are any
packaging guidelines that will need to be changed in the Fedora 26 timeline. We would like to change the branching structure in pkgdb and dist-git in the Fedora 27 release cycle (with a separate Change document). We furthermore will need to submit new Module Guidelines that describe best practices and requirements for writing and maintaining modulemd definitions - similar to how the Packaging Guidelines describe best practices and requirements for writing and maintaining .spec files. We would like to avoid writing those Module Guidelines for the F26 cycle and instead limit the number of trusted module maintainers to a small subset of the Modularity Working Group. Once they have experience building the base modules, we'll use that experience to inform the docs we write for F27, at which time we'll open up the Module Build Service to the rest of the community.
I think f27 is an aggressive target, but it's possible. The documentation and policies are key here, and those need to be done well in advance of general availability.
Much agreed. The core of the Modularity Working Group is responsible for this bit. I can't of course promise deadlines on their behalf, but it seems like we need something written down to start arguing about sometime before the F26 GA release.
On Mon, Nov 07, 2016 at 05:55:21PM -0500, Ben Rosser wrote:
On Mon, Nov 7, 2016 at 4:13 AM, Jan Kurik jkurik@redhat.com wrote:
For the Fedora 26 timeframe, we will lock down the users who can submit to the MBS to a small number of Modularity WG members. This is not ideal, but the thought is that we want to limit the amount of spam that the MBS will impose on the production koji instance - we don't want to interfere with the general release of Fedora 26.
Is the intention then that users will be able to install modules on their system and have things work within the Fedora 26 timeframe, or would that not be possible until Fedora 27, assuming nothing slips? It seems like that would require at least one other change proposal (probably a system-wide one).
I think you've got it. The intention in *this* Change proposal is not about providing modules that you can install at runtime. It's about producing an installable image and one basic yum repo from a set of base modules in the buildsystem.
It may be that one of the other teams around will submit proposals for some PoC "user space" modules to be built and distributed for F26, but that's beyond what I'm aiming for here.
I ask as someone who's aware that Modularity is being worked on but not too clear on what it's actually going to wind up looking like and how my system / our package collection is going to change as a result. By which I mean, I understand the goal and basic concept (split packages into higher level units), but I'm iffy on the implementation details, and if we're at the point where things are going to start getting deployed in upcoming releases I'd like to read more about them.
A lot of the wiki pages on modularity [1] seem to be focused on "why is modularity a good idea" or "how do I contribute to modularity projects", but neither of these is quite what I'm looking for. Is there a "Fedora Modularity for current developers" writeup somewhere? (by which I mean "Fedora developers" in general).
Yeah, this is right in line with some of what Josh was pointing out elsewhere in the thread. We're going to have to get some serious "module guidelines" along with some (hopefully limited) changes to the existing packaging guidelines for F27.
The stuff here, again, is about focusing on the base modules, making sure we can build them, combine them, produce an image from them, and seeing if it boots. If we can do that, it's good enough (in my opinion) to move to the next step of thinking how we can usefully modularize the rest of the distro.
That said, here are some writeups that help lay the basis for that work:
- This is kind of a "glossary": https://fedoraproject.org/wiki/Modularity/Getting_Started/Building_modular_t... - This is a kind of theoretical overview of what I'm trying to do in practice with this Change: https://fedoraproject.org/wiki/Modularity/Getting_Started/Constructing_a_mod... - This doesn't have many solutions, but it does raise a lot of the problems we'll have to solve in the F27 timeframe: https://fedoraproject.org/wiki/Modularity/Architecture/Module_versioning_and...
*NOTE - these pages look really short, but there are sub-pages if you look at the ToC.
Ben Rosser
-
Ben Rosser -
Jan Kurik -
Josh Boyer -
Kevin Kofler -
Ralph Bean