Hi, all,
This topic goes along the lines of Matthew’s Operating System Factory discussion[1], but with a slightly different angle. It also is the generalization of the Change we have proposed recently [2]
So let me start with the problem:
In Fedora now we have a well known and established process of how to deal with updates that bring new content through packages. There are ways to package new content side by side with the old one, to perform a non-disruptive testing, iterate and upgrade.
What I think is missing is the way to safely iterate on the process, which happens after the content is packaged. Examples are: changes to buildroot configuration as in [2], changes in comps files, for example required for Minimization Objective [3], changes in compose building process, changes in Mass Rebuild and so on.
For such changes we have to use an all-or-nothing approach, which means we either implement the change too early or postpone it for too long.
To add flexibility to the Fedora process, I’d like to propose the concept of alternative buildroots.
------ Note: The naming is hard here, and while I tend to call it “buildroot”, it actually needs to be called “alternative everything, except the srpms”.
I think we shouldn’t use the word “variant”, “spin”, “flavour” or something like that to describe it, as it is strictly the shared development playground linked to the latest Fedora Rawhide state and focused on the build and compose process. It is definitely not the alternative Fedora. It shouldn’t have releases on its own, it has no branches in the source code and it doesn’t target end-users. ------
The good thing is that, with the distributed CI approach we have implemented, we are ready to consume the feedback from such alternative setups in a non-blocking but informative way. I think we can extend our use of CI machinery so that it simplifies the development process for everyone, removing redundant heads-ups but increasing the targeted collaboration in places where it is actually needed.
### Use cases ###
The first example of such a buildroot is provided by the CPU baseline testing [1].
There will be more, since we may want to work on comps files for Minimization Objective.
There is also one particularly important case of the RHEL bootstrap: RHEL tries hard to inherit as much as possible from Fedora, but since both Fedora and RHEL have historically different build and compose configurations, it is often problematic to deal with dependency and build issues which arise from it. To the point that RHEL sometimes just can’t use Fedora as an origin, and needs to find its way around.
The alternative buildroot targeting CentOS and RHEL would provide an open shared development environment. There we can work on converging the RHEL process to Fedora, and also on improving Fedora process with things, which RHEL and CentOS have discovered.
### Proposal ###
This idea doesn’t quite fit into the Fedora Changes framework, as there is no actual change to Fedora. And we don't even have a concept of an Infrastructure Change at this point.
But there are several items one can think about:
1) Though as I said above the “buildroot” term is slightly misleading but it is a start. And the change [2] is a first step, the prototype, which goes into that direction. Its main focus is the compiler parameters rather than anything else.
2) The second part of the proposal is to setup a CentOS/RHEL-targeting buildroot and compose, with a focus on comps, dependency chains and compose differences. Which I think overlaps in a certain way with the Minimization Objective.
3) And the third part is to generally develop the alternative buildroot as a concept. It should be our toolbox item, something similar to the alternative architectures approach. So that it can be requested by anyone based on the existence of a SIG or working group, which would own and maintain it.
[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... [2] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... [3] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
-- Aleksandra Fedorova bookwar
Many of us deeply involved in RHEL's production are excited about this idea. We are looking for ways to do more of the RHEL 9 creation activities in public and this could help serve that purpose. Something along these lines would be fantastic because identical Fedora sources could be used to produce differentiated builds, where only inherited rpm macros would differ. Perhaps even better is that it looks like it would give Fedora the ability to do more to serve its own mission, by allowing customization without forking sources. Fedora would be able to grow beyond the traditional one-set-of-defaults-must-fit-all model.
On Mon, Jan 13, 2020 at 8:45 AM Aleksandra Fedorova afedorova@redhat.com wrote:
Hi, all,
This topic goes along the lines of Matthew’s Operating System Factory discussion[1], but with a slightly different angle. It also is the generalization of the Change we have proposed recently [2]
So let me start with the problem:
In Fedora now we have a well known and established process of how to deal with updates that bring new content through packages. There are ways to package new content side by side with the old one, to perform a non-disruptive testing, iterate and upgrade.
What I think is missing is the way to safely iterate on the process, which happens after the content is packaged. Examples are: changes to buildroot configuration as in [2], changes in comps files, for example required for Minimization Objective [3], changes in compose building process, changes in Mass Rebuild and so on.
For such changes we have to use an all-or-nothing approach, which means we either implement the change too early or postpone it for too long.
To add flexibility to the Fedora process, I’d like to propose the concept of alternative buildroots.
Note: The naming is hard here, and while I tend to call it “buildroot”, it actually needs to be called “alternative everything, except the srpms”.
I think we shouldn’t use the word “variant”, “spin”, “flavour” or something like that to describe it, as it is strictly the shared development playground linked to the latest Fedora Rawhide state and focused on the build and compose process. It is definitely not the alternative Fedora. It shouldn’t have releases on its own, it has no branches in the source code and it doesn’t target end-users.
The good thing is that, with the distributed CI approach we have implemented, we are ready to consume the feedback from such alternative setups in a non-blocking but informative way. I think we can extend our use of CI machinery so that it simplifies the development process for everyone, removing redundant heads-ups but increasing the targeted collaboration in places where it is actually needed.
### Use cases ###
The first example of such a buildroot is provided by the CPU baseline testing [1].
There will be more, since we may want to work on comps files for Minimization Objective.
There is also one particularly important case of the RHEL bootstrap: RHEL tries hard to inherit as much as possible from Fedora, but since both Fedora and RHEL have historically different build and compose configurations, it is often problematic to deal with dependency and build issues which arise from it. To the point that RHEL sometimes just can’t use Fedora as an origin, and needs to find its way around.
The alternative buildroot targeting CentOS and RHEL would provide an open shared development environment. There we can work on converging the RHEL process to Fedora, and also on improving Fedora process with things, which RHEL and CentOS have discovered.
### Proposal ###
This idea doesn’t quite fit into the Fedora Changes framework, as there is no actual change to Fedora. And we don't even have a concept of an Infrastructure Change at this point.
But there are several items one can think about:
- Though as I said above the “buildroot” term is slightly misleading
but it is a start. And the change [2] is a first step, the prototype, which goes into that direction. Its main focus is the compiler parameters rather than anything else.
- The second part of the proposal is to setup a CentOS/RHEL-targeting
buildroot and compose, with a focus on comps, dependency chains and compose differences. Which I think overlaps in a certain way with the Minimization Objective.
- And the third part is to generally develop the alternative
buildroot as a concept. It should be our toolbox item, something similar to the alternative architectures approach. So that it can be requested by anyone based on the existence of a SIG or working group, which would own and maintain it.
[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... [2] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... [3] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
-- Aleksandra Fedorova bookwar _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
-- Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc.
I am chiming in with a slightly long form of +1 because of the lack of comments.
I think this is HUGE. To be able to connect to new use cases and ideas and have the bonus of helping a downstream stay more closely connected feels like a big win!
regards,
bex
On Tue, Jan 14, 2020 at 5:54 AM Brendan Conoboy blc@redhat.com wrote:
Many of us deeply involved in RHEL's production are excited about this idea. We are looking for ways to do more of the RHEL 9 creation activities in public and this could help serve that purpose. Something along these lines would be fantastic because identical Fedora sources could be used to produce differentiated builds, where only inherited rpm macros would differ. Perhaps even better is that it looks like it would give Fedora the ability to do more to serve its own mission, by allowing customization without forking sources. Fedora would be able to grow beyond the traditional one-set-of-defaults-must-fit-all model.
On Mon, Jan 13, 2020 at 8:45 AM Aleksandra Fedorova afedorova@redhat.com wrote:
Hi, all,
This topic goes along the lines of Matthew’s Operating System Factory discussion[1], but with a slightly different angle. It also is the generalization of the Change we have proposed recently [2]
So let me start with the problem:
In Fedora now we have a well known and established process of how to deal with updates that bring new content through packages. There are ways to package new content side by side with the old one, to perform a non-disruptive testing, iterate and upgrade.
What I think is missing is the way to safely iterate on the process, which happens after the content is packaged. Examples are: changes to buildroot configuration as in [2], changes in comps files, for example required for Minimization Objective [3], changes in compose building process, changes in Mass Rebuild and so on.
For such changes we have to use an all-or-nothing approach, which means we either implement the change too early or postpone it for too long.
To add flexibility to the Fedora process, I’d like to propose the concept of alternative buildroots.
Note: The naming is hard here, and while I tend to call it “buildroot”, it actually needs to be called “alternative everything, except the srpms”.
I think we shouldn’t use the word “variant”, “spin”, “flavour” or something like that to describe it, as it is strictly the shared development playground linked to the latest Fedora Rawhide state and focused on the build and compose process. It is definitely not the alternative Fedora. It shouldn’t have releases on its own, it has no branches in the source code and it doesn’t target end-users.
The good thing is that, with the distributed CI approach we have implemented, we are ready to consume the feedback from such alternative setups in a non-blocking but informative way. I think we can extend our use of CI machinery so that it simplifies the development process for everyone, removing redundant heads-ups but increasing the targeted collaboration in places where it is actually needed.
### Use cases ###
The first example of such a buildroot is provided by the CPU baseline testing [1].
There will be more, since we may want to work on comps files for Minimization Objective.
There is also one particularly important case of the RHEL bootstrap: RHEL tries hard to inherit as much as possible from Fedora, but since both Fedora and RHEL have historically different build and compose configurations, it is often problematic to deal with dependency and build issues which arise from it. To the point that RHEL sometimes just can’t use Fedora as an origin, and needs to find its way around.
The alternative buildroot targeting CentOS and RHEL would provide an open shared development environment. There we can work on converging the RHEL process to Fedora, and also on improving Fedora process with things, which RHEL and CentOS have discovered.
### Proposal ###
This idea doesn’t quite fit into the Fedora Changes framework, as there is no actual change to Fedora. And we don't even have a concept of an Infrastructure Change at this point.
But there are several items one can think about:
- Though as I said above the “buildroot” term is slightly misleading
but it is a start. And the change [2] is a first step, the prototype, which goes into that direction. Its main focus is the compiler parameters rather than anything else.
- The second part of the proposal is to setup a CentOS/RHEL-targeting
buildroot and compose, with a focus on comps, dependency chains and compose differences. Which I think overlaps in a certain way with the Minimization Objective.
- And the third part is to generally develop the alternative
buildroot as a concept. It should be our toolbox item, something similar to the alternative architectures approach. So that it can be requested by anyone based on the existence of a SIG or working group, which would own and maintain it.
[1]
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
[2]
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
[3]
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
-- Aleksandra Fedorova bookwar _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
-- Brendan Conoboy / RHEL Development Coordinator / Red Hat, Inc. _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
I really like this proposal. I feel like it's something we needed for a long time. More comments inline!
On Mon, Jan 13, 2020 at 5:45 PM Aleksandra Fedorova afedorova@redhat.com wrote:
Hi, all,
This topic goes along the lines of Matthew’s Operating System Factory discussion[1], but with a slightly different angle. It also is the generalization of the Change we have proposed recently [2]
So let me start with the problem:
In Fedora now we have a well known and established process of how to deal with updates that bring new content through packages. There are ways to package new content side by side with the old one, to perform a non-disruptive testing, iterate and upgrade.
What I think is missing is the way to safely iterate on the process, which happens after the content is packaged. Examples are: changes to buildroot configuration as in [2], changes in comps files, for example required for Minimization Objective [3], changes in compose building process, changes in Mass Rebuild and so on.
Yes! I felt that a few times in the past.
We know how to make our output evolve, how to introduce new RPM versions and make other output-related changes. We test all updates thanks to Bodhi, try and test bigger changes in side tags, etc. But we don't really have a way to safely experiment with what makes it all happen — the Fedora build system. And by "build system", I mean it in the broadest sense, including Koji, compose, configurations, and many other parts).
And that's exactly where things like improving packager experience in an actual significant way (that requires some testing and iterations), or performance tuning that affects compatibility — requiring all those technical changes you've listed — happen. Or would happen!
For such changes we have to use an all-or-nothing approach, which means we either implement the change too early or postpone it for too long.
... or don't have space to experiment with bigger changes, potentially limiting our ability to innovate. And when we try to, there is a high risk of negatively impacting our contributors, because, well, production is where the testing happens.
To add flexibility to the Fedora process, I’d like to propose the concept of alternative buildroots.
Alternative buildroots are a good start.
Note: The naming is hard here, and while I tend to call it “buildroot”, it actually needs to be called “alternative everything, except the srpms”.
Oh! Ok, that's exactly what I was hoping for and that's even a better start! (Or, you know, all we need :D)
I think we shouldn’t use the word “variant”, “spin”, “flavour” or something like that to describe it, as it is strictly the shared development playground linked to the latest Fedora Rawhide state and focused on the build and compose process. It is definitely not the alternative Fedora. It shouldn’t have releases on its own, it has no branches in the source code and it doesn’t target end-users.
The good thing is that, with the distributed CI approach we have implemented, we are ready to consume the feedback from such alternative setups in a non-blocking but informative way. I think we can extend our use of CI machinery so that it simplifies the development process for everyone, removing redundant heads-ups but increasing the targeted collaboration in places where it is actually needed.
### Use cases ###
The first example of such a buildroot is provided by the CPU baseline testing [1].
There will be more, since we may want to work on comps files for Minimization Objective.
As part of the objective, we avoided such changes as they're quite disruptive. I was actually thinking about trying those "somewhere" on the side, but this proposal sounds like enabling this, done properly as part of our infra, for everyone. Cool!
There is also one particularly important case of the RHEL bootstrap: RHEL tries hard to inherit as much as possible from Fedora, but since both Fedora and RHEL have historically different build and compose configurations, it is often problematic to deal with dependency and build issues which arise from it. To the point that RHEL sometimes just can’t use Fedora as an origin, and needs to find its way around.
The alternative buildroot targeting CentOS and RHEL would provide an open shared development environment. There we can work on converging the RHEL process to Fedora, and also on improving Fedora process with things, which RHEL and CentOS have discovered.
### Proposal ###
This idea doesn’t quite fit into the Fedora Changes framework, as there is no actual change to Fedora. And we don't even have a concept of an Infrastructure Change at this point.
Infra Changes are an interesting concept for this.
One thing that struck my mind was... Testing alternative approaches on the side without affecting people is not really a change. That said, Changes we do for content, implemented more or less in the main space, force us get there faster. Sure, bigger changes are properly tested in side tags for example, but some are done directly (while being properly announced of course).
What I'm trying to say is... I think there is a balance of speed vs. having a safe space to innovate on bigger things. And we definitely need both for infra.
But there are several items one can think about:
- Though as I said above the “buildroot” term is slightly misleading
but it is a start. And the change [2] is a first step, the prototype, which goes into that direction. Its main focus is the compiler parameters rather than anything else.
I think this is a good start that will touch many places we'll need to tweak to make them flexible enough to handle multiple build configs and, of course, multiple outputs of those.
- The second part of the proposal is to setup a CentOS/RHEL-targeting
buildroot and compose, with a focus on comps, dependency chains and compose differences. Which I think overlaps in a certain way with the Minimization Objective.
Will this be a general thing, that will allow others to build such additional pipelines for testing? Or will this be solely for the CentOS/RHEL-targeting part?
- And the third part is to generally develop the alternative
buildroot as a concept. It should be our toolbox item, something similar to the alternative architectures approach. So that it can be requested by anyone based on the existence of a SIG or working group, which would own and maintain it.
I think this answer my question — we'll test it on that one thing, learn some lessons, and based on that we'll come up with a generic way for everyone. And that actually sounds much better than trying to come up with a generic thing first. Did I get that right?
----
I'm actually really excited about this effort. Definitely interested in helping with some of that!
[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... [2] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... [3] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
-- Aleksandra Fedorova bookwar _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
On Mon, Jan 13, 2020 at 05:43:26PM +0100, Aleksandra Fedorova wrote: ...snip...
Note: The naming is hard here, and while I tend to call it “buildroot”, it actually needs to be called “alternative everything, except the srpms”.
I think we shouldn’t use the word “variant”, “spin”, “flavour” or something like that to describe it, as it is strictly the shared development playground linked to the latest Fedora Rawhide state and focused on the build and compose process. It is definitely not the alternative Fedora. It shouldn’t have releases on its own, it has no branches in the source code and it doesn’t target end-users.
petri dish? too hard to say...fields (like planting a test crop in another field)? Naming is hard. :)
The good thing is that, with the distributed CI approach we have implemented, we are ready to consume the feedback from such alternative setups in a non-blocking but informative way. I think we can extend our use of CI machinery so that it simplifies the development process for everyone, removing redundant heads-ups but increasing the targeted collaboration in places where it is actually needed.
### Use cases ###
The first example of such a buildroot is provided by the CPU baseline testing [1].
There will be more, since we may want to work on comps files for Minimization Objective.
Yeah, but comps files are used in composes, so we would have to do composes to see changes there. I guess the CI part would do the composing in that case? But we would also need a different comps (so we didn't change the default one).
There is also one particularly important case of the RHEL bootstrap: RHEL tries hard to inherit as much as possible from Fedora, but since both Fedora and RHEL have historically different build and compose configurations, it is often problematic to deal with dependency and build issues which arise from it. To the point that RHEL sometimes just can’t use Fedora as an origin, and needs to find its way around.
The alternative buildroot targeting CentOS and RHEL would provide an open shared development environment. There we can work on converging the RHEL process to Fedora, and also on improving Fedora process with things, which RHEL and CentOS have discovered.
+1 !
### Proposal ###
This idea doesn’t quite fit into the Fedora Changes framework, as there is no actual change to Fedora. And we don't even have a concept of an Infrastructure Change at this point.
Well, infrastructure has a Request for resources process. However, it's a bit dated these days, not taking into account OpenShift or the like. The CPE team has been wanting to move to a higher level process for this kind of thing... but yeah, nothing is really clear, I'd say just get buyin from stakeholders/fesco.
But there are several items one can think about:
- Though as I said above the “buildroot” term is slightly misleading
but it is a start. And the change [2] is a first step, the prototype, which goes into that direction. Its main focus is the compiler parameters rather than anything else.
Yep.
- The second part of the proposal is to setup a CentOS/RHEL-targeting
buildroot and compose, with a focus on comps, dependency chains and compose differences. Which I think overlaps in a certain way with the Minimization Objective.
So, this would have composes? Or they would be test composes in CI/ODCS?
- And the third part is to generally develop the alternative
buildroot as a concept. It should be our toolbox item, something similar to the alternative architectures approach. So that it can be requested by anyone based on the existence of a SIG or working group, which would own and maintain it.
Yeah, although I think we should still examine if a thing is ready for testing this way. I don't think we should just add anything that anyone can think of. It does use resources. That said, I think if the concept for testing is reasonable we could add it.
Thanks for working on this and I'm eager to see how well we can get it working! :)
kevin
On Wed, Jan 15, 2020 at 8:31 AM Kevin Fenzi kevin@scrye.com wrote: [snip]
### Use cases ###
The first example of such a buildroot is provided by the CPU baseline testing [1].
There will be more, since we may want to work on comps files for Minimization Objective.
Yeah, but comps files are used in composes, so we would have to do composes to see changes there. I guess the CI part would do the composing in that case? But we would also need a different comps (so we didn't change the default one).
You would need a different comps, but that also means you would get to have a different comps to experiment with...
[snip]
But there are several items one can think about:
- Though as I said above the “buildroot” term is slightly misleading
but it is a start. And the change [2] is a first step, the prototype, which goes into that direction. Its main focus is the compiler parameters rather than anything else.
Yep.
- The second part of the proposal is to setup a CentOS/RHEL-targeting
buildroot and compose, with a focus on comps, dependency chains and compose differences. Which I think overlaps in a certain way with the Minimization Objective.
So, this would have composes? Or they would be test composes in CI/ODCS?
One potential output of this change would be to have editions choose what buildroot they are based on. As an example, Fedora Server, which has struggled to find its element, could actually be built in ways more conducive to server use cases: different compiler flags, kernel parameters, baselines, etc.
On Thu, Jan 16, 2020 at 11:27:36AM -0800, Brendan Conoboy wrote:
On Wed, Jan 15, 2020 at 8:31 AM Kevin Fenzi kevin@scrye.com wrote: [snip]
### Use cases ###
The first example of such a buildroot is provided by the CPU baseline testing [1].
There will be more, since we may want to work on comps files for Minimization Objective.
Yeah, but comps files are used in composes, so we would have to do composes to see changes there. I guess the CI part would do the composing in that case? But we would also need a different comps (so we didn't change the default one).
You would need a different comps, but that also means you would get to have a different comps to experiment with...
Sure.
[snip]
But there are several items one can think about:
- Though as I said above the “buildroot” term is slightly misleading
but it is a start. And the change [2] is a first step, the prototype, which goes into that direction. Its main focus is the compiler parameters rather than anything else.
Yep.
- The second part of the proposal is to setup a CentOS/RHEL-targeting
buildroot and compose, with a focus on comps, dependency chains and compose differences. Which I think overlaps in a certain way with the Minimization Objective.
So, this would have composes? Or they would be test composes in CI/ODCS?
One potential output of this change would be to have editions choose what buildroot they are based on. As an example, Fedora Server, which has struggled to find its element, could actually be built in ways more conducive to server use cases: different compiler flags, kernel parameters, baselines, etc.
Possibly, but also splitting things means they have less people using them, and also there may be less compromise to come up with something everyone can share.
But I'm perfectly happy to give this a try and see where it leads us. :)
kevin
On Thu, Jan 16, 2020 at 5:14 PM Kevin Fenzi kevin@scrye.com wrote:
On Thu, Jan 16, 2020 at 11:27:36AM -0800, Brendan Conoboy wrote:
One potential output of this change would be to have editions choose what buildroot they are based on. As an example, Fedora Server, which has struggled to find its element, could actually be built in ways more conducive to server use cases: different compiler flags, kernel parameters, baselines, etc.
Possibly, but also splitting things means they have less people using them, and also there may be less compromise to come up with something everyone can share.
This may be true, but if the number of people grows as a result, that might be worth it. No need to decide right nwo of course. Part of what is good about Aleksandra's proposal is that it's incremental- each step along the way has a specific benefit. The long term potential for this is exciting, but the short and medium term benefits seem good too. Change can happen a little at a time then be measured and adjusted as needed.
But I'm perfectly happy to give this a try and see where it leads us. :)
Cool :-)
On Thu, Jan 16, 2020 at 9:30 PM Brendan Conoboy blc@redhat.com wrote:
On Thu, Jan 16, 2020 at 5:14 PM Kevin Fenzi kevin@scrye.com wrote:
On Thu, Jan 16, 2020 at 11:27:36AM -0800, Brendan Conoboy wrote:
One potential output of this change would be to have editions choose what buildroot they are based on. As an example, Fedora Server, which has struggled to find its element, could actually be built in ways more conducive to server use cases: different compiler flags, kernel parameters, baselines, etc.
Possibly, but also splitting things means they have less people using them, and also there may be less compromise to come up with something everyone can share.
This may be true, but if the number of people grows as a result, that might be worth it. No need to decide right nwo of course. Part of what is good about Aleksandra's proposal is that it's incremental- each step along the way has a specific benefit. The long term potential for this is exciting, but the short and medium term benefits seem good too. Change can happen a little at a time then be measured and adjusted as needed.
But I'm perfectly happy to give this a try and see where it leads us. :)
Cool :-)
I don't think I've made any secret that I would love to have the ability to do awesome large-scale improvements in Fedora more easily. I've mentioned in various avenues that I'd love for us to have that to give people the ability to build an awesome distro and take it in crazy directions to see where we can truly go.
I've also said that I don't think we can handle it as our infrastructure currently stands. Our build system tooling has suffered from a decade of neglect, and attempts to reinvent or improve that have been rebuffed or failed in other ways. We have not learned from what other communities are or are not doing, either out of hubris or ignorance (I'm really hoping it's the latter, personally). Some of these problems have been broadly exposed as part of the Modularity bringup, but there's an underlying general problem that it is okay to hobble along because we've hobbled along for so long. If anyone wants to know details, feel free to ask me. I don't want to fill this with the sordid details...
I've had the pleasure of becoming involved in many other communities over the past several years, and have had exposure to how others have solved problems we've had. Consequently, I've been spoiled at $DAYJOB where some of these issues we've been talking about for a couple of years simply don't exist because our tooling and infrastructure has effectively solved it for us. That's not to say it's perfect and we have no problems at all. There are still issues and quirks that I have to work around just like Fedora does with what the project has. I'm slowly working through trying to make what I use for $DAYJOB available within Fedora so members of the Fedora community can use it for their own purposes. After all, the stuff I use is open source too (except for the stuff I wrote, which I'm trying to get that ready to be opened up too...).
I can't believe I'm saying this, but I think "alternative buildroots" is not enough. It's a step in the right direction in terms of thinking about how to make developing bigger and more interesting changes. But we need to be thinking more holistically about packager and developer workflows within Fedora. Some of this is happening under the remit of the Fedora CI work, and that's awesome! But why are these things happening in haphazard pieces where we will likely wind up with more half-implemented solutions or things that people don't really like but stomach because some overlord is forcing it?
Incrementalism has its place, but this is something where once we implement the "just enough increment", we'll stop again and everything will just be uncomfortable and somewhat unhelpful, in much of the same way the pkgdb to pagure transition was handled. The reason why it'll happen is the same reason why it happened then: this stuff is legitimately difficult to improve and it's easy to pull the brakes once you reach a certain level of usefulness.
So... I guess I'm trying to say is "think bigger and go for broke", because I think we're really only going to have one chance to do this for the decade. Think like the people who made stuff like Koji and Dist-Git a reality over nearly 15 years ago, when *those* ideas were crazy and new! Just because the project is approaching being old enough to drink in the US doesn't mean we can't do these kinds of big things still!
Le jeudi 16 janvier 2020 à 22:24 -0500, Neal Gompa a écrit :
Hi Neal,
I've also said that I don't think we can handle it as our infrastructure currently stands. Our build system tooling has suffered from a decade of neglect, and attempts to reinvent or improve that have been rebuffed or failed in other ways.
…
there's an underlying general problem that it is okay to hobble along because we've hobbled along for so long.
I agree 100%, thanks for posting this, it’s not easy be the messenger for bad news.
Incrementalism has its place, but this is something where once we implement the "just enough increment", we'll stop again and everything will just be uncomfortable and somewhat unhelpful,
And here I disagree. Beware of the “let’s rewrite everything better” trap. If anything our incremental fixes failed more often than not because key players waited on big bang magical solutions (for example, modularity).
We need to implement more incremental fixes. Incremental does not mean risk free, it means *managed* risk. Project members need to accept incremental changes need to go in, and they need to accept incremental changes may break things or require some amount of project-wide work.
And, the Fedora leadership needs to do its leadership job. Leadership does not mean promoting fuzzy long term objectives that no one can disagree on because of the fuzziness, and that will explode on someone else’s turn. Leadership means decisions for today, not just tomorrow.
Big bang initiatives have their places, but only when incrementalism has hit a technical impasse. The Boeing 737 is an example where incrementalism hit its limits. But, the 70 years of some 737 components show how long incrementalism can be successful, as long as it it managed intelligently.
Regards,
On Fri, 17 Jan 2020 at 04:33, Nicolas Mailhot via devel devel@lists.fedoraproject.org wrote:
Le jeudi 16 janvier 2020 à 22:24 -0500, Neal Gompa a écrit :
Hi Neal,
I've also said that I don't think we can handle it as our infrastructure currently stands. Our build system tooling has suffered from a decade of neglect, and attempts to reinvent or improve that have been rebuffed or failed in other ways.
…
there's an underlying general problem that it is okay to hobble along because we've hobbled along for so long.
I agree 100%, thanks for posting this, it’s not easy be the messenger for bad news.
Incrementalism has its place, but this is something where once we implement the "just enough increment", we'll stop again and everything will just be uncomfortable and somewhat unhelpful,
And here I disagree. Beware of the “let’s rewrite everything better” trap. If anything our incremental fixes failed more often than not because key players waited on big bang magical solutions (for example, modularity).
We need to implement more incremental fixes. Incremental does not mean risk free, it means *managed* risk. Project members need to accept incremental changes need to go in, and they need to accept incremental changes may break things or require some amount of project-wide work.
And, the Fedora leadership needs to do its leadership job. Leadership does not mean promoting fuzzy long term objectives that no one can disagree on because of the fuzziness, and that will explode on someone else’s turn. Leadership means decisions for today, not just tomorrow.
Big bang initiatives have their places, but only when incrementalism has hit a technical impasse. The Boeing 737 is an example where incrementalism hit its limits. But, the 70 years of some 737 components show how long incrementalism can be successful, as long as it it managed intelligently.
My 'take' on the 737MAX problems is that they decided that incrementalism wasn't going 'fast' enough but they also didn't want to add a lot of training costs or 'change aversion' items. So they put in a lot of new things which would have made the plane a different model but then tried to sell it that they hadn't changed much so you didn't need to send your pilots to retrain over the fact that various switches do the opposite of what they did before.. Boeing then further 'cut' costs by removing a lot of places where a pilot team could over-ride the computers or have a backup piece in case the first failed.
All of these things could have still been done with incrementalism, but it would have probably taken 10 more years that they felt they didn't need but that they also didn't want to lose their current customer base by making a 7337 or a 797 plane which told everyone this plane is different. One problem with rewrite everything is where you try to keep your existing base because it is safe but also try to do something completely new. If you are going to do it.. be clear about it and realize that the people you had will have little interest in it because they are happy with what they have.