Kernel configurations for Fedora -- a proposal

[Laura Abbott]

Hi,

As a follow up to the previous discussion about kernel configuration
in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
I have a prototype of what a method of keeping each configuration
file in a separate file would look like. This method takes care of
several of my gripes of the current version (and found a few errors
in the existing config files). The biggest question I have is if
this will scale for how frequently Fedora adjusts configuration
options. Some of that could possibly be solved with more scripting
improvements.

The repo is at https://pagure.io/fedora-kernel-labbott/branch/split_configs

Thanks,
Laura

[Laura Abbott]

On 11/21/2016 10:46 AM, Laura Abbott wrote:
> Hi,
>
> As a follow up to the previous discussion about kernel configuration
> in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
> I have a prototype of what a method of keeping each configuration
> file in a separate file would look like. This method takes care of
> several of my gripes of the current version (and found a few errors
> in the existing config files). The biggest question I have is if
> this will scale for how frequently Fedora adjusts configuration
> options. Some of that could possibly be solved with more scripting
> improvements.
>
> The repo is at https://pagure.io/fedora-kernel-labbott/branch/split_configs
>
> Thanks,
> Laura

A reminder of this for those who may have missed it. I'm going to take
silence as no objection or opinion.

Thanks,
Laura

[Dan Horák]

On Wed, 30 Nov 2016 08:51:29 -0800
Laura Abbott  wrote:
> On 11/21/2016 10:46 AM, Laura Abbott wrote:
> > Hi,
> >
> > As a follow up to the previous discussion about kernel configuration
> > in Fedora
> > (https://lists.fedoraproject.org/archives/list/[e-mail]...
> > ) I have a prototype of what a method of keeping each configuration
> > file in a separate file would look like. This method takes care of
> > several of my gripes of the current version (and found a few errors
> > in the existing config files). The biggest question I have is if
> > this will scale for how frequently Fedora adjusts configuration
> > options. Some of that could possibly be solved with more scripting
> > improvements.
> >
> > The repo is at
> > https://pagure.io/fedora-kernel-labbott/branch/split_configs
> >
> > Thanks,
> > Laura
>
> A reminder of this for those who may have missed it. I'm going to take
> silence as no objection or opinion.

With my s390x hat on, my wish for kernel config is to have a way to
inherit the generic Fedora config, but still be able to build for
example only white-listed network drivers. Right now globally enabled
"PCI" enables also all drivers for all PCI-based devices (network,
storage, ...), but the s390x hardware has an internal whitelist of PCI
ids, so doesn't make sense to them all except the white-listed ones
(seen eg. in mainline default config). Is it feasible with the new
scheme?

Dan

[Laura Abbott]

On 11/30/2016 08:59 AM, Dan Horák wrote:
> On Wed, 30 Nov 2016 08:51:29 -0800
> Laura Abbott  wrote:
> > On 11/21/2016 10:46 AM, Laura Abbott wrote:
> > > Hi,
> > >
> > > As a follow up to the previous discussion about kernel configuration
> > > in Fedora
> > > (https://lists.fedoraproject.org/archives/list/[e-mail]...
> > > ) I have a prototype of what a method of keeping each configuration
> > > file in a separate file would look like. This method takes care of
> > > several of my gripes of the current version (and found a few errors
> > > in the existing config files). The biggest question I have is if
> > > this will scale for how frequently Fedora adjusts configuration
> > > options. Some of that could possibly be solved with more scripting
> > > improvements.
> > >
> > > The repo is at
> > > https://pagure.io/fedora-kernel-labbott/branch/split_configs
> > >
> > > Thanks,
> > > Laura
> >
> > A reminder of this for those who may have missed it. I'm going to take
> > silence as no objection or opinion.
>
> With my s390x hat on, my wish for kernel config is to have a way to
> inherit the generic Fedora config, but still be able to build for
> example only white-listed network drivers. Right now globally enabled
> "PCI" enables also all drivers for all PCI-based devices (network,
> storage, ...), but the s390x hardware has an internal whitelist of PCI
> ids, so doesn't make sense to them all except the white-listed ones
> (seen eg. in mainline default config). Is it feasible with the new
> scheme?

I don't think I completely understand your question. This should be
possible with the existing setup where config-s390x can set CONFIG_FOO=n
to turn off any drivers it doesn't want enabled. Is there something
more than that you want to happen?

Thanks,
Laura
> Dan

[Jarod Wilson]

On 2016-11-30 11:59 AM, Dan Horák wrote:
> On Wed, 30 Nov 2016 08:51:29 -0800
> Laura Abbott  wrote:
> > On 11/21/2016 10:46 AM, Laura Abbott wrote:
> > > Hi,
> > >
> > > As a follow up to the previous discussion about kernel configuration
> > > in Fedora
> > > (https://lists.fedoraproject.org/archives/list/[e-mail]...
> > > ) I have a prototype of what a method of keeping each configuration
> > > file in a separate file would look like. This method takes care of
> > > several of my gripes of the current version (and found a few errors
> > > in the existing config files). The biggest question I have is if
> > > this will scale for how frequently Fedora adjusts configuration
> > > options. Some of that could possibly be solved with more scripting
> > > improvements.
> > >
> > > The repo is at
> > > https://pagure.io/fedora-kernel-labbott/branch/split_configs
> > >
> > > Thanks,
> > > Laura
> >
> > A reminder of this for those who may have missed it. I'm going to take
> > silence as no objection or opinion.
>
> With my s390x hat on, my wish for kernel config is to have a way to
> inherit the generic Fedora config, but still be able to build for
> example only white-listed network drivers. Right now globally enabled
> "PCI" enables also all drivers for all PCI-based devices (network,
> storage, ...), but the s390x hardware has an internal whitelist of PCI
> ids, so doesn't make sense to them all except the white-listed ones
> (seen eg. in mainline default config). Is it feasible with the new
> scheme?

Yes, absolutely, but it's also doable with the current scheme, as Laura 
was suggesting.

[Peter Robinson]

On Wed, Nov 30, 2016 at 4:51 PM, Laura Abbott  wrote:
> On 11/21/2016 10:46 AM, Laura Abbott wrote:
> > Hi,
> >
> > As a follow up to the previous discussion about kernel configuration
> > in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
> > I have a prototype of what a method of keeping each configuration
> > file in a separate file would look like. This method takes care of
> > several of my gripes of the current version (and found a few errors
> > in the existing config files). The biggest question I have is if
> > this will scale for how frequently Fedora adjusts configuration
> > options. Some of that could possibly be solved with more scripting
> > improvements.
> >
> > The repo is at https://pagure.io/fedora-kernel-labbott/branch/split_configs
> >
> > Thanks,
> > Laura
>
> A reminder of this for those who may have missed it. I'm going to take
> silence as no objection or opinion.

My silence is neither but rather "not had time to read the thread and
digest it yet". Is the proposed change in a git repoo somewhere I
could checkout to see?

P

[Laura Abbott]

On 11/30/2016 05:54 PM, Peter Robinson wrote:
> On Wed, Nov 30, 2016 at 4:51 PM, Laura Abbott  wrote:
> > On 11/21/2016 10:46 AM, Laura Abbott wrote:
> > > Hi,
> > >
> > > As a follow up to the previous discussion about kernel configuration
> > > in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
> > > I have a prototype of what a method of keeping each configuration
> > > file in a separate file would look like. This method takes care of
> > > several of my gripes of the current version (and found a few errors
> > > in the existing config files). The biggest question I have is if
> > > this will scale for how frequently Fedora adjusts configuration
> > > options. Some of that could possibly be solved with more scripting
> > > improvements.
> > >
> > > The repo is at https://pagure.io/fedora-kernel-labbott/branch/split_configs
> > >
> > > Thanks,
> > > Laura
> >
> > A reminder of this for those who may have missed it. I'm going to take
> > silence as no objection or opinion.
>
> My silence is neither but rather "not had time to read the thread and
> digest it yet". Is the proposed change in a git repoo somewhere I
> could checkout to see?
>
> P

The repo is at https://pagure.io/fedora-kernel-labbott/branch/split_configs

[Thorsten Leemhuis]

Lo! On 21.11.2016 19:46, Laura Abbott wrote:
> As a follow up to the previous discussion about kernel configuration
> in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
> I have a prototype of what a method of keeping each configuration
> file in a separate file would look like. This method takes care of
> several of my gripes of the current version (and found a few errors
> in the existing config files). The biggest question I have is if
> this will scale for how frequently Fedora adjusts configuration
> options. Some of that could possibly be solved with more scripting
> improvements.
>
> The repo is at https://pagure.io/fedora-kernel-labbott/branch/split_configs

I took a brief look. Overall I thought "if it makes maintaining the
configs easier, go ahead, I don't care much". There are just a few
things I noticed while looking at it:

* nitpicking: I found the filenames "generate_(all|debug)_configs.sh"
misleading, because those scripts do not generate a config, as they
afaics just put a pre-generated config in place so they are going to be
used.

* I really like that this finally gets rids of the noise in the config
file diffs that the frequent enabling/disabling using "make
(debug|release)" creates currently.

* Just thinking aloud: I wonder if the pre-generated *debug.configs are
a good idea. Wouldn't it be more obvious what is happening if we'd ship
one base config (e.g. where debugging is turned off) and then have
something in the spec file itself that builds slightly modified version
depending on what is needed in the current build? Having a mechanism
like this might be handy for other situations as well. For example we
could have something in the spec file that automatically disables
"CONFIG_ARM64_VA_BITS_48" and "CONFIG_ARM64_VA_BITS" when building for
"$fedora_release <= F25"; this way we'd make sure things like that do
not accidental make it into older releases during a rebase.

HTH, CU, knurd

[Laura Abbott]

On 12/06/2016 04:44 AM, Thorsten Leemhuis wrote:
> Lo! On 21.11.2016 19:46, Laura Abbott wrote:
> > As a follow up to the previous discussion about kernel configuration
> > in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
> > I have a prototype of what a method of keeping each configuration
> > file in a separate file would look like. This method takes care of
> > several of my gripes of the current version (and found a few errors
> > in the existing config files). The biggest question I have is if
> > this will scale for how frequently Fedora adjusts configuration
> > options. Some of that could possibly be solved with more scripting
> > improvements.
> >
> > The repo is at https://pagure.io/fedora-kernel-labbott/branch/split_configs
>
> I took a brief look. Overall I thought "if it makes maintaining the
> configs easier, go ahead, I don't care much". There are just a few
> things I noticed while looking at it:
> nitpicking: I found the filenames "generate_(all|debug)_configs.sh"
>
> misleading, because those scripts do not generate a config, as they
> afaics just put a pre-generated config in place so they are going to be
> used.
> I really like that this finally gets rids of the noise in the config
>
> file diffs that the frequent enabling/disabling using "make
> (debug|release)" creates currently.
> Just thinking aloud: I wonder if the pre-generated *debug.configs are
>
> a good idea. Wouldn't it be more obvious what is happening if we'd ship
> one base config (e.g. where debugging is turned off) and then have
> something in the spec file itself that builds slightly modified version
> depending on what is needed in the current build? Having a mechanism
> like this might be handy for other situations as well. For example we
> could have something in the spec file that automatically disables
> "CONFIG_ARM64_VA_BITS_48" and "CONFIG_ARM64_VA_BITS" when building for
> "$fedora_release <= F25"; this way we'd make sure things like that do
> not accidental make it into older releases during a rebase.

I'm not sure how that would be more obvious. Generating the configs
makes it easier to check each file to see what's present vs not
being able to see what's enabled until it is built. I'm wary
to put anything more in the .spec file than we have to since the
file is complicated enough as is. Having the spec file enforce config
policy would be a step in the wrong direction.

Thanks,
Laura
> HTH, CU, knurd
> _______________________________________________
> kernel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Thorsten Leemhuis]

On 06.12.2016 19:46, Laura Abbott wrote:
> On 12/06/2016 04:44 AM, Thorsten Leemhuis wrote:
> > Lo! On 21.11.2016 19:46, Laura Abbott wrote:
> > > As a follow up to the previous discussion about kernel configuration
> > > in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
> > > I have a prototype of what a method of keeping each configuration
> > > file in a separate file would look like. This method takes care of
> > > several of my gripes of the current version (and found a few errors
> > > in the existing config files). The biggest question I have is if
> > > this will scale for how frequently Fedora adjusts configuration
> > > options. Some of that could possibly be solved with more scripting
> > > improvements.
>
> […]
> > Just thinking aloud: I wonder if the pre-generated *debug.configs are
> >
> > a good idea. Wouldn't it be more obvious what is happening if we'd ship
> > one base config (e.g. where debugging is turned off) and then have
> > something in the spec file itself that builds slightly modified version
> > depending on what is needed in the current build? Having a mechanism
> > like this might be handy for other situations as well. For example we
> > could have something in the spec file that automatically disables
> > "CONFIG_ARM64_VA_BITS_48" and "CONFIG_ARM64_VA_BITS" when building for
> > "$fedora_release <= F25"; this way we'd make sure things like that do
> > not accidental make it into older releases during a rebase.
>
> I'm not sure how that would be more obvious. Generating the configs
> makes it easier to check each file to see what's present vs not
> being able to see what's enabled until it is built. I'm wary
> to put anything more in the .spec file than we have to since the
> file is complicated enough as is.

Yeah, I can see that (albeit I found the
"generate_(all|debug)_configs.sh" scrits a bit confusing at first and
would prefer to see the commands directly in the spec file). But OTOH:
> Having the spec file enforce config
> policy would be a step in the wrong direction.

I tend to think: putting too much tricks into our SCM is the wrong
direction. Sure, it might make things easier to maintain for Fedora
developers. But it's not a fedpkg checkout advanced users start with
when they want to build a modified package -- instead it's typically the
SRPM they will use as base. That's why we have individual patches there
and that's why I think some logic is better suited there -- for example
when it comes to configurations options that are better disabled
(ideally automatically!) when you rebuild a rawhide kernel on the latest
or and earlier Fedora release.

The latter is something I do, so I obviously have a interest to make
that easy. Thus I wanted my voice to get heard in this discussion, which
I archived now, so I'm happy :-) IOW: it's not a big issue for me, I'm
fine with putting this subthread to a rest now.

Cu, knurd

[Laura Abbott]

On 12/07/2016 01:16 AM, Thorsten Leemhuis wrote:
> On 06.12.2016 19:46, Laura Abbott wrote:
> > On 12/06/2016 04:44 AM, Thorsten Leemhuis wrote:
> > > Lo! On 21.11.2016 19:46, Laura Abbott wrote:
> > > > As a follow up to the previous discussion about kernel configuration
> > > > in Fedora (https://lists.fedoraproject.org/archives/list/[e-mail]...)
> > > > I have a prototype of what a method of keeping each configuration
> > > > file in a separate file would look like. This method takes care of
> > > > several of my gripes of the current version (and found a few errors
> > > > in the existing config files). The biggest question I have is if
> > > > this will scale for how frequently Fedora adjusts configuration
> > > > options. Some of that could possibly be solved with more scripting
> > > > improvements.
> >
> > […]
> > > Just thinking aloud: I wonder if the pre-generated *debug.configs are
> > >
> > > a good idea. Wouldn't it be more obvious what is happening if we'd ship
> > > one base config (e.g. where debugging is turned off) and then have
> > > something in the spec file itself that builds slightly modified version
> > > depending on what is needed in the current build? Having a mechanism
> > > like this might be handy for other situations as well. For example we
> > > could have something in the spec file that automatically disables
> > > "CONFIG_ARM64_VA_BITS_48" and "CONFIG_ARM64_VA_BITS" when building for
> > > "$fedora_release <= F25"; this way we'd make sure things like that do
> > > not accidental make it into older releases during a rebase.
> >
> > I'm not sure how that would be more obvious. Generating the configs
> > makes it easier to check each file to see what's present vs not
> > being able to see what's enabled until it is built. I'm wary
> > to put anything more in the .spec file than we have to since the
> > file is complicated enough as is.
>
> Yeah, I can see that (albeit I found the
> "generate_(all|debug)_configs.sh" scrits a bit confusing at first and
> would prefer to see the commands directly in the spec file). But OTOH:

The amount of code in those scripts used to be much bigger so I can
look to pull those functions into the spec file.
> > Having the spec file enforce config
> > policy would be a step in the wrong direction.
>
> I tend to think: putting too much tricks into our SCM is the wrong
> direction. Sure, it might make things easier to maintain for Fedora
> developers. But it's not a fedpkg checkout advanced users start with
> when they want to build a modified package -- instead it's typically the
> SRPM they will use as base. That's why we have individual patches there
> and that's why I think some logic is better suited there -- for example
> when it comes to configurations options that are better disabled
> (ideally automatically!) when you rebuild a rawhide kernel on the latest
> or and earlier Fedora release.

Right but all the individual CONFIG_FOO=y files are not present in
the RPM. Only the fully generated kernel-$arch.config are present
in the SRPM. I updated the latest version to have config-local
(renamed to kernel-local) so options can be added there as well.

The decision of whether or not to use the debug config for rawhide
is already controlled via %define debugbuildsenabled. Is there more
logic you would like to see expanded apart from that?
> The latter is something I do, so I obviously have a interest to make
> that easy. Thus I wanted my voice to get heard in this discussion, which
> I archived now, so I'm happy :-) IOW: it's not a big issue for me, I'm
> fine with putting this subthread to a rest now.

Appreciated!
> Cu, knurd

Thanks,
Laura