Fedora 33 Self-Contained Change proposal: Distribute .repo files for modular repositories from a separate package

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/ModularReposSubpackage

== Summary ==
Reintroduce the fedora-repos-modular package. Have the
<code>/etc/yum.repos.d/*-modular.repo</code> files in it instead of
fedora-repos.
We install/keep fedora-repos-modular by default, users (admins) can
uninstall it if desired. No defaults are changed.

== Owner ==
* Name: [[User:Churchyard|Miro Hrončok]]
* Email: mhroncok(a)redhat.com


== Detailed Description ==
As a Fedora user, who doesn't consume any modules, I'd like an easy
way to disable modular repos to save some traffic, disk space and
time.

Currently, I can do it by editing all
<code>/etc/yum.repos.d/*-modular.repo</code> files and changing:

 enabled=1

To:

 enabled=0

This has downsides: When the files are changed in next update, I won't
get them updated, because they are shipped as
<code>%config(noreplace)</code>. (If they were not shipped as
<code>%config(noreplace)</code>, it would be even worse, as my changes
would be overridden.) It's also multiple files. I can make mistakes
and break other files by accident.

In order to not to have to resort to manually editing RPM-package
shipped configuration, I propose to have a better way of disabling
modular repos, namely via: <code>sudo dnf remove
fedora-repos-modular</code>.

In this change, we move modular repos into a separate package
(subpackage of fedora-repos).

Basically revert this plus some extra comps/kickstarts changes:

https://src.fedoraproject.org/rpms/fedora-repos/c/7b32bee388d093c446017f1...

Current fedora-repos Pull Request:

https://src.fedoraproject.org/rpms/fedora-repos/pull-request/62


== Feedback ==
This was proposed in early May 2020 on the devel mailing list:

https://lists.fedoraproject.org/archives/list/[e-mail].o...

The proposal received only positive feedback.

This was then opened as a pull request in:

https://src.fedoraproject.org/rpms/fedora-repos/pull-request/62

Where it received thorough review and positive feedback, until it was
blocked by an unrelated modularity discussion that slightly touched
this topic. FESCo sentiment was positive or neutral (however, one
FESCo member doubted the usefulness of this, saying it is pointless).

https://src.fedoraproject.org/rpms/fedora-repos/pull-request/62#comment-4...

https://pagure.io/fesco/issue/2114#comment-654115 and below

Later, the spirit of this change was approved by FESCo in
https://pagure.io/fesco/issue/2406

== Why don't we...? ==

=== Why don't we have config overrides for system provided repo files? ===

It would be great if DNF supported system-repos in
<code>/usr/share</code> and override options in <code>/etc</code>,
but
that is not (yet) the case. Feel free to work on this feature, but the
change owner won't.

=== Why don't we disable modular repos by default? ===

This would require much bigger and heated discussion. Let it happen
elsewhere. This self-contained change proposal maintains the status
quo and only creates a way for users to opt out form the modular
repos, when they so desire. It does not disable anything.

=== Why are you against ...? ===

This change is not against modularity, it is not against default
modular streams, it is not against the people who created modularity,
it is not against ELN, it is not against you. This change proposal
maintains the status quo by default and only creates a way for users
to opt out form the modular repos, when they so desire. The
maintenance burden of having one extra repo package is considered
insignificant compared to the benefit. This is not a death of a
thousands cuts proposal with a hidden agenda, this is a proposal that
adds a significant benefit.

== Benefit to Fedora ==
Users who don't consume modules have the ability to disable modular
repositories in a safe and supported way. This can save their traffic,
disk space and time.

Container maintainers can benefit form this change as well by
disabling modular repos first by a single command or not having it at
all from start, if they so desire.

Package maintainers can benefit from this change by not using modular
repos when repoquerying non-modular repos, if they so desire.

Mirror operators can benefit from this change by saving their traffic
if the users actually do disable the repos, while there is no downside
if they don't.

== Scope ==
* Proposal owners:
* Finish and merge https://pagure.io/fesco/issue/2114
* Ensure fedora-repos-modular are installed by default on Fedora 33
(fresh installs)
* Ensure fedora-repos-modular are installed by default on Fedora 33
(upgrades from previous releases)

* Other developers: no action required
* Release engineering: no action required (possibly building the
updated fedora-repos package if provenpackagers can't)
* Policies and guidelines: nada
* Trademark approval: zip


== Upgrade/compatibility impact ==
It is expected that users who upgrade from previous Fedora releases
will have fedora-repos-modular installed during the upgrade to Fedora
33 or 34.
It is expected that rawhide users who will have fedora-repos-modular
installed during regular packages upgrade.

If this is not the case, it should be treated like a bug.

== How To Test ==
Install Fedora 33 (any edition or flavor), check that modular repos
are still installed and enabled by default.

Update to Fedora 33 (from Fedora 31 or 32), check that modular repos
are still installed and enabled by default.

Check that you can remove the fedora-repos-modular package (e.g. via
<code>sudo dnf remove fedora-repos-modular</code>) and it removes the
modular repos.

Check that you can install the fedora-repos-modular package again
(e.g. via <code>sudo dnf install fedora-repos-modular</code>) and it
adds the modular repos, enabled.

== User Experience ==
Users can disable or enable modular repos by removing or installing
fedora-repos-modular. This is significantly better UX than editing
config files.

== Dependencies ==
No dependencies.

== Contingency Plan ==
* Contingency mechanism: If the outlined constraints are seriously
broken and beyond fixing, QA can decide to revert the change.
* Contingency deadline: Beta Freeze
* Blocks release? No
* Blocks product? No

== Documentation ==
Documentation shall be added about disabling/enabling the modular repos.




-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis

[Miroslav Suchý]

Dne 15. 06. 20 v 22:10 Ben Cotton napsal(a):
>  We install/keep fedora-repos-modular by default, users (admins) can
>  uninstall it if desired. No defaults are changed 
How you manage to have it installed by default? It is not obvious from the linked PR to
me.
I just want to be sure it will be installed when @buildsys-build group is installed.
Otherwise some mock builds may fail.

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

[Miro Hrončok]

On 16. 06. 20 1:31, Miroslav Suchý wrote:
>  Dne 15. 06. 20 v 22:10 Ben Cotton napsal(a):
> > We install/keep fedora-repos-modular by default, users (admins) can
> > uninstall it if desired. No defaults are changed
>  
>  How you manage to have it installed by default? It is not obvious from the linked PR to
> me. 
I don't know yet. There are 3 possible ways I was abele to think of:

  - kickstarts
  - comps
  - compose configuration?

Originally, I was thinking I'll just add it to the same kickstart/comps as 
fedora-repos, unfortunately, I cannot see fedora-repos in either, so I will have 
to figure out trough what does the fedora-repos go in (I guess it is 
fedora-release-common).

I think we can add it to fedora-repo.ks or @core but I will yet to consult this 
with releng.

>  I just want to be sure it will be installed when @buildsys-build
> group is installed. Otherwise some mock builds may fail. 
What kind of mock builds need Fedora modular repos? Could you elaborate? I 
thought mock uses repos defined in mock roots configuration.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Panu Matilainen]

On 6/16/20 3:15 AM, Miro Hrončok wrote:
>  On 16. 06. 20 1:31, Miroslav Suchý wrote:
> > Dne 15. 06. 20 v 22:10 Ben Cotton napsal(a):
> >> We install/keep fedora-repos-modular by default, users (admins) can
> >> uninstall it if desired. No defaults are changed
> >
> > How you manage to have it installed by default? It is not obvious from 
> > the linked PR to me.
>  
>  I don't know yet. There are 3 possible ways I was abele to think of:
>  
>    - kickstarts
>    - comps
>    - compose configuration?
>  
>  Originally, I was thinking I'll just add it to the same kickstart/comps 
>  as fedora-repos, unfortunately, I cannot see fedora-repos in either, so 
>  I will have to figure out trough what does the fedora-repos go in (I 
>  guess it is fedora-release-common). 
Yeah it's a hard dependency of fedora-release-common. I suppose one 
possibility would be adding a recommends on fedora-repos-modular to 
fedora-release-common, but weak dependencies have an annoying tendency 
to creep back in when you're not looking, a kickstart/comps defaults are 
nicer in that regard.

Big +1 for the feature though.

	- Panu -

[Miro Hrončok]

On 16. 06. 20 10:03, Panu Matilainen wrote:
>  Yeah it's a hard dependency of fedora-release-common. I suppose
> one possibility 
>  would be adding a recommends on fedora-repos-modular to fedora-release-common, 
>  but weak dependencies have an annoying tendency to creep back in when you're not 
>  looking, a kickstart/comps defaults are nicer in that regard. 
I was originally thinking that fedora-repos should recommend 
fedora-repos-modular, but due to the annoying nature of getting recommends back 
on every upgrade of the related package I abandoned the idea.

Relevant dnf bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1699672

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Vít Ondruch]

Dne 16. 06. 20 v 11:04 Miro Hrončok napsal(a):
>  On 16. 06. 20 10:03, Panu Matilainen wrote:
> > Yeah it's a hard dependency of fedora-release-common. I suppose one
> > possibility would be adding a recommends on fedora-repos-modular to
> > fedora-release-common, but weak dependencies have an annoying
> > tendency to creep back in when you're not looking, a kickstart/comps
> > defaults are nicer in that regard.
>
>  I was originally thinking that fedora-repos should recommend
>  fedora-repos-modular, but due to the annoying nature of getting
>  recommends back on every upgrade of the related package I abandoned
>  the idea.
>
>  Relevant dnf bugzilla:
>  https://bugzilla.redhat.com/show_bug.cgi?id=1699672
>  
Not mentioned that weak dependencies are disabled in Mock.


Vít

[Tomas Orsava]

On 6/15/20 10:10 PM, Ben Cotton wrote:
>  https://fedoraproject.org/wiki/Changes/ModularReposSubpackage
>
>  == Summary ==
>  Reintroduce the fedora-repos-modular package. Have the
>  <code>/etc/yum.repos.d/*-modular.repo</code> files in it instead of
>  fedora-repos.
>  We install/keep fedora-repos-modular by default, users (admins) can
>  uninstall it if desired. No defaults are changed. 

+1 I would love to see this feature for my personal use.

[James Cassell]

On Tue, Jun 16, 2020, at 5:57 AM, Vít Ondruch wrote:
>  
>  Dne 16. 06. 20 v 11:04 Miro Hrončok napsal(a):
>  > On 16. 06. 20 10:03, Panu Matilainen wrote:
>  >> Yeah it's a hard dependency of fedora-release-common. I suppose one
>  >> possibility would be adding a recommends on fedora-repos-modular to
>  >> fedora-release-common, but weak dependencies have an annoying
>  >> tendency to creep back in when you're not looking, a kickstart/comps
>  >> defaults are nicer in that regard.
>  >
>  > I was originally thinking that fedora-repos should recommend
>  > fedora-repos-modular, but due to the annoying nature of getting
>  > recommends back on every upgrade of the related package I abandoned
>  > the idea.
>  >
>  > Relevant dnf bugzilla:
>  > https://bugzilla.redhat.com/show_bug.cgi?id=1699672
>  >
>  
>  Not mentioned that weak dependencies are disabled in Mock.
>   
I can't decide whether that's a bug or a feature...


V/r,
James Cassell

[Miro Hrončok]

On 16. 06. 20 11:57, Vít Ondruch wrote:
>  Not mentioned that weak dependencies are disabled in Mock. 
I don't understand why would the user need fedora-repos-modular automatically 
pulled into mock when they install fedora-repos there. Could you please elaborate?

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Dridi Boukelmoune]

>  == How To Test ==
>  Install Fedora 33 (any edition or flavor), check that modular repos
>  are still installed and enabled by default.
>
>  Update to Fedora 33 (from Fedora 31 or 32), check that modular repos
>  are still installed and enabled by default.
>
>  Check that you can remove the fedora-repos-modular package (e.g. via
>  <code>sudo dnf remove fedora-repos-modular</code>) and it removes the
>  modular repos.
>
>  Check that you can install the fedora-repos-modular package again
>  (e.g. via <code>sudo dnf install fedora-repos-modular</code>) and it
>  adds the modular repos, enabled. 
If someone manually disables the modular repositories prior to this
change, uninstalling and reinstalling the package wouldn't make any
difference because the files are %config(noreplace) and would be left
alone because of the modification.

Unless you had plans to deal with that too.

>  == User Experience ==
>  Users can disable or enable modular repos by removing or installing
>  fedora-repos-modular. This is significantly better UX than editing
>  config files. 
Agreed, I hope this change will follow through.

Dridi

[Miro Hrončok]

On 17. 06. 20 12:01, Dridi Boukelmoune wrote:
> > == How To Test ==
> > Install Fedora 33 (any edition or flavor), check that modular repos
> > are still installed and enabled by default.
> >
> > Update to Fedora 33 (from Fedora 31 or 32), check that modular repos
> > are still installed and enabled by default.
> >
> > Check that you can remove the fedora-repos-modular package (e.g. via
> > <code>sudo dnf remove fedora-repos-modular</code>) and it removes the
> > modular repos.
> >
> > Check that you can install the fedora-repos-modular package again
> > (e.g. via <code>sudo dnf install fedora-repos-modular</code>) and it
> > adds the modular repos, enabled.
>  If someone manually disables the modular repositories prior to this
>  change, uninstalling and reinstalling the package wouldn't make any
>  difference because the files are %config(noreplace) and would be left
>  alone because of the modification.
>  
>  Unless you had plans to deal with that too. 
I don't, I'll adapt the how to test section.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Vít Ondruch]

Dne 17. 06. 20 v 0:21 Miro Hrončok napsal(a):
>  On 16. 06. 20 11:57, Vít Ondruch wrote:
> > Not mentioned that weak dependencies are disabled in Mock.
>
>  I don't understand why would the user need fedora-repos-modular
>  automatically pulled into mock when they install fedora-repos there.
>  Could you please elaborate?
>  
Quoting the original email which started this sub-thread:


>  Dne 15. 06. 20 v 22:10 Ben Cotton napsal(a):
> > We install/keep fedora-repos-modular by default, users (admins) can
> > uninstall it if desired. No defaults are changed
>
>  How you manage to have it installed by default? It is not obvious from 
the linked
PR to me.
>  I just want to be sure it will be installed when @buildsys-build
> group 
is installed. Otherwise some mock builds may fail.
>  --
>  Miroslav Suchy, RHCA
>  Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys 


Vít

[Miro Hrončok]

On 17. 06. 20 14:31, Vít Ondruch wrote:
>  Quoting the original email which started this sub-thread:
>  
>  
> > Dne 15. 06. 20 v 22:10 Ben Cotton napsal(a):
> >> We install/keep fedora-repos-modular by default, users (admins) can
> >> uninstall it if desired. No defaults are changed
> > How you manage to have it installed by default? It is not obvious from
>  the linked PR to me.
> > I just want to be sure it will be installed when @buildsys-build group
>  is installed. Otherwise some mock builds may fail. 
However, my question in the original email remains unanswered.

What kind of mock builds need Fedora modular repos inside the mock root?
I thought mock uses repos defined in mock roots configuration.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Miroslav Suchý]

Dne 17. 06. 20 v 15:01 Miro Hrončok napsal(a):
>  However, my question in the original email remains unanswered.
>  
>  What kind of mock builds need Fedora modular repos inside the mock root?
>  I thought mock uses repos defined in mock roots configuration. 
Miro is right, and I raised it incorrectly.
Mock will use the repos from config.

I actually think about it since this discussion, that we may remove:
  * fedora-repos
  * fedora-gpg-keys
from @buildsys-build group as they are not needed for a build. But they are required by
fedora-release-common. :(

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys