F35 Change: Third-party Software Mechanism (System-Wide Change proposal)

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/Third_Party_Software_Mechanism

== Summary ==
Update mechanism for opting-in to "Third-Party Software Repositories"
so that the repositories are immediately enabled.

== Owner ==
* Name: [[User:otaylor| Owen Taylor]]
* Email: otaylor(a)redhat.com


== Detailed Description ==
''''''Note that this proposal is about a change to how
third-party
repositories are enabled, not about including anything new in
Fedora.'''''

Currently, when the user opts in to "Enable Third-Party Software
repositories", the` fedora-workstation-repositories` package is
installed, but with the repositories disabled.
With this change, `fedora-workstation-repositories` will be installed
by default (required by `fedora-release-workstation`), and opting in
to "Third-party Software Repositories" will actually enable the
repositories.

Fedora Workstation Issue:
[https://pagure.io/fedora-workstation/issue/105 #105 Ship
fedora-workstation-repositories on install media]

== Conformance to Fedora policies ==

[https://docs.fedoraproject.org/en-US/fesco/Third_Party_Repository_Policy/
Third-Party Repository Policy]:

<blockquote>
The third-party nature of the repository must be apparent to the user
when they enable it, as should the non-free status of its content, if
such. To ensure this, repository files must initially include the
enabled=0 (or equivalent) setting, and the user must explicitly enable
third-party repositories to install from them.
</blockquote>

This proposals is a new implementation of "explictly enable
third-party repositories". There is no proposed change to which
third-party repostories are shipped - and in particular this change
does not include splitting fedora-workstation-repositories to conform
to the recommendation of the current guidelines.

== Technical implementation ==

* There is a <code>fedora-third-party</code> package with a
<code>fedora-third-party</code> script with
<code>enable/disable/refresh/query</code> subcommands. The status is
stored in <code>/etc/fedora-third-party.conf</code>
* Packages like <code>fedora-workstation-repositories</code> that
include third-party repositories will drop config files into
<code>/etc/fedora-third-party.d/*.conf</code>. There will be a
post-transaction file trigger to run <code>fedora-third-party
refresh</code>, which applies the users opt-in status to newly
installed repository files.
* We add a new page to GNOME Initial Setup that asks a single
question, *along the lines of*:<br>
'''Enable Third Party Software repositories?''' <br>
☑ Access additional software from selected third party sources. Some
of this software is proprietary and therefore has restrictions on use,
sharing, and access to source code. <br>
[Find out
more...](https://fedoraproject.org/wiki/Workstation/Third_Party_Software_...
* If the user leaves the box checked, GNOME Initial setup runs
`fedora-third-party enable`.
* For upgrades, GNOME Software shows an info-bar with the same
question if no status is stored in `/etc/fedora-thirdparty.conf`

== Feedback ==
<!-- Summarize the feedback from the community and address why you
chose not to accept proposed alternatives. This section is optional
for all change proposals but is strongly suggested. Incorporating
feedback here as it is raised gives FESCo a clearer view of your
proposal and leaves a good record for the future. If you get no
feedback, that is useful to note in this section as well. For
innovative or possibly controversial ideas, consider collecting
feedback before you file the change proposal. -->

== Benefit to Fedora ==

The main benefit of this proposal is the removal of the state where
the user has opted in to third party repositories but they are not
actually enabled. PackageKit supports the
<code>enabled_metadata=1</code> key in a repository file, which allows
applications to be searched in this state, but this is not supported
by DNF.

The new method is also easily extensible to Flatpaks, where there also
no equivalent to <code>enabled_metadata=1</code>, even in GNOME
Software.

== Scope ==
* Proposal owners: Create and test proposed
<code>fedora-third-party</code> package. Implement the graphical
controls for this in GNOME Software and gnome-initial-setup.
* Release engineering: [https://pagure.io/releng/issue/10186 #10186]
No changes are required.
* Policies and guidelines: Third-party Software guidelines will need
minor changes to remove references to `enabled_metadata=1`. Pending
finalization of technical implementation.
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: No real alignment

== Upgrade/compatibility impact ==
Because the "opt-in" status to 3rd party software is currently
represented by whether fedora-workstation-repositories is installed,
and because fedora-workstation-repositories will become an
installed-by-default package, users will need to opt-in again.

They can do this either by responding in the infobar that will be
displayed in GNOME Software, or by running <code>fedora-third-party
enable</code> on the command line.

== How To Test ==
* A fresh install of Fedora Workstation where the user ''does not''
opt-in should have all repositories disabled.
* A fresh install of Fedora Workstation where the user ''does'' opt-in
should have all 3rd-party repositories enabled.
* On an upgrade from F34, if the user opts-out, the enablement status
of third-party repositories should be ''unchanged'' (try enabling one
before the upgrade)
* On an upgrade from F35, if the user opts-in, all 3rd party
repositories should be enabled.

== User Experience ==
The user will get less confusing behavior around third-party
repositories - enabled will mean enabled and will take affect no
matter how they are installing packages.

See https://hackmd.io/@owtaylor/fedora-third-party-repos for a
detailed description of the *current* experience along with some notes
about the desired behavior.

== Dependencies ==
The changes are limited to the following packages:

* The new `fedora-third-party` package
* `fedora-workstation-repositories`
* `gnome-software`
* `gnome-initial-setup`
* `fedora-release-workstation` and other release packages that will
now require fedora-workstation-repositories.

This change proposal is a prerequisite for a separate change proposal
to add a filtered view of Flathub to the set of third-party
repositories.

== Contingency Plan ==
* Contingency mechanism: revert all changes back to the F34 state.
(This will also require reverting the filtered-view-of-Flathub
change.)
* Contingency deadline: beta freeze
* Blocks release? Yes - this needs to be finished or reverted

== Documentation ==
'''This should be a link to a man page for the `fedora-third-party`
tool'''

== Release Notes ==
Fedora optionally provides repository definitions allowing users to
install certain third-party software. This used to be done as a
two-step process where when the user asked to enable third-party
repositories, the repository definitions were installed but not
actually enabled, and they had to be separately enabled. With Fedora
35, this is simplified so that the repository definitions are
installed by default, but only enabled if the user opts in.

If you are upgrading from an older version of Fedora, you'll need to
opt-in again - this can be done by running GNOME Software and
accepting the prompt that is shown on the initial page. Alternatively,
you can run <code>fedora-third-party enable</code> from the command
line. If you do not wish to enable third-party repositories, no action
is needed.


-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis

[Lars Seipel]

On Tue, Jun 29, 2021 at 04:24:58PM -0400, Ben Cotton wrote:
> * We add a new page to GNOME Initial Setup that asks a single
> question, *along the lines of*:<br>
> '''Enable Third Party Software repositories?''' <br>
> ☑ Access additional software from selected third party sources. Some
> of this software is proprietary and therefore has restrictions on use,
> sharing, and access to source code. <br>
> [Find out
> more...](https://fedoraproject.org/wiki/Workstation/Third_Party_Software_...
> * If the user leaves the box checked, GNOME Initial setup runs
> `fedora-third-party enable`. 
Leaves the box checked? Either I'm misunderstanding or this contradicts
the remaining text which talks about "opt-in".

[Owen Taylor]

On Wed, Jun 30, 2021 at 6:50 AM Lars Seipel <ls(a)slrz.net&gt; wrote:
>  >[Find out
> more...](https://fedoraproject.org/wiki/Workstation/Third_Party_Software_...
>  >* If the user leaves the box checked, GNOME Initial setup runs
>  >`fedora-third-party enable`.
>
>  Leaves the box checked? Either I'm misunderstanding or this contradicts
>  the remaining text which talks about "opt-in". 
Thanks for noticing the discrepancy!

The current mockup for how the page in GNOME Initial Setup should look
is at the top of:

 https://hackmd.io/@owtaylor/fedora-third-party-repos

and is in-fact a classic opt-in. (*)

The mockup tries to reduce the risk of an inexperienced user being
confused by terms like "third party" and "repository" by mentioning
specific software.I'll look at embedding that in the change proposal
and will make sure that the text is consistent.

- Owen

(*) The text of the Third-Party Repository policy is "the user must
explicitly enable third-party repositories to install from them" - I
don't think a checked-by-default box would be in agreement with that.

[Miroslav Suchý]

Dne 29. 06. 21 v 22:24 Ben Cotton napsal(a):
>  ''''''Note that this proposal is about a
> change to how third-party
>  repositories are enabled, not about including anything new in
>  Fedora.''''' 
I completely missed the "Third party repos" thing. I had to google it. To save
you some time:

   https://fedoramagazine.org/third-party-repositories-fedora/

Miroslav

[Zbigniew Jędrzejewski-Szmek]

>  == Technical implementation ==
>  
>  * There is a <code>fedora-third-party</code> package with a
>  <code>fedora-third-party</code> script with
>  <code>enable/disable/refresh/query</code> subcommands. The status is
>  stored in <code>/etc/fedora-third-party.conf</code>
>  * Packages like <code>fedora-workstation-repositories</code> that
>  include third-party repositories will drop config files into
>  <code>/etc/fedora-third-party.d/*.conf</code>. There will be a
>  post-transaction file trigger to run <code>fedora-third-party
>  refresh</code>, which applies the users opt-in status to newly
>  installed repository files. 
For packaged stuff, please do:

  s|/etc/fedora-third-party.conf|/usr/lib/fedora-third-party.conf|

We shouldn't add yet more stuff in /etc/.

Zbyszek



> * We add a new page to GNOME Initial Setup that asks a single
> question, *along the lines of*:<br>
> '''Enable Third Party Software repositories?''' <br>
> ☑ Access additional software from selected third party sources. Some
> of this software is proprietary and therefore has restrictions on use,
> sharing, and access to source code. <br>
> [Find out
more...](https://fedoraproject.org/wiki/Workstation/Third_Party_Software_...
> * If the user leaves the box checked, GNOME Initial setup runs
> `fedora-third-party enable`.
> * For upgrades, GNOME Software shows an info-bar with the same
> question if no status is stored in `/etc/fedora-thirdparty.conf`
> 
> == Feedback ==
> <!-- Summarize the feedback from the community and address why you
> chose not to accept proposed alternatives. This section is optional
> for all change proposals but is strongly suggested. Incorporating
> feedback here as it is raised gives FESCo a clearer view of your
> proposal and leaves a good record for the future. If you get no
> feedback, that is useful to note in this section as well. For
> innovative or possibly controversial ideas, consider collecting
> feedback before you file the change proposal. -->
> 
> == Benefit to Fedora ==
> 
> The main benefit of this proposal is the removal of the state where
> the user has opted in to third party repositories but they are not
> actually enabled. PackageKit supports the
> <code>enabled_metadata=1</code> key in a repository file, which allows
> applications to be searched in this state, but this is not supported
> by DNF.
> 
> The new method is also easily extensible to Flatpaks, where there also
> no equivalent to <code>enabled_metadata=1</code>, even in GNOME
> Software.
> 
> == Scope ==
> * Proposal owners: Create and test proposed
> <code>fedora-third-party</code> package. Implement the graphical
> controls for this in GNOME Software and gnome-initial-setup.
> * Release engineering: [https://pagure.io/releng/issue/10186 #10186]
> No changes are required.
> * Policies and guidelines: Third-party Software guidelines will need
> minor changes to remove references to `enabled_metadata=1`. Pending
> finalization of technical implementation.
> * Trademark approval: N/A (not needed for this Change)
> * Alignment with Objectives: No real alignment
> 
> == Upgrade/compatibility impact ==
> Because the "opt-in" status to 3rd party software is currently
> represented by whether fedora-workstation-repositories is installed,
> and because fedora-workstation-repositories will become an
> installed-by-default package, users will need to opt-in again.
> 
> They can do this either by responding in the infobar that will be
> displayed in GNOME Software, or by running <code>fedora-third-party
> enable</code> on the command line.
> 
> == How To Test ==
> * A fresh install of Fedora Workstation where the user ''does not''
> opt-in should have all repositories disabled.
> * A fresh install of Fedora Workstation where the user ''does''
opt-in
> should have all 3rd-party repositories enabled.
> * On an upgrade from F34, if the user opts-out, the enablement status
> of third-party repositories should be ''unchanged'' (try enabling
one
> before the upgrade)
> * On an upgrade from F35, if the user opts-in, all 3rd party
> repositories should be enabled.
> 
> == User Experience ==
> The user will get less confusing behavior around third-party
> repositories - enabled will mean enabled and will take affect no
> matter how they are installing packages.
> 
> See https://hackmd.io/@owtaylor/fedora-third-party-repos for a
> detailed description of the *current* experience along with some notes
> about the desired behavior.
> 
> == Dependencies ==
> The changes are limited to the following packages:
> 
> * The new `fedora-third-party` package
> * `fedora-workstation-repositories`
> * `gnome-software`
> * `gnome-initial-setup`
> * `fedora-release-workstation` and other release packages that will
> now require fedora-workstation-repositories.
> 
> This change proposal is a prerequisite for a separate change proposal
> to add a filtered view of Flathub to the set of third-party
> repositories.
> 
> == Contingency Plan ==
> * Contingency mechanism: revert all changes back to the F34 state.
> (This will also require reverting the filtered-view-of-Flathub
> change.)
> * Contingency deadline: beta freeze
> * Blocks release? Yes - this needs to be finished or reverted
> 
> == Documentation ==
> '''This should be a link to a man page for the `fedora-third-party`
tool'''
> 
> == Release Notes ==
> Fedora optionally provides repository definitions allowing users to
> install certain third-party software. This used to be done as a
> two-step process where when the user asked to enable third-party
> repositories, the repository definitions were installed but not
> actually enabled, and they had to be separately enabled. With Fedora
> 35, this is simplified so that the repository definitions are
> installed by default, but only enabled if the user opts in.
> 
> If you are upgrading from an older version of Fedora, you'll need to
> opt-in again - this can be done by running GNOME Software and
> accepting the prompt that is shown on the initial page. Alternatively,
> you can run <code>fedora-third-party enable</code> from the command
> line. If you do not wish to enable third-party repositories, no action
> is needed.
> 
> 
> -- 
> Ben Cotton
> He / Him / His
> Fedora Program Manager
> Red Hat
> TZ=America/Indiana/Indianapolis
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Miroslav Suchý]

Dne 30. 06. 21 v 13:52 Zbigniew Jędrzejewski-Szmek napsal(a):
>  For packaged stuff, please do:
>
>     s|/etc/fedora-third-party.conf|/usr/lib/fedora-third-party.conf|
>
>  We shouldn't add yet more stuff in/etc/. 
I guess that /usr/lib is typo and you meant /var/lib/

Miroslav

[Zbigniew Jędrzejewski-Szmek]

On Wed, Jun 30, 2021 at 02:09:19PM +0200, Miroslav Suchý wrote:
>  Dne 30. 06. 21 v 13:52 Zbigniew Jędrzejewski-Szmek napsal(a):
>  >For packaged stuff, please do:
>  >
>  >   s|/etc/fedora-third-party.conf|/usr/lib/fedora-third-party.conf|
>  >
>  >We shouldn't add yet more stuff in/etc/.
>  
>  I guess that /usr/lib is typo and you meant /var/lib/ 
No, /usr/lib.

This was in response to:
>  Packages like fedora-workstation-repositories that include
>  third-party repositories will drop config files into
>  /etc/fedora-third-party.d/*.conf 
Those config files should be in /usr/lib/fedora-third-party.d/.

>  There is a fedora-third-party package with a fedora-third-party
>  script with enable/disable/refresh/query subcommands. The status is
>  stored in /etc/fedora-third-party.conf 
That part is fine. When the user does a manual configuration step,
this should be stored in /etc/.

(Essentially, the usual systemd-style config file split.)

Zbyszek

[Owen Taylor]

On Wed, Jun 30, 2021 at 7:53 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:

>  > * There is a <code>fedora-third-party</code> package
> with a
>  > <code>fedora-third-party</code> script with
>  > <code>enable/disable/refresh/query</code> subcommands. The status is
>  > stored in <code>/etc/fedora-third-party.conf</code>
>  > * Packages like <code>fedora-workstation-repositories</code> that
>  > include third-party repositories will drop config files into
>  > <code>/etc/fedora-third-party.d/*.conf</code>. There will be a
>  > post-transaction file trigger to run <code>fedora-third-party
>  > refresh</code>, which applies the users opt-in status to newly
>  > installed repository files.
>
>  For packaged stuff, please do:
>
>    s|/etc/fedora-third-party.conf|/usr/lib/fedora-third-party.conf|
>
>  We shouldn't add yet more stuff in /etc/. 
Yes, agreed. Thanks for the suggestion!

Owen

[Mark Otaris]

I don’t agree with this change, as it seems obvious that many users who do not want
proprietary software installed do not want repositories with proprietary software in them
installed either (whether or not these repositories are enabled) and would want to have to
opt-in to that too. Additionally, disabled repositories show up from time to time, bugs
allowing, so they are not as inactive as the name implies. Not having them installed
serves as a safeguard.

Also, users who use some software from the third-party repositories likely *do not* want
all of them enabled. The stated benefit of the proposed change (“the removal of the state
where the user has opted in to third party repositories but they are not actually
enabled”) is not the benefit; the actual benefit that is wanted is an improvement to the
user experience that is made easier to provide by installing the repos by default.

That said, the problems here are more on Council than FESCo, and Council already approved
this, and people don’t agree with me.

[Owen Taylor]

On Sat, Jul 3, 2021 at 4:39 PM Mark Otaris <mark(a)net-c.com&gt; wrote:
>  I don’t agree with this change, as it seems obvious that many users who do not want
> proprietary software installed do not want repositories with proprietary software in them
> installed either (whether or not these repositories are enabled) and would want to have to
> opt-in to that too. Additionally, disabled repositories show up from time to time, bugs
> allowing, so they are not as inactive as the name implies. Not having them installed
> serves as a safeguard. 
Hi Mark, thanks for the feedback!

While there can be bugs of any type, I don't see any reason to
*expect* a bug that would make dnf or PackageKit show search results
or listings from disabled repositories.Wouldn't that be noticed and
fixed quickly?

Is the number of users who don't even want a text file mentioning the
existence of third-party software on their system significant? If
FESCO feels that people wanting to remove the package entirely is a
concern, we could soften the proposed fedora-release-workstation
Requires: on fedora-workstation-repositories to a Recommends, to allow
removing the package. I'd personally prefer to leave the Requires to
prevent silent breakage.

>  Also, users who use some software from the third-party repositories
> likely *do not* want all of them enabled 
It will be possible to opt-out and then enable individual
repositories, or opt-in and disable individual repositories. What it
doesn't provide is protection against a repository accidentally or
intentionally offering an upgrade to a system package. I think this
was one of the things that the disabled=1, disabled_metadata=0 system
was meant to make better, but the dnf mainainers have explicitly
expressed disinterested in implementing this in the past. As a
PackageKit-only thing, we're providing users a fragmented and strange
user experience. And there's even plans to eventually retire and
remove PackageKit in favor of using dnf everywhere.

Perhaps a more productive approach to this issue would be to allow a
repository to be marked "LeafOnly" - meaning the packages it contains
should not be used to satisfy dependencies of other repositories, and
should not be considered for upgrades of packages installed from other
repositories. Not considering how hard that would be to actually
implement via libsolv and what corner cases would be a problem:-)

>  The stated benefit of the proposed change (“the removal of the state
> where the user has opted in to third party repositories but they are not actually
> enabled”) is not the benefit; the actual benefit that is wanted is an improvement to the
> user experience that is made easier to provide by installing the repos by default. 
Well, true - if it didn't affect either the cli or gui user
experience, this state wouldn't be a problem! :-)

Thanks!
Owen