F36 Change: Enable exclude_from_weak_autodetect by default in LIBDNF (System-Wide Change proposal)
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
== Owner == * Name: [[User:jmracek| Jaroslav Mracek]] * Email: jmracek@redhat.com
== Detailed Description == The feature is designed to prevent an install of removed weak dependencies from the system by users and to not install weak dependencies missing after system deployment. It will change the behavior of DNF, microdnf, and PackageKit. The feature will be backported to all Fedoras, but in default, the feature will be off. Additional information: https://bugzilla.redhat.com/show_bug.cgi?id=1699672
The default value for exclude_from_weak_autodetect configuration can be overridden in `/etc/dnf/dnf.conf`
== Feedback == The feature was requested by [[User:Churchyard|Miro Hrončok]] and supported by many others: See [https://bugzilla.redhat.com/show_bug.cgi?id=1699672 rhbz#1699672] for more feedback.
== Benefit to Fedora == After the installation of a fresh system, the first upgrade will not install a lot of weak dependencies. Some of them were excluded from the kick-start installation set for good reasons (security, image size, minimal functional set, ...), but after the first update, all weak dependencies are installed, therefore some features of deployment simply disappear.
== Scope == * Proposal owners: ** The feature is ready in Pull Request - https://github.com/rpm-software-management/libdnf/pull/1279 ** PRs only wait for a release of libsolv ** The Feature will be enabled in upstream as default, therefore from Fedora 36, we start to release libdnf without a revert patch of default in comparison to upstream.
* Other developers: The change requires a new release of libsolv.
* Release engineering: * Policies and guidelines: A packaging guideline should be added that discourages or forbids weak dependencies on fully versioned (sub)packages (see [https://bugzilla.redhat.com/show_bug.cgi?id=1699672#c44 the details]). * Trademark approval: N/A (not needed for this Change) * Alignment with Objectives:
== Upgrade/compatibility impact == No manual changes will be required. After the libdnf update, this feature will be on by default.
== How To Test == 1. Install package without satisfied weak dependencies 2. Upgrade the upgrade. With exclude_from_weak_autodetect=true, it will not install weak dependencies of already installed packages. With exclude_from_weak_autodetect=false, weak dependencies will be installed during upgrades.
== User Experience == The change in default will help to keep some values for particular deployments (a minimal system will be still minimal without disabling weak dependencies). Users will be able to remove particular weak dependencies and they will be not installed on the first upgrade. In case when the feature will not work according to the user expectation it can be switched off in the dnf configuration file.
== Dependencies == libsolv - Required code changes are already in the libsolv upstream. We only wait for the next libsolv release.
== Contingency Plan ==
There are no external dependencies, therefore we can easily postpone the feature and the change of default behavior.
* Contingency mechanism: (What to do? Who will do it?) * Contingency deadline: beta freeze * Blocks release? No
== Documentation == The feature will be documented in dnf man pages.
On Thu, Sep 16, 2021 at 3:18 PM Ben Cotton bcotton@redhat.com wrote:
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
== Owner ==
- Name: [[User:jmracek| Jaroslav Mracek]]
- Email: jmracek@redhat.com
== Detailed Description == The feature is designed to prevent an install of removed weak dependencies from the system by users and to not install weak dependencies missing after system deployment. It will change the behavior of DNF, microdnf, and PackageKit. The feature will be backported to all Fedoras, but in default, the feature will be off. Additional information: https://bugzilla.redhat.com/show_bug.cgi?id=1699672
The default value for exclude_from_weak_autodetect configuration can be overridden in `/etc/dnf/dnf.conf`
== Feedback == The feature was requested by [[User:Churchyard|Miro Hrončok]] and supported by many others: See [https://bugzilla.redhat.com/show_bug.cgi?id=1699672 rhbz#1699672] for more feedback.
== Benefit to Fedora == After the installation of a fresh system, the first upgrade will not install a lot of weak dependencies. Some of them were excluded from the kick-start installation set for good reasons (security, image size, minimal functional set, ...), but after the first update, all weak dependencies are installed, therefore some features of deployment simply disappear.
== Scope ==
- Proposal owners:
** The feature is ready in Pull Request - https://github.com/rpm-software-management/libdnf/pull/1279 ** PRs only wait for a release of libsolv ** The Feature will be enabled in upstream as default, therefore from Fedora 36, we start to release libdnf without a revert patch of default in comparison to upstream.
Other developers: The change requires a new release of libsolv.
Release engineering:
Policies and guidelines: A packaging guideline should be added that
discourages or forbids weak dependencies on fully versioned (sub)packages (see [https://bugzilla.redhat.com/show_bug.cgi?id=1699672#c44 the details]).
- Trademark approval: N/A (not needed for this Change)
- Alignment with Objectives:
== Upgrade/compatibility impact == No manual changes will be required. After the libdnf update, this feature will be on by default.
== How To Test ==
- Install package without satisfied weak dependencies
- Upgrade the upgrade. With exclude_from_weak_autodetect=true, it
will not install weak dependencies of already installed packages. With exclude_from_weak_autodetect=false, weak dependencies will be installed during upgrades.
== User Experience == The change in default will help to keep some values for particular deployments (a minimal system will be still minimal without disabling weak dependencies). Users will be able to remove particular weak dependencies and they will be not installed on the first upgrade. In case when the feature will not work according to the user expectation it can be switched off in the dnf configuration file.
== Dependencies == libsolv - Required code changes are already in the libsolv upstream. We only wait for the next libsolv release.
== Contingency Plan ==
There are no external dependencies, therefore we can easily postpone the feature and the change of default behavior.
- Contingency mechanism: (What to do? Who will do it?)
- Contingency deadline: beta freeze
- Blocks release? No
== Documentation == The feature will be documented in dnf man pages.
Woot! I'm looking forward to this feature!
-- 真実はいつも一つ！/ Always, there's only one truth!
On Thu, Sep 16, 2021 at 9:22 PM Neal Gompa ngompa13@gmail.com wrote:
On Thu, Sep 16, 2021 at 3:18 PM Ben Cotton bcotton@redhat.com wrote:
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
== Owner ==
- Name: [[User:jmracek| Jaroslav Mracek]]
- Email: jmracek@redhat.com
== Detailed Description == The feature is designed to prevent an install of removed weak dependencies from the system by users and to not install weak dependencies missing after system deployment. It will change the behavior of DNF, microdnf, and PackageKit. The feature will be backported to all Fedoras, but in default, the feature will be off. Additional information: https://bugzilla.redhat.com/show_bug.cgi?id=1699672
The default value for exclude_from_weak_autodetect configuration can be overridden in `/etc/dnf/dnf.conf`
== Feedback == The feature was requested by [[User:Churchyard|Miro Hrončok]] and supported by many others: See [https://bugzilla.redhat.com/show_bug.cgi?id=1699672 rhbz#1699672] for more feedback.
== Benefit to Fedora == After the installation of a fresh system, the first upgrade will not install a lot of weak dependencies. Some of them were excluded from the kick-start installation set for good reasons (security, image size, minimal functional set, ...), but after the first update, all weak dependencies are installed, therefore some features of deployment simply disappear.
== Scope ==
- Proposal owners:
** The feature is ready in Pull Request - https://github.com/rpm-software-management/libdnf/pull/1279 ** PRs only wait for a release of libsolv ** The Feature will be enabled in upstream as default, therefore from Fedora 36, we start to release libdnf without a revert patch of default in comparison to upstream.
Other developers: The change requires a new release of libsolv.
Release engineering:
Policies and guidelines: A packaging guideline should be added that
discourages or forbids weak dependencies on fully versioned (sub)packages (see [https://bugzilla.redhat.com/show_bug.cgi?id=1699672#c44 the details]).
- Trademark approval: N/A (not needed for this Change)
- Alignment with Objectives:
== Upgrade/compatibility impact == No manual changes will be required. After the libdnf update, this feature will be on by default.
== How To Test ==
- Install package without satisfied weak dependencies
- Upgrade the upgrade. With exclude_from_weak_autodetect=true, it
will not install weak dependencies of already installed packages. With exclude_from_weak_autodetect=false, weak dependencies will be installed during upgrades.
== User Experience == The change in default will help to keep some values for particular deployments (a minimal system will be still minimal without disabling weak dependencies). Users will be able to remove particular weak dependencies and they will be not installed on the first upgrade. In case when the feature will not work according to the user expectation it can be switched off in the dnf configuration file.
== Dependencies == libsolv - Required code changes are already in the libsolv upstream. We only wait for the next libsolv release.
== Contingency Plan ==
There are no external dependencies, therefore we can easily postpone the feature and the change of default behavior.
- Contingency mechanism: (What to do? Who will do it?)
- Contingency deadline: beta freeze
- Blocks release? No
== Documentation == The feature will be documented in dnf man pages.
Woot! I'm looking forward to this feature!
I second that, I'm looking forward to this.
However, we probably need to adapt some stuff if this is enabled by default, because some things have abused the current behavior. For exmple, if I remember correctly, mozilla-openh264 / gstreamer support for it are installed as weak dependencies with post-GA package updates.
Fabio
On 16. 09. 21 21:43, Fabio Valentini wrote:
For exmple, if I remember correctly, mozilla-openh264 / gstreamer support for it are installed as weak dependencies with post-GA package updates.
Why exactly is this a problem?
On Thu, Sep 16, 2021 at 4:02 PM Miro Hrončok mhroncok@redhat.com wrote:
On 16. 09. 21 21:43, Fabio Valentini wrote:
For exmple, if I remember correctly, mozilla-openh264 / gstreamer support for it are installed as weak dependencies with post-GA package updates.
Why exactly is this a problem?
Because it will not happen anymore. We need a new way to trigger its installation post-install.
On 16. 09. 21 22:03, Neal Gompa wrote:
On Thu, Sep 16, 2021 at 4:02 PM Miro Hrončok mhroncok@redhat.com wrote:
On 16. 09. 21 21:43, Fabio Valentini wrote:
For exmple, if I remember correctly, mozilla-openh264 / gstreamer support for it are installed as weak dependencies with post-GA package updates.
Why exactly is this a problem?
Because it will not happen anymore. We need a new way to trigger its installation post-install.
Oh. We don't have them installed on the media, but we want them to be pulled in on upgrades? Is there some legal requirement that forbids us to have them installed by default directly?
On Thu, Sep 16, 2021 at 5:24 PM Miro Hrončok mhroncok@redhat.com wrote:
On 16. 09. 21 22:03, Neal Gompa wrote:
On Thu, Sep 16, 2021 at 4:02 PM Miro Hrončok mhroncok@redhat.com wrote:
On 16. 09. 21 21:43, Fabio Valentini wrote:
For exmple, if I remember correctly, mozilla-openh264 / gstreamer support for it are installed as weak dependencies with post-GA package updates.
Why exactly is this a problem?
Because it will not happen anymore. We need a new way to trigger its installation post-install.
Oh. We don't have them installed on the media, but we want them to be pulled in on upgrades? Is there some legal requirement that forbids us to have them installed by default directly?
Yes. The patent license for H.264 (AVC) for use with OpenH264 is only conveyed when distributed by Cisco, which we do through the RPM repository hosted by them.
-- 真実はいつも一つ！/ Always, there's only one truth!
On 16. 09. 21 23:26, Neal Gompa wrote:
On Thu, Sep 16, 2021 at 5:24 PM Miro Hrončok mhroncok@redhat.com wrote:
On 16. 09. 21 22:03, Neal Gompa wrote:
On Thu, Sep 16, 2021 at 4:02 PM Miro Hrončok mhroncok@redhat.com wrote:
On 16. 09. 21 21:43, Fabio Valentini wrote:
For exmple, if I remember correctly, mozilla-openh264 / gstreamer support for it are installed as weak dependencies with post-GA package updates.
Why exactly is this a problem?
Because it will not happen anymore. We need a new way to trigger its installation post-install.
Oh. We don't have them installed on the media, but we want them to be pulled in on upgrades? Is there some legal requirement that forbids us to have them installed by default directly?
Yes. The patent license for H.264 (AVC) for use with OpenH264 is only conveyed when distributed by Cisco, which we do through the RPM repository hosted by them.
Thanks for refreshing my memory.
We could include a meta-package in the default installation, which Recommends the Cisco package on GA but is switched to Requires in a 0-day update.
On 16/09/2021 23:31, Miro Hrončok wrote:
We could include a meta-package in the default installation, which Recommends the Cisco package on GA but is switched to Requires in a 0-day update.
We can't require packages from third-party repositories. This is forbidden by packaging guidelines.
Last year I received a lot of negative feedback when asked if I could add a weak dependency on package from the RPM Fusion repository.
On Fri, Sep 17, 2021 at 10:59:36AM +0200, Vitaly Zaitsev via devel wrote:
On 16/09/2021 23:31, Miro Hrončok wrote:
We could include a meta-package in the default installation, which Recommends the Cisco package on GA but is switched to Requires in a 0-day update.
We can't require packages from third-party repositories. This is forbidden by packaging guidelines.
I'm do not think this applies to the cisco h264 repo. We build those packages and sign them and simply have cisco redistribute them. So, IMHO this is not the same as a 3rd party repo. A 3rd party is distributing, but we control all the contents.
Last year I received a lot of negative feedback when asked if I could add a weak dependency on package from the RPM Fusion repository.
yes, don't do that definitely.
kevin
Thanks for this change to finally make Recommends work as expected. It's been needed for a long time.
On Thu, Sep 16 2021 at 11:31:40 PM +0200, Miro Hrončok mhroncok@redhat.com wrote:
We could include a meta-package in the default installation, which Recommends the Cisco package on GA but is switched to Requires in a 0-day update.
The plan is to provide a one-shot service that will install the packages manually. But yeah, this would work too.
Michael
Neal Gompa wrote:
On Thu, Sep 16, 2021 at 4:02 PM Miro Hrončok mhroncok@redhat.com wrote:
On 16. 09. 21 21:43, Fabio Valentini wrote:
For exmple, if I remember correctly, mozilla-openh264 / gstreamer support for it are installed as weak dependencies with post-GA package updates.
Why exactly is this a problem?
Because it will not happen anymore.
And that is a feature. A system upgrade should never add stuff that is not a hard dependency behind the user's back.
There is a better H.264 implementation in another third-party repository.
Kevin Kofler
On Thu, Sep 16, 2021 at 9:18 PM Ben Cotton bcotton@redhat.com wrote:
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
Exciting. I have the following questions: 1. Do I understand correctly that this will be enabled by default in F36 and later and disabled by default in F35 and older? The proposal text is *very* convoluted and doesn't explain this clearly (I think it should be edited to be clearer). 2. What happens if package P (already installed on the user's system) starts recommending package Q (not installed on the user's system)? Will Q get auto-installed together with P's update, or not? I believe it's important to keep auto-installation enabled for *new* weak relationships. 3. Similarly to above (perhaps exactly the same case), what happens when package Q (not installed) starts supplementing package P (installed), will it get auto-installed or not? 4. If there's a scenario where we want some packages pulled automatically on the first update after installation, but we don't want to include them on the media (possibly because of size constraints or something else), how can that be achieved? Has somebody reviewed the kickstarts for cases like these? I think at least some localization-related files are automatically installed post-installation.
On 17. 09. 21 9:28, Kamil Paral wrote:
On Thu, Sep 16, 2021 at 9:18 PM Ben Cotton <bcotton@redhat.com mailto:bcotton@redhat.com> wrote:
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect <https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect> == Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.Exciting. I have the following questions:
- Do I understand correctly that this will be enabled by default in F36 and
later and disabled by default in F35 and older? The proposal text is *very* convoluted and doesn't explain this clearly (I think it should be edited to be clearer).
Yes. So the default behavior only changes on the new Fedora release, but users can opt in to it on older Fedoras as well. (I agree.)
- What happens if package P (already installed on the user's system) starts
recommending package Q (not installed on the user's system)? Will Q get auto-installed together with P's update, or not? I believe it's important to keep auto-installation enabled for *new* weak relationships.
I believe it should get auto installed, but I have not yet verified the current implementation does that.
- Similarly to above (perhaps exactly the same case), what happens when
package Q (not installed) starts supplementing package P (installed), will it get auto-installed or not?
I believe it should get auto installed, but I have not yet verified the current implementation does that.
- If there's a scenario where we want some packages pulled automatically on
the first update after installation, but we don't want to include them on the media (possibly because of size constraints or something else), how can that be achieved? Has somebody reviewed the kickstarts for cases like these? I think at least some localization-related files are automatically installed post-installation.
See the other part of this thread about openh264.
- What happens if package P (already installed on the user's system)
starts recommending package Q (not installed on the user's system)? Will Q get auto-installed together with P's update, or not? I believe it's important to keep auto-installation enabled for *new* weak relationships.
New weak dependencies of package P are installed. Installed P-1-1.noarch (no recommends) Available P-1-2.noarch (recommends ddd) will install ddd on upgrade if possible.
- Similarly to above (perhaps exactly the same case), what happens when
package Q (not installed) starts supplementing package P (installed), will it get auto-installed or not?
No, Q will be not installed. With supplements it is difficult to known when it appears, because that information is not on RPMDB.
On Mon, Sep 27, 2021 at 8:30 AM Jaroslav Mracek jmracek@redhat.com wrote:
- What happens if package P (already installed on the user's system)
starts recommending package Q (not installed on the user's system)? Will
Q
get auto-installed together with P's update, or not? I believe it's important to keep auto-installation enabled for *new* weak relationships.
New weak dependencies of package P are installed. Installed P-1-1.noarch (no recommends) Available P-1-2.noarch (recommends ddd) will install ddd on upgrade if possible.
- Similarly to above (perhaps exactly the same case), what happens when
package Q (not installed) starts supplementing package P (installed),
will
it get auto-installed or not?
No, Q will be not installed. With supplements it is difficult to known when it appears, because that information is not on RPMDB.
While it makes sense technically, this might be quite confusing for packagers. Up until now I think there were no real-world differences between forward (recommends) and backward (supplements) dependencies. This (and also the first answer) should get documented in the Change proposal and in the packaging guidelines [1]. Can you please add an action item to the proposal to adjust relevant Fedora docs?
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/WeakDependencies/
On 27. 09. 21 10:22, Kamil Paral wrote:
> 3. Similarly to above (perhaps exactly the same case), what happens when > package Q (not installed) starts supplementing package P (installed), will > it get auto-installed or not? No, Q will be not installed. With supplements it is difficult to known when it appears, because that information is not on RPMDB.While it makes sense technically, this might be quite confusing for packagers.
I've checked the status quo.
Package "reproducer_reversed" starts supplementing package "rpm". "rpm" is installed, but "reproducer_reversed" is not.
1. dnf upgarde, no rpm update available: reproducer_reversed is not pulled in 2. dnf reinstall rpm: reproducer_reversed is pulled in 3. dnf downgrade rpm: reproducer_reversed is pulled in 4. dnf upgrade rpm: reproducer_reversed is pulled in 5. dnf upgrade, rpm update avilable: reproducer_reversed is pulled in
Would this change proposal actually change the observed behavior? In what way?
On 27. 09. 21 15:01, Miro Hrončok wrote:
On 27. 09. 21 10:22, Kamil Paral wrote:
> 3. Similarly to above (perhaps exactly the same case), what happens when > package Q (not installed) starts supplementing package P (installed), will > it get auto-installed or not?
No, Q will be not installed. With supplements it is difficult to known when it appears, because that information is not on RPMDB.
While it makes sense technically, this might be quite confusing for packagers.
I've checked the status quo.
Package "reproducer_reversed" starts supplementing package "rpm". "rpm" is installed, but "reproducer_reversed" is not.
- dnf upgarde, no rpm update available: reproducer_reversed is not pulled in
- dnf reinstall rpm: reproducer_reversed is pulled in
- dnf downgrade rpm: reproducer_reversed is pulled in
- dnf upgrade rpm: reproducer_reversed is pulled in
- dnf upgrade, rpm update avilable: reproducer_reversed is pulled in
Would this change proposal actually change the observed behavior? In what way?
I forgot to mention, here is a copr repo with reproducer_reversed: to play with
https://copr.fedorainfracloud.org/coprs/churchyard/reproducer_reversed/
On Mon, Sep 27, 2021 at 3:03 PM Miro Hrončok mhroncok@redhat.com wrote:
I've checked the status quo.
Package "reproducer_reversed" starts supplementing package "rpm". "rpm" is installed, but "reproducer_reversed" is not.
- dnf upgarde, no rpm update available: reproducer_reversed is not pulled
in 2. dnf reinstall rpm: reproducer_reversed is pulled in 3. dnf downgrade rpm: reproducer_reversed is pulled in 4. dnf upgrade rpm: reproducer_reversed is pulled in 5. dnf upgrade, rpm update avilable: reproducer_reversed is pulled in
Would this change proposal actually change the observed behavior? In what way?
Based on Jaroslav's response, I'm afraid the new behavior will be that "reproducer_reversed" doesn't get pulled in in any of those cases (or perhaps just in case #2). But let's wait for Jaroslav to provide a definitive answer.
On Wed, Sep 29, 2021 at 2:49 AM Kamil Paral kparal@redhat.com wrote:
On Mon, Sep 27, 2021 at 3:03 PM Miro Hrončok mhroncok@redhat.com wrote:
I've checked the status quo.
Package "reproducer_reversed" starts supplementing package "rpm". "rpm" is installed, but "reproducer_reversed" is not.
- dnf upgarde, no rpm update available: reproducer_reversed is not pulled in
- dnf reinstall rpm: reproducer_reversed is pulled in
- dnf downgrade rpm: reproducer_reversed is pulled in
- dnf upgrade rpm: reproducer_reversed is pulled in
- dnf upgrade, rpm update avilable: reproducer_reversed is pulled in
Would this change proposal actually change the observed behavior? In what way?
Based on Jaroslav's response, I'm afraid the new behavior will be that "reproducer_reversed" doesn't get pulled in in any of those cases (or perhaps just in case #2). But let's wait for Jaroslav to provide a definitive answer.
It might be worth renaming the option "exclude_from_weak_autodetect" to imply its actual effect.
Strawman idea: "weakexclude_unsatisfied_weakdeps_on_upgrade"?
-- 真実はいつも一つ！/ Always, there's only one truth!
On 11. 10. 21 21:10, Neal Gompa wrote:
On Wed, Sep 29, 2021 at 2:49 AM Kamil Paral kparal@redhat.com wrote:
On Mon, Sep 27, 2021 at 3:03 PM Miro Hrončok mhroncok@redhat.com wrote:
I've checked the status quo.
Package "reproducer_reversed" starts supplementing package "rpm". "rpm" is installed, but "reproducer_reversed" is not.
- dnf upgarde, no rpm update available: reproducer_reversed is not pulled in
- dnf reinstall rpm: reproducer_reversed is pulled in
- dnf downgrade rpm: reproducer_reversed is pulled in
- dnf upgrade rpm: reproducer_reversed is pulled in
- dnf upgrade, rpm update avilable: reproducer_reversed is pulled in
Would this change proposal actually change the observed behavior? In what way?
Based on Jaroslav's response, I'm afraid the new behavior will be that "reproducer_reversed" doesn't get pulled in in any of those cases (or perhaps just in case #2). But let's wait for Jaroslav to provide a definitive answer.
It might be worth renaming the option "exclude_from_weak_autodetect" to imply its actual effect.
Strawman idea: "weakexclude_unsatisfied_weakdeps_on_upgrade"?
If I understand this right, it won't be only on upgrade. Also on reinstall, downgrade, etc.
Correct, it will effect all dnf operations
Jaroslav
On Mon, Oct 11, 2021 at 9:57 PM Miro Hrončok mhroncok@redhat.com wrote:
On 11. 10. 21 21:10, Neal Gompa wrote:
On Wed, Sep 29, 2021 at 2:49 AM Kamil Paral kparal@redhat.com wrote:
On Mon, Sep 27, 2021 at 3:03 PM Miro Hrončok mhroncok@redhat.com
wrote:
I've checked the status quo.
Package "reproducer_reversed" starts supplementing package "rpm".
"rpm" is
installed, but "reproducer_reversed" is not.
- dnf upgarde, no rpm update available: reproducer_reversed is not
pulled in
- dnf reinstall rpm: reproducer_reversed is pulled in
- dnf downgrade rpm: reproducer_reversed is pulled in
- dnf upgrade rpm: reproducer_reversed is pulled in
- dnf upgrade, rpm update avilable: reproducer_reversed is pulled in
Would this change proposal actually change the observed behavior? In
what way?
Based on Jaroslav's response, I'm afraid the new behavior will be that
"reproducer_reversed" doesn't get pulled in in any of those cases (or perhaps just in case #2). But let's wait for Jaroslav to provide a definitive answer.
It might be worth renaming the option "exclude_from_weak_autodetect" to imply its actual effect.
Strawman idea: "weakexclude_unsatisfied_weakdeps_on_upgrade"?
If I understand this right, it won't be only on upgrade. Also on reinstall, downgrade, etc.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
On Tue, Oct 12, 2021 at 9:12 AM Jaroslav Mracek jmracek@redhat.com wrote:
Correct, it will effect all dnf operations
So then drop the "_on_upgrade" part? "weakexclude_unsatisfied_weakdeps"
On Tue, Oct 12, 2021 at 09:14:21AM -0400, Neal Gompa wrote:
On Tue, Oct 12, 2021 at 9:12 AM Jaroslav Mracek jmracek@redhat.com wrote:
Correct, it will effect all dnf operations
So then drop the "_on_upgrade" part? "weakexclude_unsatisfied_weakdeps"
Or maybe "exclude_old_unsatisfied_weak_deps" ?
My colleague told me a very nice argument when we discussed the name of the option. There is no way to name it by the way that everyone will understand it, therefore let's make it simple. Anyway I will discuss it in the team
Jaroslav
On Tue, Oct 12, 2021 at 3:15 PM Neal Gompa ngompa13@gmail.com wrote:
On Tue, Oct 12, 2021 at 9:12 AM Jaroslav Mracek jmracek@redhat.com wrote:
Correct, it will effect all dnf operations
So then drop the "_on_upgrade" part? "weakexclude_unsatisfied_weakdeps"
-- 真実はいつも一つ！/ Always, there's only one truth!
Yes it is correct. Supplements that are not installed during the first install, cannot be installed anymore with enabled autodetection. There is no way to calculate it correctly without storing all provides at the time of installation for each package.
Jaroslav
On Wed, Sep 29, 2021 at 8:52 AM Kamil Paral kparal@redhat.com wrote:
On Mon, Sep 27, 2021 at 3:03 PM Miro Hrončok mhroncok@redhat.com wrote:
I've checked the status quo.
Package "reproducer_reversed" starts supplementing package "rpm". "rpm" is installed, but "reproducer_reversed" is not.
- dnf upgarde, no rpm update available: reproducer_reversed is not
pulled in 2. dnf reinstall rpm: reproducer_reversed is pulled in 3. dnf downgrade rpm: reproducer_reversed is pulled in 4. dnf upgrade rpm: reproducer_reversed is pulled in 5. dnf upgrade, rpm update avilable: reproducer_reversed is pulled in
Would this change proposal actually change the observed behavior? In what way?
Based on Jaroslav's response, I'm afraid the new behavior will be that "reproducer_reversed" doesn't get pulled in in any of those cases (or perhaps just in case #2). But let's wait for Jaroslav to provide a definitive answer.
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
I like the proposal, but I very much dislike the name (including the configuration option name). I don't have better name at hand, but I think it would be worth of a bit of brainstorming.
Vít
Dne 16. 09. 21 v 21:17 Ben Cotton napsal(a):
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
== Owner ==
- Name: [[User:jmracek| Jaroslav Mracek]]
- Email: jmracek@redhat.com
== Detailed Description == The feature is designed to prevent an install of removed weak dependencies from the system by users and to not install weak dependencies missing after system deployment. It will change the behavior of DNF, microdnf, and PackageKit. The feature will be backported to all Fedoras, but in default, the feature will be off. Additional information: https://bugzilla.redhat.com/show_bug.cgi?id=1699672
The default value for exclude_from_weak_autodetect configuration can be overridden in `/etc/dnf/dnf.conf`
== Feedback == The feature was requested by [[User:Churchyard|Miro Hrončok]] and supported by many others: See [https://bugzilla.redhat.com/show_bug.cgi?id=1699672 rhbz#1699672] for more feedback.
== Benefit to Fedora == After the installation of a fresh system, the first upgrade will not install a lot of weak dependencies. Some of them were excluded from the kick-start installation set for good reasons (security, image size, minimal functional set, ...), but after the first update, all weak dependencies are installed, therefore some features of deployment simply disappear.
== Scope ==
- Proposal owners:
** The feature is ready in Pull Request - https://github.com/rpm-software-management/libdnf/pull/1279 ** PRs only wait for a release of libsolv ** The Feature will be enabled in upstream as default, therefore from Fedora 36, we start to release libdnf without a revert patch of default in comparison to upstream.
Other developers: The change requires a new release of libsolv.
Release engineering:
Policies and guidelines: A packaging guideline should be added that
discourages or forbids weak dependencies on fully versioned (sub)packages (see [https://bugzilla.redhat.com/show_bug.cgi?id=1699672#c44 the details]).
- Trademark approval: N/A (not needed for this Change)
- Alignment with Objectives:
== Upgrade/compatibility impact == No manual changes will be required. After the libdnf update, this feature will be on by default.
== How To Test ==
- Install package without satisfied weak dependencies
- Upgrade the upgrade. With exclude_from_weak_autodetect=true, it
will not install weak dependencies of already installed packages. With exclude_from_weak_autodetect=false, weak dependencies will be installed during upgrades.
== User Experience == The change in default will help to keep some values for particular deployments (a minimal system will be still minimal without disabling weak dependencies). Users will be able to remove particular weak dependencies and they will be not installed on the first upgrade. In case when the feature will not work according to the user expectation it can be switched off in the dnf configuration file.
== Dependencies == libsolv - Required code changes are already in the libsolv upstream. We only wait for the next libsolv release.
== Contingency Plan ==
There are no external dependencies, therefore we can easily postpone the feature and the change of default behavior.
- Contingency mechanism: (What to do? Who will do it?)
- Contingency deadline: beta freeze
- Blocks release? No
== Documentation == The feature will be documented in dnf man pages.
On 17. 09. 21 12:21, Vít Ondruch wrote:
I like the proposal, but I very much dislike the name (including the configuration option name). I don't have better name at hand, but I think it would be worth of a bit of brainstorming.
Change name:
Don't reinstall already not installed weak dependencies during package updates
Dne 17. 09. 21 v 12:40 Miro Hrončok napsal(a):
On 17. 09. 21 12:21, Vít Ondruch wrote:
I like the proposal, but I very much dislike the name (including the configuration option name). I don't have better name at hand, but I think it would be worth of a bit of brainstorming.
Change name:
Don't reinstall already not installed weak dependencies during package updates
Exclude once ignored weak dependencies
or
Ignore once excluded weak dependencies
Vít
Naming is really difficult topic. We would like to have the same name like libsolv has.
Ben Cotton wrote:
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
+1, finally a Change I am actually looking forward to!
Kevin Kofler
Hi everyone,
On Thu, 2021-09-16 at 15:17 -0400, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
I am not sure if this was intended, but this change has broken rich weak dependencies when both packages are not installed as part of the same transaction.
In my yt-dlp package's specfile[1], I have three subpackages for shell completions: `yt-dlp-bash-completion`, `yt-dlp-zsh-completion`, and `yt-dlp-fish-completion. Here is the `bash-completion` block:
``` spec %package bash-completion Summary: Bash completion for %{name} Requires: %{name} = %{version} Requires: bash-completion Supplements: (%{name} and bash-completion) BuildArch: noarch ```
The intended effect is for the shell completions to be installed at the time `yt-dlp` itself is installed if the respective shell package (`bash-completion`, `zsh` or `fish`) is already present while still allowing users to opt out. However, now this does not work; dnf will only install the completions if both `yt-dlp` and the shell package are installed as part of the same transaction. I can confirm that this is caused by this change, because adding `-- setopt=exclude_from_weak_autodetect=false` fixes the problem. Replacing `Supplements` with forward facing boolean `Requires` did not work either.
``` spec Recommends: (%{name}-bash-completion if bash-completion) Recommends: (%{name}-zsh-completion if zsh) Recommends: (%{name}-fish-completion if fish) ```
While I agree that {rich,} weak dependencies should not be reinstalled as part of updates, I do believe that they should be installed if one of the packages is being installed for the first time.
I also think we should consider implementing better guidelines for shell completions in Fedora. I believe that shell completions should be split into subpackages and that these subpackages should depend on the shells themselves or a `-filesystem` package that actually own the directories. Right now, directory ownership is kind of a mess. At least on my system, there are several packages that own /usr/share/bash- completion, /usr/share/zsh/vendor-completions, /usr/share/zsh/site- functions, and /usr/share/fish/vendor_completions.d/. We can also use this oppurtunity to create macros for each of these directories.
Management of shell completion packages was discussed further in my package review ticket [2].
I am relatively new to Fedora, so please correct me if I got anything wrong.
Thanks, Maxwell
[1]: https://src.fedoraproject.org/rpms/yt-dlp/blob/rawhide/f/yt-dlp.spec [2]: https://bugzilla.redhat.com/show_bug.cgi?id=2012522
Hi everyone,
I am resending this message, because I think it got lost. I sent it at the begining of a weekend, so people must not have seen it.
I am CC'ing the change owner, as I feel that more clarification is required. I maintain that this change should only apply to updates; `dnf install`, `dnf reinstall` should behave as they have been. At least, this change and all of its effects should be fully explained to packagers.
We should probably discuss the shell completion stuff seperately.
Thanks, Maxwell
On Fri, 2021-11-12 at 15:48 -0600, Maxwell G wrote:
Hi everyone,
On Thu, 2021-09-16 at 15:17 -0400, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
I am not sure if this was intended, but this change has broken rich weak dependencies when both packages are not installed as part of the same transaction.
In my yt-dlp package's specfile[1], I have three subpackages for shell completions: `yt-dlp-bash-completion`, `yt-dlp-zsh-completion`, and `yt-dlp-fish-completion. Here is the `bash-completion` block:
%package bash-completion Summary: Bash completion for %{name} Requires: %{name} = %{version} Requires: bash-completion Supplements: (%{name} and bash-completion) BuildArch: noarchThe intended effect is for the shell completions to be installed at the time `yt-dlp` itself is installed if the respective shell package (`bash-completion`, `zsh` or `fish`) is already present while still allowing users to opt out. However, now this does not work; dnf will only install the completions if both `yt-dlp` and the shell package are installed as part of the same transaction. I can confirm that this is caused by this change, because adding `-- setopt=exclude_from_weak_autodetect=false` fixes the problem. Replacing `Supplements` with forward facing boolean `Requires` did not work either.
Recommends: (%{name}-bash-completion if bash-completion) Recommends: (%{name}-zsh-completion if zsh) Recommends: (%{name}-fish-completion if fish)While I agree that {rich,} weak dependencies should not be reinstalled as part of updates, I do believe that they should be installed if one of the packages is being installed for the first time.
I also think we should consider implementing better guidelines for shell completions in Fedora. I believe that shell completions should be split into subpackages and that these subpackages should depend on the shells themselves or a `-filesystem` package that actually own the directories. Right now, directory ownership is kind of a mess. At least on my system, there are several packages that own /usr/share/bash- completion, /usr/share/zsh/vendor-completions, /usr/share/zsh/site- functions, and /usr/share/fish/vendor_completions.d/. We can also use this oppurtunity to create macros for each of these directories.
Management of shell completion packages was discussed further in my package review ticket [2].
I am relatively new to Fedora, so please correct me if I got anything wrong.
Thanks, Maxwell
On Fri, Nov 12, 2021 at 10:50 PM Maxwell G via devel < devel@lists.fedoraproject.org> wrote:
Hi everyone,
On Thu, 2021-09-16 at 15:17 -0400, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect
== Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package.
I am not sure if this was intended, but this change has broken rich weak dependencies when both packages are not installed as part of the same transaction.
In my yt-dlp package's specfile[1], I have three subpackages for shell completions: `yt-dlp-bash-completion`, `yt-dlp-zsh-completion`, and `yt-dlp-fish-completion. Here is the `bash-completion` block:
%package bash-completion Summary: Bash completion for %{name} Requires: %{name} = %{version} Requires: bash-completion Supplements: (%{name} and bash-completion) BuildArch: noarchThe intended effect is for the shell completions to be installed at the time `yt-dlp` itself is installed if the respective shell package (`bash-completion`, `zsh` or `fish`) is already present while still allowing users to opt out. However, now this does not work; dnf will only install the completions if both `yt-dlp` and the shell package are installed as part of the same transaction. I can confirm that this is caused by this change, because adding `-- setopt=exclude_from_weak_autodetect=false` fixes the problem. Replacing `Supplements` with forward facing boolean `Requires` did not work either.
Recommends: (%{name}-bash-completion if bash-completion) Recommends: (%{name}-zsh-completion if zsh) Recommends: (%{name}-fish-completion if fish)While I agree that {rich,} weak dependencies should not be reinstalled as part of updates, I do believe that they should be installed if one of the packages is being installed for the first time.
I also think we should consider implementing better guidelines for shell completions in Fedora. I believe that shell completions should be split into subpackages and that these subpackages should depend on the shells themselves or a `-filesystem` package that actually own the directories. Right now, directory ownership is kind of a mess. At least on my system, there are several packages that own /usr/share/bash- completion, /usr/share/zsh/vendor-completions, /usr/share/zsh/site- functions, and /usr/share/fish/vendor_completions.d/. We can also use this oppurtunity to create macros for each of these directories.
Management of shell completion packages was discussed further in my package review ticket [2].
I am relatively new to Fedora, so please correct me if I got anything wrong.
Thanks, Maxwell
Hey Maxwell, can you please file a new bug in bugzilla against dnf and copy the problem description into it? Then make your new bug block bug 2013327 [1], which is the tracker for this Change. This way you'll make sure that this problem doesn't get lost and the maintainer has to deal with it before the appropriate Change deadline. Thanks!
On Friday, December 10, 2021 8:29:10 AM CST Kamil Paral wrote:
Hey Maxwell, can you please file a new bug in bugzilla against dnf and copy the problem description into it? Then make your new bug block bug 2013327 [1], which is the tracker for this Change. This way you'll make sure that this problem doesn't get lost and the maintainer has to deal with it before the appropriate Change deadline. Thanks!
I just did[1]. Thank you for pointing me in the right direction!
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=2033130
It looks like that the feature made a user case with langpacks broken. See additional reports: Bug 2048394 - dnf should pull weak dependencies in install transaction Bug 2033130 - exclude_from_weak_autodetect=true effectively renders rich weak dependencies useless Bug 2042808 - weakdeps not working on rawhide system
The feature was requested: Bug 2005305 - dnf should not pull (already unmet) weak dependencies on updates Bug 1699672 - RFE: dnf should not pull (already broken) weak dependencies on updates
Firs of all I need to clarify that the feature cannot be implemented only on upgrades - because there are technical reasons for that - 1. DNF creates one transaction for all operations (install, upgrades are performed together). 2.a Install operation or commands (not only install) also triggers update. (example - I have already installed foo-1-1.noarch. Then I will install bar-2-2.noarch that requires foo-2. It means the install command will trigger upgrade that dnf cannot detect in advance. And if foo recommends something, it will be installed) 2.b Install operation with --best (default in RHEL) triggers always upgrade when package is already installed but in lower version.
Be honest I do not know what to do. Basically I see only 3 option with one additional: 1. Keep it like it is 2. Disable autodetection 3. Start to ignore rich dependencies for autodetection of unmet weak dependencies. We have a problem to detect rich dependencies correctly in autodetections because we do not know whether their conditions were met or not in past. 4. In theory the auto-detection can be only triggered by upgrade command but it will create an inconsistency in DNF behavior when upgrade operation is triggered by the another command (install, buildeps, downgrade, ...) - not preferable, see above.
-
Ben Cotton -
Fabio Valentini -
Jaroslav Mracek -
Kamil Paral -
Kevin Fenzi -
Kevin Kofler -
Maxwell G -
Michael Catanzaro -
Miro Hrončok -
Neal Gompa -
Vitaly Zaitsev -
Vít Ondruch -
Zbigniew Jędrzejewski-Szmek