Hi,
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
Dnf hasn't been working properly since F22, while I had not a single problem with yum ever. Still I have to use "dnf clean all" before updating, just to be sure to get all available updates.
There are bug reports reporting the same behaviour, but no solution. As far as I realise, there isn't a way to get yum back.
Any chance that Fedora gets a properly working packet manager in the near future?
[root@chiara ~]# dnf --refresh upgrade RPM Fusion for Fedora 22 - Free - Updates 210 kB/s | 29 kB 00:00 RPM Fusion for Fedora 22 - Nonfree - Updates 149 kB/s | 15 kB 00:00 RPM Fusion for Fedora 22 - Free 1.3 MB/s | 551 kB 00:00 RPM Fusion for Fedora 22 - Nonfree 746 kB/s | 170 kB 00:00 Last metadata expiration check performed 0:00:00 ago on Tue Aug 11 10:24:20 2015. Dependencies resolved. ============================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================== Installing: geany-libgeany x86_64 1.25-2.fc22 updates 1.0 M Upgrading: geany x86_64 1.25-2.fc22 updates 2.8 M gnumeric x86_64 1:1.12.23-1.fc22 updates 12 M goffice x86_64 0.10.23-1.fc22 updates 1.9 M libgudev1 x86_64 219-21.fc22 updates 63 k libgudev1-devel x86_64 219-21.fc22 updates 76 k libsolv x86_64 0.6.11-2.fc22 updates 333 k qtsingleapplication x86_64 2.6.1-23.fc22 updates 42 k systemd x86_64 219-21.fc22 updates 5.9 M systemd-compat-libs x86_64 219-21.fc22 updates 136 k systemd-devel x86_64 219-21.fc22 updates 163 k systemd-libs x86_64 219-21.fc22 updates 351 k systemd-python x86_64 219-21.fc22 updates 96 k systemd-python3 x86_64 219-21.fc22 updates 98 k
Transaction Summary ============================================================================================================================================================================================== Install 1 Package Upgrade 13 Packages
Total download size: 25 M Is this ok [y/N]: n Operation aborted. [root@chiara ~]# dnf clean all Cleaning repos: fedora rpmfusion-free-updates rpmfusion-nonfree-updates rpmfusion-free updates rpmfusion-nonfree Cleaning up Everything [root@chiara ~]# dnf --refresh upgrade Fedora 22 - x86_64 2.2 MB/s | 41 MB 00:19 RPM Fusion for Fedora 22 - Free - Updates 158 kB/s | 29 kB 00:00 RPM Fusion for Fedora 22 - Nonfree - Updates 150 kB/s | 15 kB 00:00 RPM Fusion for Fedora 22 - Free 1.2 MB/s | 551 kB 00:00 Fedora 22 - x86_64 - Updates 2.2 MB/s | 14 MB 00:06 RPM Fusion for Fedora 22 - Nonfree 735 kB/s | 170 kB 00:00 Last metadata expiration check performed 0:00:01 ago on Tue Aug 11 10:25:29 2015. Dependencies resolved. ============================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================== Installing: geany-libgeany x86_64 1.25-2.fc22 updates 1.0 M Upgrading: dnf noarch 1.0.2-3.fc22 updates 240 k dnf-automatic noarch 1.0.2-3.fc22 updates 83 k dnf-conf noarch 1.0.2-3.fc22 updates 81 k dnf-yum noarch 1.0.2-3.fc22 updates 74 k geany x86_64 1.25-2.fc22 updates 2.8 M gnumeric x86_64 1:1.12.23-1.fc22 updates 12 M goffice x86_64 0.10.23-1.fc22 updates 1.9 M hawkey x86_64 0.5.9-3.fc22 updates 90 k libgudev1 x86_64 219-21.fc22 updates 63 k libgudev1-devel x86_64 219-21.fc22 updates 76 k libinput x86_64 0.21.0-3.fc22 updates 89 k libsolv x86_64 0.6.11-2.fc22 updates 333 k pavucontrol x86_64 3.0-2.fc22 updates 141 k python-dnf noarch 1.0.2-3.fc22 updates 432 k python-hawkey x86_64 0.5.9-3.fc22 updates 76 k python3-dnf noarch 1.0.2-3.fc22 updates 436 k python3-hawkey x86_64 0.5.9-3.fc22 updates 71 k qtsingleapplication x86_64 2.6.1-23.fc22 updates 42 k rubygems noarch 2.4.8-100.fc22 updates 269 k systemd x86_64 219-21.fc22 updates 5.9 M systemd-compat-libs x86_64 219-21.fc22 updates 136 k systemd-devel x86_64 219-21.fc22 updates 163 k systemd-libs x86_64 219-21.fc22 updates 351 k systemd-python x86_64 219-21.fc22 updates 96 k systemd-python3 x86_64 219-21.fc22 updates 98 k youtube-dl noarch 2015.08.06.1-1.fc22 updates 1.5 M
Transaction Summary ============================================================================================================================================================================================== Install 1 Package Upgrade 26 Packages
Total download size: 29 M Is this ok [y/N]:
On Tue, 11 Aug 2015 10:35:04 +0200 Heinz Diehl wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
I don't think that's new with dnf. I've seen similar from yum. It all depends on which mirrors it picked to get the data from and the timing of mirrors getting updated.
On 08/11/2015 12:16 PM, Tom Horsley wrote:
On Tue, 11 Aug 2015 10:35:04 +0200 Heinz Diehl wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
Last Sunday, I've had a case, where I resorted to rm -rf /var/cache/dnf because neither "dnf clean all" nor "dnf --refresh" seems to have worked.
No matter what I did dnf seems have refetched the same outdated mirror presenting me the same updates.
I don't think that's new with dnf. I've seen similar from yum.
Well, IIRC, in its infancy yum has had similar issues. The common work around was to "yum clean metadata", then.
Barring the fact Fedora mirrors seem to be broken quite often, these day, with dnf, the situation seems to have worsened. AFAICT, it doesn't correctly validate metadata and/or seems to prefer to refetch broken/outdated/dead mirrors.
It all depends on which mirrors it picked to get the data from and the timing of mirrors getting updated.
Which only means one thing - What I wrote above ;)
Ralf
On Tue, 11 Aug 2015 12:50:02 +0200, Ralf Corsepius wrote:
Last Sunday, I've had a case, where I resorted to rm -rf /var/cache/dnf because neither "dnf clean all" nor "dnf --refresh" seems to have worked.
No matter what I did dnf seems have refetched the same outdated mirror presenting me the same updates.
Out of pure coincidence I'd say.
After a "dnf clean all", some files are left below /var/cache/dnf, but take a look yourself. No repodata files are left, and nothing like the previous mirror you've been assigned to.
Barring the fact Fedora mirrors seem to be broken quite often, these day, with dnf, the situation seems to have worsened. AFAICT, it doesn't correctly validate metadata and/or seems to prefer to refetch broken/outdated/dead mirrors.
Mirroring is a difficult problem. Mirror admins need to know exactly what they are doing. Or else they offer the latest repodata without having mirrored all packages. Yum has choked on that problem often.
Observing Yum and DNF printing lots of errors while trying to find a usable mirror, casts a shadow on the Fedora products as a whole.
One may think that if mirror manager knows the checksums of the last and previous repo metadata releases, it could assign the package tools to a matching mirror that is up-to-date. But either that isn't done, or it's broken. The user gets the impression that the mirrors are out of sync way too often.
On Tue, 2015-08-11 at 10:35 +0200, Heinz Diehl wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
So two update commands at different times give different results?
Dnf hasn't been working properly since F22, while I had not a single problem with yum ever. Still I have to use "dnf clean all" before updating, just to be sure to get all available updates.
No you don't, as has been explained several times recently. You can use "clean metadata" or "--refresh". Doing both is redundant.
poc
On 08/11/2015 12:51 PM, Patrick O'Callaghan wrote:
On Tue, 2015-08-11 at 10:35 +0200, Heinz Diehl wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
So two update commands at different times give different results?
IIUC, you are misunderstanding.
The issues behind this are - "dnf --refetch" is refetching different versions of metadata from different (and differently sync'ed and/or broken) mirrors - fedora's mirrorlists are pointing to mirrors being out of sync.
In addition to that, https://mirrors.fedoraproject.org seems to have been down and inaccessible for several hours, last weekend, which caused additional issues with dnf (and yum).
Ralf
On Tue, 2015-08-11 at 13:13 +0200, Ralf Corsepius wrote:
On 08/11/2015 12:51 PM, Patrick O'Callaghan wrote:
On Tue, 2015-08-11 at 10:35 +0200, Heinz Diehl wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
So two update commands at different times give different results?
IIUC, you are misunderstanding.
The issues behind this are
- "dnf --refetch" is refetching different versions of metadata from
different (and differently sync'ed and/or broken) mirrors
- fedora's mirrorlists are pointing to mirrors being out of sync.
What matters is whether dnf is seeing the same state the two times it runs. In this case it clearly isn't. That can be due to delayed syncing or simply to updates appearing between one run and the next.
In addition to that, https://mirrors.fedoraproject.org seems to have been down and inaccessible for several hours, last weekend, which caused additional issues with dnf (and yum).
Same result. If dnf is run twice you can't guarantee it will give the same result. That's an inherent feature of loosely distributed systems (where there isn't a distributed consensus protocol). Obviously the wider apart the two runs, the more differences will tend to appear, but the presence of differences does not in itself indicate a problem.
poc
On 08/11/2015 01:32 PM, Patrick O'Callaghan wrote:
On Tue, 2015-08-11 at 13:13 +0200, Ralf Corsepius wrote:
On 08/11/2015 12:51 PM, Patrick O'Callaghan wrote:
On Tue, 2015-08-11 at 10:35 +0200, Heinz Diehl wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
So two update commands at different times give different results?
IIUC, you are misunderstanding.
The issues behind this are
- "dnf --refetch" is refetching different versions of metadata from
different (and differently sync'ed and/or broken) mirrors
- fedora's mirrorlists are pointing to mirrors being out of sync.
What matters is whether dnf is seeing the same state the two times it runs. In this case it clearly isn't.
Exactly. With "dnf --refresh" it often does not see the same state.
However, it should! The fact it does not see the same state, means https://mirrors.fedoraproject.org pushing bogus information.
In addition to that, https://mirrors.fedoraproject.org seems to have been down and inaccessible for several hours, last weekend, which caused additional issues with dnf (and yum).
Same result. If dnf is run twice you can't guarantee it will give the same result.
If dnf and mirror-management was functional, then - except in those rare situations when the master has just been updated - they must point to mirrors carrying an identical state.
That's an inherent feature of loosely distributed systems (where there isn't a distributed consensus protocol). Obviously the wider apart the two runs, the more differences will tend to appear, but the presence of differences does not in itself indicate a problem.
My assumption is: mirror-manager is dysfunctional and dnf isn't sufficiently robust.
Ralf
On Tue, 2015-08-11 at 15:42 +0200, Ralf Corsepius wrote:
On 08/11/2015 01:32 PM, Patrick O'Callaghan wrote:
On Tue, 2015-08-11 at 13:13 +0200, Ralf Corsepius wrote:
On 08/11/2015 12:51 PM, Patrick O'Callaghan wrote:
On Tue, 2015-08-11 at 10:35 +0200, Heinz Diehl wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
So two update commands at different times give different results?
IIUC, you are misunderstanding.
The issues behind this are
- "dnf --refetch" is refetching different versions of metadata
from different (and differently sync'ed and/or broken) mirrors
- fedora's mirrorlists are pointing to mirrors being out of sync.
What matters is whether dnf is seeing the same state the two times it runs. In this case it clearly isn't.
Exactly. With "dnf --refresh" it often does not see the same state.
However, it should! The fact it does not see the same state, means https://mirrors.fedoraproject.org pushing bogus information.
In addition to that, https://mirrors.fedoraproject.org seems to have been down and inaccessible for several hours, last weekend, which caused additional issues with dnf (and yum).
Same result. If dnf is run twice you can't guarantee it will give the same result.
If dnf and mirror-management was functional, then - except in those rare situations when the master has just been updated - they must point to mirrors carrying an identical state.
Only if all the mirrors you access are in sync with each other. Without a consensus protocol this cannot be guaranteed.
That's an inherent feature of loosely distributed systems (where there isn't a distributed consensus protocol). Obviously the wider apart the two runs, the more differences will tend to appear, but the presence of differences does not in itself indicate a problem.
My assumption is: mirror-manager is dysfunctional and dnf isn't sufficiently robust.
"Robustness" can mean different things, at least two of which are:
1) dnf always gets the latest updates any reachable mirror has 2) All the mirrors show the same updates so dnf can get any of them
These two criteria are not the same, unless there's a consensus protocol, which I'd lay serious money there isn't as it's expensive to do and is fragile in the face of network outages.
I'm not saying that dnf couldn't be better than it is (I've had problems similar to what you report), but it cannot be perfect in these conditions. Remember that yum wasn't either, it's just more mature.
I suggest you check out the vast literature on distributed synchronization if you want to know more about this stuff.
poc
On 11.08.2015, Patrick O'Callaghan wrote:
So two update commands at different times give different results?
If "two update commands issued directly after another" qualify as "at different times", then yes. In fact, there was not more than max. one minute between the two.
Dnf hasn't been working properly since F22, while I had not a single problem with yum ever. Still I have to use "dnf clean all" before updating, just to be sure to get all available updates.
No you don't, as has been explained several times recently. You can use "clean metadata" or "--refresh". Doing both is redundant.
Obviously, you haven't read my mail with enough attention. The time between the commands is clearly stated, and so are the commands itself. I already used the "--refresh" parameter, and it wasn't enough to get all available updates. Thus "clean all".
On Tue, 11 Aug 2015 15:41:35 +0200, Heinz Diehl wrote:
On 11.08.2015, Patrick O'Callaghan wrote:
So two update commands at different times give different results?
If "two update commands issued directly after another" qualify as "at different times", then yes. In fact, there was not more than max. one minute between the two.
Yet two completely separate contacts with Fedora's metalink server.
Trouble-shooting these kinds of problems would need to include a closer look at what mirrors you are assigned to in both cases.
On 11.08.2015, Michael Schwendt wrote:
Yet two completely separate contacts with Fedora's metalink server.
Trouble-shooting these kinds of problems would need to include a closer look at what mirrors you are assigned to in both cases.
Ok, I see. So what command should I use to keep my system updated? Usually, I update once a week (or two).
On Tue, Aug 11, 2015 at 04:35:56PM +0200, Heinz Diehl wrote:
Yet two completely separate contacts with Fedora's metalink server. Trouble-shooting these kinds of problems would need to include a closer look at what mirrors you are assigned to in both cases.
Ok, I see. So what command should I use to keep my system updated? Usually, I update once a week (or two).
Usually, just plain `dnf upgrade`. If there's an urgent update, you might want to worry about the kind of issues you're seeing here; otherwise, it'll eventually settle itself out.
On 08/11/2015 04:53 PM, Matthew Miller wrote:
On Tue, Aug 11, 2015 at 04:35:56PM +0200, Heinz Diehl wrote:
Yet two completely separate contacts with Fedora's metalink server. Trouble-shooting these kinds of problems would need to include a closer look at what mirrors you are assigned to in both cases.
Ok, I see. So what command should I use to keep my system updated? Usually, I update once a week (or two).
Usually, just plain `dnf upgrade`. If there's an urgent update, you might want to worry about the kind of issues you're seeing here; otherwise, it'll eventually settle itself out.
This was the case last weekend: firefox-39.0.3
Ralf
On Tue, 2015-08-11 at 15:41 +0200, Heinz Diehl wrote:
On 11.08.2015, Patrick O'Callaghan wrote:
So two update commands at different times give different results?
If "two update commands issued directly after another" qualify as "at different times", then yes. In fact, there was not more than max. one minute between the two.
Not relevant, as mschwendt@gmail.com has explained.
Dnf hasn't been working properly since F22, while I had not a single problem with yum ever. Still I have to use "dnf clean all" before updating, just to be sure to get all available updates.
No you don't, as has been explained several times recently. You can use "clean metadata" or "--refresh". Doing both is redundant.
Obviously, you haven't read my mail with enough attention. The time between the commands is clearly stated, and so are the commands itself. I already used the "--refresh" parameter, and it wasn't enough to get all available updates. Thus "clean all".
Once again, "clean metadata", not "clean all" (unless you like refetching rpms you already have).
poc
Heinz Diehl htd+ml@fritha.org wrote:
F22, in short: first running "dnf --refresh upgrade" shows some new packets. Then "dnf clean all" followed by "dnf --refresh upgrade" shows the same packets to be updated, and *some more*.
Yes, you are correct. Several people have verified this behavior, they reported this as bug, and according to Fedora and DNF developers it is intentional.
"dnf --refresh" is more like "dnf clean expire-cache", which sometimes gives additional updates to plain "dnf upgrade", but there still seems some caching involved that keeps it from providing all updates available.
"dnf clean all" does the job, of course, but for that specific purpose, "dnf clean metadata" is sufficient. Both force dnf to download and rebuild all metadata. The metadata is the key to the solution.
Unfortunately, this needs lots of bandwidth and CPU. If you just want fresh updates, there's usually no need to process the big base fedora metadata over and over again.
For me, this does the trick:
dnf --disablerepo=fedora clean metadata ; dnf upgrade
It's pretty fast and gives latest updates. Well, there's still a chance to get redirected to a mirror which isn't synced with latest stuff. But there's not much you can do about that.
Greetings, Andreas
On Sat, 15 Aug 2015 13:21:49 +0000 (UTC), Andreas M. Kirchwitz wrote:
"dnf --refresh" is more like "dnf clean expire-cache", which sometimes gives additional updates to plain "dnf upgrade", but there still seems some caching involved that keeps it from providing all updates available.
Doubtful.
"dnf update --refresh" here (Rawhide) always redownloads the metadata. That's behaviour like running after "dnf clean metadata", not "dnf clean expire-cache". [1]
Expired metadata could be reactivated after asking mirror manager whether they are latest.
-
Andreas M. Kirchwitz -
Heinz Diehl -
Matthew Miller -
Michael Schwendt -
Patrick O'Callaghan -
Ralf Corsepius -
Tom Horsley