what is wrong with dnf?

[Tom Horsley]

I ran "dnf -y update" this morning. It said the last
metadata update was 8 days ago and there was nothing
to do.

I then ran "dnf clean all" and "dnf -y update" again
and it is loading 414 new updates.

What really bad setting is it consulting to think that
8 day old metadata is up to date, and where the heck do
I fix it?

[Alex Gurenko]

Not sure, where the TTL for meta data is set, but you can always use --refresh parameter
to force-refresh meta-data without running dnf clean all.


---
Best regards,
 Alex

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, November 27, 2018 3:34 PM, Tom Horsley <horsley1953(a)gmail.com&gt; wrote:

>  I ran "dnf -y update" this morning. It said the last
>  metadata update was 8 days ago and there was nothing
>  to do.
>
>  I then ran "dnf clean all" and "dnf -y update" again
>  and it is loading 414 new updates.
>
>  What really bad setting is it consulting to think that
>  8 day old metadata is up to date, and where the heck do
>  I fix it?
>
>  users mailing list -- users(a)lists.fedoraproject.org
>  To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]

[Ed Greshko]

On 11/27/18 9:34 PM, Tom Horsley wrote:
>  I ran "dnf -y update" this morning. It said the last
>  metadata update was 8 days ago and there was nothing
>  to do.
>
>  I then ran "dnf clean all" and "dnf -y update" again
>  and it is loading 414 new updates.
>
>  What really bad setting is it consulting to think that
>  8 day old metadata is up to date, and where the heck do
>  I fix it?
>  
You statement seems to imply that everyone should be having a problem.  I'm not having
any
issues, so it would seem not to be a "default" setting issue.

Have you made any changes to your "dnf.conf"?  Specifically to metadata_expire.

Have you disabled the dnf-makecache.timer unit?

-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme

[Tom Horsley]

On Tue, 27 Nov 2018 22:52:38 +0800
Ed Greshko wrote:

>  Have you disabled the dnf-makecache.timer unit? 
Yes. I want to run updates when I want them, not when
some random timer wants them (especially since the
background stuff always manages to have dnf locked
when I want to do foreground activity :-).

[Ed Greshko]

On 11/27/18 11:28 PM, Tom Horsley wrote:
>  On Tue, 27 Nov 2018 22:52:38 +0800
>  Ed Greshko wrote:
>
> > Have you disabled the dnf-makecache.timer unit?
>  Yes. I want to run updates when I want them, not when
>  some random timer wants them (especially since the
>  background stuff always manages to have dnf locked
>  when I want to do foreground activity :-). 
That doesn't do any update.  Just updates the cache/metadata. 

So, you're getting the expected behavior after making changes. 


-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme

[Sam Varshavchik]

Tom Horsley writes:

>  I ran "dnf -y update" this morning. It said the last
>  metadata update was 8 days ago and there was nothing
>  to do.
>
>  I then ran "dnf clean all" and "dnf -y update" again
>  and it is loading 414 new updates.
>
>  What really bad setting is it consulting to think that
>  8 day old metadata is up to date, and where the heck do
>  I fix it? 
Something's definitely not quite right with DNF's caching in F29.

I have a local LAN repository whose metadata_expire is intentionally set to  
one minute, because as soon as I have a new package to install I push it  
into the repository and then I want to have the update installed immediately.

Even when I explicitly specify --refresh dnf will often tell me that the  
metadata was refresh only a few hours ago, and that's it's done.

Ok, well, that's definitely more than a minute. I just run the same "dnf  
upgrade --refresh" command a second time. This always makes dnf redownload  
the repo and see the updated packages, since its metadata is older than the  
set expiration interval.

This was more stable in F28. dnf reliably refreshed the cached metadata when  
it expired. I can't quite work out what's up with dnf now. Sometimes it  
refreshes the expired metadata. Sometimes it does that only on a 2nd  
attempt. There's no rhyme or reason to it.

[Matthew Miller]

On Tue, Nov 27, 2018 at 06:11:44PM -0500, Sam Varshavchik wrote:
>  Something's definitely not quite right with DNF's caching in
> F29.
>  
>  I have a local LAN repository whose metadata_expire is intentionally
>  set to one minute, because as soon as I have a new package to
>  install I push it into the repository and then I want to have the
>  update installed immediately.
>  
>  Even when I explicitly specify --refresh dnf will often tell me that
>  the metadata was refresh only a few hours ago, and that's it's done.
>  
>  Ok, well, that's definitely more than a minute. I just run the same
>  "dnf upgrade --refresh" command a second time. This always makes dnf
>  redownload the repo and see the updated packages, since its metadata
>  is older than the set expiration interval.
>  
>  This was more stable in F28. dnf reliably refreshed the cached
>  metadata when it expired. I can't quite work out what's up with dnf
>  now. Sometimes it refreshes the expired metadata. Sometimes it does
>  that only on a 2nd attempt. There's no rhyme or reason to it. 
If you can find a reproducer, a bug report for this would be helpful.


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Tom Horsley]

As near as I can tell, dnf won't ever download
data from repos merely because of an update command
(no matter how far out of date the metadata is).
This seems like an "improvement" they must have
made recently.

Looking at the makecache timer service, it seems
like you now need to use two commands to download
updates manually at a time of your own choosing:

dnf makecache
dnf update

That worked for me when the dnf update by itself
didn't.

[Ed Greshko]

On 11/28/18 8:57 AM, Tom Horsley wrote:
>  As near as I can tell, dnf won't ever download
>  data from repos merely because of an update command
>  (no matter how far out of date the metadata is).
>  This seems like an "improvement" they must have
>  made recently.
>
>  Looking at the makecache timer service, it seems
>  like you now need to use two commands to download
>  updates manually at a time of your own choosing:
>
>  dnf makecache
>  dnf update
>
>  That worked for me when the dnf update by itself
>  didn't. 
Only one command is needed and was pointed out earlier in the thread.

[egreshko@meimei ~]$ sudo dnf upgrade
Last metadata expiration check: 0:00:39 ago on Wed 28 Nov 2018 09:02:36 AM CST.
Dependencies resolved.
Nothing to do.
Complete!

[egreshko@meimei ~]$ sudo dnf --refresh upgrade
Adobe Systems Incorporated                         2.1 kB/s | 2.9 kB     00:01   
Fedora Modular 29 - x86_64                         4.0 kB/s | 7.2 kB     00:01   
Fedora Modular 29 - x86_64 - Updates               4.0 kB/s | 7.4 kB     00:01   
Fedora 29 - x86_64 - Updates                       7.5 kB/s | 5.9 kB     00:00   
Fedora 29 - x86_64                                 8.9 kB/s | 7.3 kB     00:00   
google-chrome                                      949  B/s | 1.3 kB     00:01   
google-earth-pro                                   3.4 kB/s | 1.3 kB     00:00   
google-earth                                       3.4 kB/s | 1.3 kB     00:00   
RPM Fusion for Fedora 29 - Free tainted             17 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Free - Updates           14 kB/s |  12 kB     00:00   
RPM Fusion for Fedora 29 - Free                     18 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree tainted          17 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree - Updates        15 kB/s |  12 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree                  18 kB/s |  14 kB     00:00   
skype (stable)                                     1.5 kB/s | 2.9 kB     00:01   
Dependencies resolved.
Nothing to do.
Complete!



-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme

[Tom Horsley]

On Wed, 28 Nov 2018 09:05:53 +0800
Ed Greshko wrote:

>  [egreshko@meimei ~]$ sudo dnf --refresh upgrade 
But that wastes time re-downloading all the repos, not just
the update repos (which are presumably the only ones that
might have changed).

I'd hope makecache pays attention to the metadata_expire
parameters.

[Ed Greshko]

On 11/28/18 9:21 AM, Tom Horsley wrote:
>  On Wed, 28 Nov 2018 09:05:53 +0800
>  Ed Greshko wrote:
>
> > [egreshko@meimei ~]$ sudo dnf --refresh upgrade
>  But that wastes time re-downloading all the repos, not just
>  the update repos (which are presumably the only ones that
>  might have changed).
>
>  I'd hope makecache pays attention to the metadata_expire
>  parameters. 
Instead of hoping, try it?

[egreshko@meimei ~]$ time sudo dnf --refresh upgrade
Adobe Systems Incorporated                         2.1 kB/s | 2.9 kB     00:01   
Fedora Modular 29 - x86_64                         4.1 kB/s | 7.5 kB     00:01   
Fedora Modular 29 - x86_64 - Updates               9.2 kB/s | 7.4 kB     00:00   
Fedora 29 - x86_64 - Updates                       8.2 kB/s | 6.6 kB     00:00   
Fedora 29 - x86_64                                 8.6 kB/s | 7.6 kB     00:00   
google-chrome                                      3.3 kB/s | 1.3 kB     00:00   
google-earth-pro                                   3.4 kB/s | 1.3 kB     00:00   
google-earth                                       3.4 kB/s | 1.3 kB     00:00   
RPM Fusion for Fedora 29 - Free tainted            7.7 kB/s |  14 kB     00:01   
RPM Fusion for Fedora 29 - Free - Updates           14 kB/s |  12 kB     00:00   
RPM Fusion for Fedora 29 - Free                     18 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree tainted          17 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree - Updates        15 kB/s |  12 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree                  18 kB/s |  14 kB     00:00   
skype (stable)                                     1.5 kB/s | 2.9 kB     00:01   
Dependencies resolved.
Nothing to do.
Complete!

real    0m18.654s
user    0m3.771s
sys     0m0.284s

[egreshko@meimei ~]$ time sudo dnf makecache
Adobe Systems Incorporated                         8.1 kB/s | 2.9 kB     00:00   
Fedora Modular 29 - x86_64                         4.0 kB/s | 7.5 kB     00:01   
Fedora Modular 29 - x86_64 - Updates               9.2 kB/s | 7.4 kB     00:00   
Fedora 29 - x86_64 - Updates                        10 kB/s | 6.6 kB     00:00   
Fedora 29 - x86_64                                 9.4 kB/s | 7.6 kB     00:00   
google-chrome                                      3.4 kB/s | 1.3 kB     00:00   
google-earth-pro                                   3.4 kB/s | 1.3 kB     00:00   
google-earth                                       3.4 kB/s | 1.3 kB     00:00   
RPM Fusion for Fedora 29 - Free tainted             17 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Free - Updates           14 kB/s |  12 kB     00:00   
RPM Fusion for Fedora 29 - Free                     18 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree tainted          17 kB/s |  14 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree - Updates        14 kB/s |  12 kB     00:00   
RPM Fusion for Fedora 29 - Nonfree                  18 kB/s |  14 kB     00:00   
skype (stable)                                     1.4 kB/s | 2.9 kB     00:02   
Metadata cache created.

real    0m17.973s
user    0m2.210s
sys     0m0.209s



-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme

[Ed Greshko]

On 11/28/18 9:21 AM, Tom Horsley wrote:
>  But that wastes time re-downloading all the repos 
So?   Make an for dnf that does the --refresh and also disables the repos you don't
care
to about?

-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme

[Ed Greshko]

On 11/28/18 9:30 AM, Ed Greshko wrote:
>  On 11/28/18 9:21 AM, Tom Horsley wrote:
> > But that wastes time re-downloading all the repos
>  So?   Make an for dnf that does the --refresh and also disables the repos you don't
> care
>  to about?
>  
Make an "alias".

-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme

[Ed Greshko]

On 11/28/18 9:30 AM, Ed Greshko wrote:
>  On 11/28/18 9:30 AM, Ed Greshko wrote:
> > On 11/28/18 9:21 AM, Tom Horsley wrote:
> >> But that wastes time re-downloading all the repos
> > So?   Make an for dnf that does the --refresh and also disables the repos you
> don't care
> > to about?
> >
>  Make an "alias".
>  
[egreshko@acer ~]$ which dnfup
alias dnfup='sudo dnf --refresh
--disablerepo=fedora,fedora-cisco-openh264,rpmfusion-free,rpmfusion-nonfree,adobe-linux-x86_64,fedora-modular
upgrade'
        /usr/bin/sudo

[egreshko@acer ~]$ time dnfup
Fedora Modular 29 - x86_64 - Updates               4.6 kB/s |  15 kB     00:03   
Fedora 29 - x86_64 - Updates                       6.9 kB/s |  16 kB     00:02   
google-chrome                                      794  B/s | 1.3 kB     00:01   
RPM Fusion for Fedora 29 - Free - Updates          6.3 kB/s |  13 kB     00:02   
RPM Fusion for Fedora 29 - Nonfree - Updates       3.9 kB/s |  13 kB     00:03   
Dependencies resolved.
Nothing to do.
Complete!

real    0m15.856s
user    0m3.143s
sys     0m0.333s

Looks like I saved around 3~4 seconds.

-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme

[Ron Yorston]

Matthew Miller wrote:
> If you can find a reproducer, a bug report for this would be helpful.
I think this one covers the issue:

   https://bugzilla.redhat.com/show_bug.cgi?id=1648274

As noted there, there's no need to muck about with the cache or the
--refresh option.  Simply running 'dnf update' twice makes things work
as they should.

Ron

[Sam Varshavchik]

Tom Horsley writes:

>  On Wed, 28 Nov 2018 09:05:53 +0800
>  Ed Greshko wrote:
>
>  > [egreshko@meimei ~]$ sudo dnf --refresh upgrade
>
>  But that wastes time re-downloading all the repos, not just
>  the update repos (which are presumably the only ones that
>  might have changed). 
Also, that does. not. work.

I have plenty of case evidence that --refresh does not always actually  
refresh a repo whose metadata has expired. You have to run it twice, for  
some inexplicable reason.

[Ed Greshko]

On 11/28/18 8:07 PM, Sam Varshavchik wrote:
>  Tom Horsley writes:
>
> > On Wed, 28 Nov 2018 09:05:53 +0800
> > Ed Greshko wrote:
> >
> > > [egreshko@meimei ~]$ sudo dnf --refresh upgrade
> >
> > But that wastes time re-downloading all the repos, not just
> > the update repos (which are presumably the only ones that
> > might have changed).
>
>  Also, that does. not. work.
>
>  I have plenty of case evidence that --refresh does not always actually refresh a repo
>  whose metadata has expired. You have to run it twice, for some inexplicable reason.
>
>  
Sorry, maybe you have had to run it twice.  I, however, haven't had that
experience on any
of my systems.

Maybe I've just been lucky in always getting fully sync'd mirrors on the first
try?

-- 
Right: I dislike the default color scheme Wrong: What idiot picked the default color
scheme