/var/cache/PackageKit/metadata gets very big

[D. Hugh Redelmeier]

On my Fedora 23 systems, /var/cache/PackageKit/metadata/updates/packages 
is full of .rpm files.  For example, 2.7G on my desktop.

/var/cache/PackageKit/metadata/fedora/packages has some too.

"sudo dnf clean packages" doesn't seem to change this.  What does it 
change?

What is the purpose of this directory of RPMs?

Why is it considered metadata?

The more obvious place would be /var/cache/PackageKit/downloads/
Why isn't this used?

Is there a way of transplanting the RPMs to another system so that
downloading could be avoided?  I have half a dozen fedora systems and
it seems a waste to download each update for each system.

What is the proper way of deleting these RPMs to free up the space?  I
ask because on some machines (but not all) the space is burdensome.

[stan]

On Thu, 17 Mar 2016 10:22:36 -0400 (EDT)
"D. Hugh Redelmeier" <hugh(a)mimosa.com&gt; wrote:

>  On my Fedora 23
>  systems, /var/cache/PackageKit/metadata/updates/packages is full
>  of .rpm files.  For example, 2.7G on my desktop.
>  
>  /var/cache/PackageKit/metadata/fedora/packages has some too.
>  
>  "sudo dnf clean packages" doesn't seem to change this.  What does it 
>  change? 
The dnf package storage is cleared.  PackageKit is independent of
dnf.

>  
>  What is the purpose of this directory of RPMs? 
Saved in case of problems.

>  Why is it considered metadata?
>  
>  The more obvious place would be /var/cache/PackageKit/downloads/
>  Why isn't this used? 
You'd have to ask the PackageKit devs.

>  
>  Is there a way of transplanting the RPMs to another system so that
>  downloading could be avoided?  I have half a dozen fedora systems and
>  it seems a waste to download each update for each system. 
rsync them into /var/cache/dnf/[hashed directory entry]/packages/ on the
other systems.  Then run dnf upgrade.  It will reuse the local
packages.  You might have to set dnf to keep rpms in /etc/dnf/dnf.conf,
and manually run dnf clean packages after the update.

>  What is the proper way of deleting these RPMs to free up the space? 
> I
>  ask because on some machines (but not all) the space is burdensome. 
There is a setting in the file
/etc/PackageKit/PackageKit.conf

# Keep the packages after they have been downloaded
#KeepCache=false

Remove, as root, the hash mark on this configuration option, and
packages won't be kept.

[D. Hugh Redelmeier]

| From: stan <stanl-fedorauser(a)vfemail.net&gt;

Thanks for your useful response.  But there are still things that
puzzle/annoy me.

| On Thu, 17 Mar 2016 10:22:36 -0400 (EDT)
| "D. Hugh Redelmeier" <hugh(a)mimosa.com&gt; wrote:
| 
| > On my Fedora 23
| > systems, /var/cache/PackageKit/metadata/updates/packages is full
| > of .rpm files.  For example, 2.7G on my desktop.
| > /var/cache/PackageKit/metadata/fedora/packages has some too.

| > "sudo dnf clean packages" doesn't seem to change this.  What does it

| > change?
| 
| The dnf package storage is cleared.  PackageKit is independent of
| dnf.

Isn't dnf the user interface to PackageKit?

"man PackageKit" points to pkmon and pkcon, both with the identical
description: "PackageKit console client".  One of those is wrong.  In
any case, those commands don't seem to have knobs to adjust PackageKit
hoarding instincts.

| > What is the purpose of this directory of RPMs?
| 
| Saved in case of problems.

What kind of problems?

Perhaps they are used to reduce the size of download (so only a
difference needs to be transferred).

In any case, lots of the saved RPMs are obsoleted by other saved RPMs.
That seems like a waste.  So I deleted all the obsolete ones and saved
a lot of space.  I only bothered with
	/var/cache/PackageKit/metadata/updates/packages
so far.  That saved me almost 2G!

Here's how I did it.  Not quite a script yet.

	cd /var/cache/PackageKit/metadata/updates/packages

	# create a list of installed RPM files
	rpm -qa | sed -e 's/$/.rpm/' | sort >~/0installed

	# create a list of hoarded files
	ls >~/0saved

	# create a list of files that are obsoleted
	diff ~/0saved ~/0installed | sed -n -e 's/^< //p' >~/0obsolete

	# Exercise for the reader: delete files list in ~/0obsolete
	# This is the first step that requires root.
	# Note: ~/ means something different for each user.

Before I did this, "dnf update" failed due to lack of space.  After,
it just worked.  So I guess PackageKit isn't upset by files just
disappearing from this directory.

| > Is there a way of transplanting the RPMs to another system so that
| > downloading could be avoided?
| 
| rsync them into /var/cache/dnf/[hashed directory entry]/packages/

Thanks.  That seems a little hacky so I won't bother.

| > What is the proper way of deleting these RPMs to free up the space?  I
| > ask because on some machines (but not all) the space is burdensome.
| 
| There is a setting in the file
| /etc/PackageKit/PackageKit.conf
| 
| # Keep the packages after they have been downloaded
| #KeepCache=false

Thanks.  It didn't seem to affect old files (it didn't buy back space
that I needed for "dnf update"). I didn't try to debug this.  Perhaps
because PackageKit is a daemon and I didn't tap it on the shoulder to
reload the config.

[stan]

On Fri, 18 Mar 2016 17:14:29 -0400 (EDT)
"D. Hugh Redelmeier" <hugh(a)mimosa.com&gt; wrote:


>  Isn't dnf the user interface to PackageKit? 
I always disable PackageKit, so I'm probably not the best person to
answer this.  But, my understanding is that PackageKit operates
independently of dnf.  That when PackageKit updates the system, it
doesn't invoke dnf, but interacts with rpm directly.  I might be wrong.

pkmon is a console debug tool kit for PackageKit, pkcon is just cli
interface without any debug capability.

>  What kind of problems?
>  
>  Perhaps they are used to reduce the size of download (so only a
>  difference needs to be transferred). 
No, that's drpm.  If you have a flaky connection, and your update
crashes, if this isn't turned on, all the rpms go away, and your
download starts over from the beginning.

And it allows someone to do what you were thinking of doing, updating a
single box, and then re-using all those RPMs for othep systems.
Especially worthwhile on slow connections, or where people get charged
for usage.

>  Here's how I did it.  Not quite a script yet.
>  
>  	cd /var/cache/PackageKit/metadata/updates/packages
>  
>  	# create a list of installed RPM files
>  	rpm -qa | sed -e 's/$/.rpm/' | sort >~/0installed
>  
>  	# create a list of hoarded files
>  	ls >~/0saved
>  
>  	# create a list of files that are obsoleted
>  	diff ~/0saved ~/0installed | sed -n -e 's/^< //p' >~/0obsolete
>  
>  	# Exercise for the reader: delete files list in ~/0obsolete
>  	# This is the first step that requires root.
>  	# Note: ~/ means something different for each user. 
I use a python program I wrote, that uses regexes to compare the
filenames, and deletes the earlier version.  It's a hack, but it
doesn't have to be perfect.  If it is +/- 5%, no big deal.  And it
seems to be a lot better than that, maybe .5% or less.
 
>  Before I did this, "dnf update" failed due to lack of
> space.  After,
>  it just worked.  So I guess PackageKit isn't upset by files just
>  disappearing from this directory. 
From this, it seems that PackageKit and dnf are sharing an rpm package
directory.

[John Pilkington]

On 18/03/16 22:15, stan wrote:
>  On Fri, 18 Mar 2016 17:14:29 -0400 (EDT)
>  "D. Hugh Redelmeier" <hugh(a)mimosa.com&gt; wrote:
>
>
> > Isn't dnf the user interface to PackageKit?
>
>  I always disable PackageKit, so I'm probably not the best person to
>  answer this.  But, my understanding is that PackageKit operates
>  independently of dnf.  That when PackageKit updates the system, it
>  doesn't invoke dnf, but interacts with rpm directly.  I might be wrong.
>
>  pkmon is a console debug tool kit for PackageKit, pkcon is just cli
>  interface without any debug capability.
>
> > What kind of problems?
> >
> > Perhaps they are used to reduce the size of download (so only a
> > difference needs to be transferred).
>
>  No, that's drpm.  If you have a flaky connection, and your update
>  crashes, if this isn't turned on, all the rpms go away, and your
>  download starts over from the beginning.
>
>  And it allows someone to do what you were thinking of doing, updating a
>  single box, and then re-using all those RPMs for othep systems.
>  Especially worthwhile on slow connections, or where people get charged
>  for usage.
>
> > Here's how I did it.  Not quite a script yet.
> >
> > 	cd /var/cache/PackageKit/metadata/updates/packages
> >
> > 	# create a list of installed RPM files
> > 	rpm -qa | sed -e 's/$/.rpm/' | sort >~/0installed
> >
> > 	# create a list of hoarded files
> > 	ls >~/0saved
> >
> > 	# create a list of files that are obsoleted
> > 	diff ~/0saved ~/0installed | sed -n -e 's/^< //p' >~/0obsolete
> >
> > 	# Exercise for the reader: delete files list in ~/0obsolete
> > 	# This is the first step that requires root.
> > 	# Note: ~/ means something different for each user.
>
>  I use a python program I wrote, that uses regexes to compare the
>  filenames, and deletes the earlier version.  It's a hack, but it
>  doesn't have to be perfect.  If it is +/- 5%, no big deal.  And it
>  seems to be a lot better than that, maybe .5% or less.
>
> > Before I did this, "dnf update" failed due to lack of space.  After,
> > it just worked.  So I guess PackageKit isn't upset by files just
> > disappearing from this directory.
>
>   From this, it seems that PackageKit and dnf are sharing an rpm package
>  directory.
>  
Isn't this what repomanage is for? I used it years ago to clear outdated 
rpms from a local repo.   google 'repomanage  xargs'.

Now (fc23) it appears to be in

python3-dnf-plugins-extras-repomanage

John P