RPM 4.10 pre-release snapshots available

[Panu Matilainen]

On 03/07/2012 04:09 PM, Panu Matilainen wrote:
>
> We're a bit late to our own party this year but rpm 4.10 pre-release
> version should be headed for rawhide in not too far future. In the
> meanwhile, in what is starting to become a tradition at this point,
> Fedora compatible [*] SRPM(s) are available at
> http://laiskiainen.org/rpm/srpms/ for brave testers who wish to take it
> out for an early spin.

One thing I forgot to mention: the current snapshot versions is that 
they lie about their library sonames. There will be a soname version 
bump when it enters rawhide, but to make early testing less painful it's 
currently claiming to be compatible with rpm 4.9.x API. Which it is for 
nearly all practical purposes, the differences requiring soname change 
are in dark corners which are very unlikely to be used by anybody/anything.

> This is not supposed to eat your system alive - I run daily snapshots of
> rpm on my systems throughout development and it certainly hasn't eaten
> mine - but as usual, proceed with caution.
>
> What's noteworthy about this version ... is that there isn't a whole lot
> that you should notice. Probably the most user-visible thing is erasure
> progress meters on -Uvh (and -evh), pretty much everything else is
> behind the scenes improvements and fixes across the board.

To elaborate a bit... a lot of effort in this version has gone into 
areas that would be only visible when things go wrong, and are often not 
particularly easy testable. But to give possible testers something to 
watch out for beyond just breaks the world/seems to work kind of issues:

One of those fully behind-the-scenes improvements is that rpm now 
(finally) automatically reloads SELinux labels if the policy changes 
inside a transaction. This is by no means fix for all the bad rpm <-> 
selinux interactions but it does certainly help.

Another thing observant testers might notice is that rpm pays more 
attention to file states wrt file depedencies now: files that are not / 
wont be actually installed are not considered as providers. Take for 
example bi-arch "shared" executables:

[pmatilai at localhost rpm-4.9.x]$ ./rpm -q --whatprovides /sbin/ldconfig
glibc-2.14.90-24.fc16.6.x86_64
glibc-2.14.90-24.fc16.6.i686
[pmatilai at localhost rpm-4.9.x]$

[pmatilai at localhost rpm]$ ./rpm -q --whatprovides /sbin/ldconfig
glibc-2.14.90-24.fc16.6.x86_64
[pmatilai at localhost rpm]$

This is because despite the glibc.i686 package claiming to also own 
/sbin/ldconfig, it's the file from glibc.x86_64 that is actually 
installed, and removing that would also remove /sbin/ldconfig. glibc is 
a bad example as you can't really remove it in practise, but for other 
cases this can prevent breaking packages that depend on a file actually 
being there.

Similarly installing packages with eg --nodocs will now fail with 
missing dependencies if there are dependencies on files that wouldn't be 
installed because of --nodocs, etc.

This file state-awareness can cause rpm and yum depsolve to disagree on 
in cases where they previously did not, until yum (and other depsolvers 
for that matter) is changed to better cope with it.

There are some optimizations as well, notably header load times are much 
improved. On 'rpm -qa' style one-shot access its mostly lost in the 
noise of other things, but on repeated header loads from the db the 
difference can be quite dramatic. This is with hot cache, best of two 
runs included (differences between runs were marginal anyway), see end 
of this mail for the script used for these timings:

Digest and signature checking disabled:

rpm-4.9:
         [pmatilai at localhost code]$ ./dbloop.py --no-checks
         loaded 180600 headers in 14.868624 seconds

rpm-4.10:
         [pmatilai at localhost code]$ ./dbloop.py --no-checks
         loaded 180600 headers in 7.772259 seconds

Digest and signature checking enabled, public keys for all signed
packages present:

rpm-4.9:
         [pmatilai at localhost code]$ ./dbloop.py
         loaded 180600 headers in 15.813404 seconds

rpm-4.10:
         [pmatilai at localhost code]$ ./dbloop.py
         loaded 180600 headers in 8.529538 seconds

Digest and signature checking enabled, keys for handful of packages 
(primarily f15 packages in f16 the repo) missing:

rpm-4.9:
         [pmatilai at localhost code]$ ./dbloop.py
         loaded 180500 headers in 22.172823 seconds

rpm-4.10:
         [pmatilai at localhost code]$ ./dbloop.py
         loaded 180500 headers in 8.468787 seconds

Digest and signature checking enabled but no keys imported (ie after
'rpm -e --allmatches gpg-pubkey'):

rpm-4.9:
         [pmatilai at localhost code]$ ./dbloop.py
         loaded 180300 headers in 60.147473 seconds

rpm-4.10:
         [pmatilai at localhost code]$ ./dbloop.py
         loaded 180300 headers in 8.179442 seconds


Of course you're hardly going to see performance differences this big in 
real-world usage so take it with a healthy dose of salt :)

--- dumb rpmdb header load benchmark  ---
#!/usr/bin/python
import rpm, time, sys

ts = rpm.ts()

if '--no-checks' in sys.argv:
     ts.setVSFlags(-1)

num_hdrs = 0
start = time.time()
for i in range(0, 100):
     for h in ts.dbMatch():
         num_hdrs += 1
stop = time.time()

print "loaded %d headers in %f seconds" % (num_hdrs, stop-start)

--- cut here ---

	- Panu -