Software Management call for RFEs

Zdenek Pavlas zpavlas at redhat.com
Mon May 27 15:29:54 UTC 2013


> Can you point me to the primary.xml -> SQLite translation in yum?  I've
> got a fairly efficient primary.xml parser.

Just set mddownloadpolicy=xml in yum.conf.  It should work, but
since downloading sqlite.bz2 is much better, very few use this.
Yum uses fairly efficient parser, written in C, using libxml2 
(the yum-metadata-parser package). It's always bundled, because
Yum has to support xml-only repositories anyway.

Oh, there's a typo in yum.conf.5 .. fixed.

> It might be interesting to
> see if it's possible to reduce the latency introduced by the SQLite
> conversion to close to zero.  (Decompression and INSERTs can be
> interleaved with downloading, and maybe the index creation improvements
> in SQLite are sufficient these days.)

We have to checksum the downloaded data before processing, and this
kills pipelining. Also, when updating primary_db with a bunch of INSERTs 
and DELETEs, your database differs from the one on server:

- different *.sqlite checksum
- different pkgId <=> PkgKey mapping
- different order of packages from SELECTs

For speed, Yum joins primary_db and filelists_db via pkgKey,
so #2 breaks Yum, unless you always download/delta-update both-
so this kills the win in "we don't need filelists" case.


More information about the devel mailing list