createrepo/rpm-metadata repository format creator ver 0.3.4

[Ralf Corsepius]

On Sat, 2004-07-24 at 17:19, seth vidal wrote:
> On Sat, 2004-07-24 at 17:12 +0200, Ralf Corsepius wrote:
> > On Sat, 2004-07-24 at 13:24, Sindre Pedersen Bjordal wrote:
> > > lør, 24.07.2004 kl. 13.01 skrev Ralf Corsepius:
> > > > On Fri, 2004-07-23 at 22:24, seth vidal wrote:
> > > > > On Fri, 2004-07-23 at 15:54 +0200, Matias Feliciano wrote:
> > > > > > Le ven 23/07/2004 à 07:13, seth vidal a écrit :
> > > > > > > [...]
> > > > > > > http://linux.duke.edu/metadata/generate/createrepo-0.3.4.tar.gz
> > > > > > > http://linux.duke.edu/metadata/generate/createrepo-0.3.4-1.src.rpm
> > > > > > > 
> > > > > > > [...]
> > > > > > > primary.xml.gz - this holds the primary metadata for the packages
> > > > > > >                  (names, summary, requires, provides, deps, etc)
> > > > > > 
> > > > > > bzip2 instead of gzip ?
> > > > > > apt use bzip2.
> > > > > > 
> > > > > 
> > > > > bzip2 is more processor intensive
> > > > apt uses bzip2 for its repositories for quite a while. The processor
> > > > load this had introduced had never been an actual problem. Conversely,
> > > > users on low bandwith connections had appreciated it.
> > > > 
> > > > Ralf
> > > > 
> > > 
> > > Just out of curiosity, how big a difference does bz2 make in compression
> > 
> > On createrepo generated *.xml files the difference in file size is ca.
> > factor 1.4 to 1.8
> > 
> > > and increase of processor load?
> > Noticeable, but far from being critical:
> > 
> > Bunzip'ing a createrepo generated repo of FC1 (>3000 entries),
> > on a  PIII/1GHz w/ FC1:
^^^^^^^^^^^^^^^^^^^^^^^^^

> [...]
> What speed machine was this on?
Cf. above.

>  Try it out on something slow.
With a complete set of repodata of FC1 on an Intel Pentium/166MHz/MMX
w/64MB-RAM  and FC2 while X11+Gnome-Desktop is running:

# cd repodata
# ls -l
-rw-r--r--  1 root root 133342 Jul 24 17:47 filelists.xml.gz
-rw-r--r--  1 root root 186481 Jul 24 17:47 other.xml.gz
-rw-r--r--  1 root root 895076 Jul 24 17:47 primary.xml.gz
-rw-r--r--  1 root root    903 Jul 24 17:47 repomd.xml

# time gunzip *.gz
real    0m2.606s
user    0m1.365s
sys     0m0.450s# bzip -9 filelists.xml other.xml primary.xml
[Ca. 8 minutes pass.]

# ls -l
-rw-r--r--  1 root root  87713 Jul 24 17:47 filelists.xml.bz2
-rw-r--r--  1 root root  99000 Jul 24 17:47 other.xml.bz2
-rw-r--r--  1 root root 617991 Jul 24 17:47 primary.xml.bz2
-rw-r--r--  1 root root    903 Jul 24 17:47 repomd.xml

# time bunzip2 *.bz2
real    0m45.320s
user    0m11.373s
sys     0m0.776s

For comparison, the same on 3rd machine, a P4/2.6GHz w/FC1:

# time gunzip *.gz
real    0m0.401s
user    0m0.080s
sys     0m0.000s

# bzip2 -9 filelists.xml other.xml primary.xml
[Several seconds.]

#  time bunzip2 *.bz2
real    0m1.550s
user    0m1.190s
sys     0m0.020s

Ralf