TeX Live 2008/9 packaging and you

[Jindrich Novy]

On Wed, Jun 03, 2009 at 01:56:24PM -0400, Josh Boyer wrote:
> On Tue, Jun 02, 2009 at 01:27:26PM +0200, Jindrich Novy wrote:
> >Hi all,
> >
> >sorry for long mail. I thought it's a good idea to catch up with you
> >guys before I update to the new TeX Live in rawhide and discuss possible
> >problems. To make really long story short, the new TeX Live comes with
> >a huge set of subpackages (about 4000) so what I want to ask you if it
> >will just slow down yum and general updating terribly or is it
> >generally acceptable? (considering the metadata size growth?)
> 
> Thanks for the heads up.  I've also CC'd Seth for comments on potential
> yum/repodata issues.
> 
> >For more details please read this (to-be) announcement:
> >
> >I finally manage to package the whole TeX Live 2008 distribution
> >including binaries coming from the actual 2009 TeX Live development tree:
> >http://koji.fedoraproject.org/koji/taskinfo?taskID=1382525
> >
> >Not only that it uses the latest rpm features such as:
> >https://fedoraproject.org/wiki/Features/NoarchSubpackages
> >New version texlive-2008 (to be in f12):
> >* one single texlive package generating 3944 subpackages / 1065 MiB
> >* spec file automagically generated from upstream metadata with
> >  http://people.redhat.com/jnovy/files/tl2008/tl2rpm.c
> >* upstream collections/packages are separate source tarballs
> >  (one could update one small source tarball if a particular style
> >  needs update and doesn't need to repack the whole single tarball or
> >  patch it)
> >* new texlive is maximally scalable depending on which features are
> >  needed:
> >  - basic installation needs only 12 MiB of packages to be downloaded!
> >  - essential features such as pdflatex is supported in this basic
> >    scheme
> >* it is a full (not truncated or otherwise crippled) version of TeX
> >  Live
> 
> So while I think flexibility is good, having almost 4000 subpackages seems
> a bit excessive.  Is the split done for licensing reasons alone?  If so, could
> we make larger groups by license or even just a single 'texlive-misc'
> subpackage?

Yup, 4000 subpackages is pain. The split to ~4000 subpackages is done not only
because of the licensing reasons. The main motivation was that the TeX
Live itself is composed from schemes (e.g. scheme-minimal,
scheme-basic), they are composed from collections (collection-latex, collection-langgreek,
etc.) which are composed from TL packages.

My first thought was to package TeX Live on collection level what
could reduce subpackage count down to ~150. But I can't do that
because packages do overlap among collections and schemes as well (schemes and
collecions are meta-packages requiring particular packages). So
the atomic part has to be packages :-P

> 
> From a rel-eng point of view, we have to sign all RPMs, subpackage or not.  So
> when we get around to signing texlive, we'd need to now sign ~4000 packages.
> That will add a significant amount of time to signing.

Indeed, signing such a huge amount of packages manually is not a way
to go. Fortunatelly I have a way to workaround that (read below).

> 
> There is also the updates route to consider.  I'm not sure if bodhi is up to
> handling an update with that many RPMs involved.  It may well be, but it would
> need testing, which also brings up the signing issue since we sign updates too.
> 

I was thinking about the update route as well what could reduce number
of updates going out.

Each TL package has a revision number from upstream. So each package
has its own release based on TL package version (if present in metadata) and revision:

texlive-fontools-2008-12775.noarch.rpm
texlive-fontools-doc-2008-12775.noarch.rpm
texlive-fontools.ARCH-2009-12687.i386.rpm
texlive-fontspec-2008-v1.18.10220.noarch.rpm
texlive-fontspec-doc-2008-v1.18.10220.noarch.rpm
texlive-fontspec-source-2008-v1.18.10220.noarch.rpm
texlive-fonttable-2008-1.4.13033.noarch.rpm
texlive-fonttable-doc-2008-1.4.13033.noarch.rpm
texlive-fonttable-source-2008-1.4.13033.noarch.rpm
texlive-fontwrap-2008-8624.noarch.rpm
texlive-fontwrap-doc-2008-8624.noarch.rpm

Version of a package is 2008 (and 2009 for arch dependent bits as they
are coming from TL2009 sources and uses different versioning).

Furthermore each package has its own tar.lzma tarball from which it's
built.

So the thing is that even if we have everything stuffed within one
src.rpm we could do a per-TL-package updates.


Sample situation:
1. User complains that texlive-fontspec is buggy and needs updating
2. I will grab fontspec.tar.lzma with revision 11000 from upstream
3. regenerate texlive.spec
4. rebuild texlive

Results:
Since no NEVRA actually changes, just the texlive-fontspec one, only
these RPMs need to be signed and released via update system:

texlive-fontspec-2008-v1.18.11000.noarch.rpm
texlive-fontspec-doc-2008-v1.18.11000.noarch.rpm
texlive-fontspec-source-2008-v1.18.11000.noarch.rpm


Other situation:
Sometimes happens that binary or all RPMs need a rebuild. What I'm
going to do is to add "1.", "2.", etc. on top of the release of each
package in spec - automatically via the tl2rpm.c which is used to
generate the whole texlive.spec.

With any major TeX Live release (once a year) the version in all
packages is bumped, e.g. 2008 -> 2009.

Any thoughts?

Jindrich

> I'm sure others have questions as well.
> 
> josh

-- 
Jindrich Novy <jnovy at redhat.com>   http://people.redhat.com/jnovy/