Noarch subpackage problem

[Dennis Gilmore]

On Tuesday 24 February 2009 03:59:17 pm Mike Bonnet wrote:
> Toshio Kuratomi wrote:
> > So we had a discussion on IRC today about the failure cases of noarch
> > subpackages.  I think we should make some changes to the way we check
> > that noarch subpackages are sane.
> >
> > Currently, when a noarch subpackage is built, rpmdiff is run on the
> > noarch packages that were built by each builder.
> >
> > Of the checks that rpmdiff does, we discard all of them except Provides,
> > Requires, and the list of files.  My concern is that if you throw out
> > md5sum and filesize in these checks there's a lot of margin for creating
> > subpackages that are not actually noarch.
>
> Actually the full list of tags we diff are:
>
> name
> summary
> description
> group
> license
> url
> prein (script)
> postin (script)
> preun (script)
> postun (script)
>
> For Requires, Provides, Conflicts, and Obsoletes we check that the lists
>   (including versions) are identical across all subpackages.
>
> We verify that that file lists are identical across all subpackages, and
> for each file in the file lists we verify that the following attributes
> are identical:
>
> mode
> flags
> nlink
> state
> vflags
> user
> group
>
> > For instance, if bitedness ends up in include files that are placed in a
> > noarch subpackage, those subpackages won't be caught by this check.
> > That would allow a package to go out that could prevent building with
> > the incorrect header.
> >
> > The reason that filesize and md5sum are discarded is that
> > arch-inspecific files can have timestamps embedded into them at build
> > time.  This means, for instance, that documentation can differ between
> > builds of a subpackage despite it being a prime candidate for a noarch
> > subpackage.
> >
> > An idea for a change would be to extend rpmdiff to be able to list
> > changes in md5sum between all files except those marked as %doc.  This
> > would let documentation packages through even if timestamps were
> > embedded but not let a noarch package with differing headers through,
> > for instance.
a docs subpackage likely doesn't mark every file as %doc  if those docs are 
created at build time they will always vary across arches. some firmware blobs  
are created at build time.  not every piece of content that can be considered 
noarch can be guaranteed to have been built prior to build time. things like 
bios's for qemu. some of the kernel-firmware is built from source. and some is 
extracted from source.  there are alot of valid reasons why a noarch 
subpackage will have differing timestamps/md5's just as there is cases where 
differences should not occur.  differences in pkg makeup though are certain 
failure cases.  for instance one arch produces a different set of files. or 
different provides/requires.

> Another class of files that are noarch-but-different are .pyc/.pyo
> files, as Julian Sikorski found out:
>
> https://www.redhat.com/archives/fedora-devel-list/2009-February/msg01826.ht
>ml
>
> Issues like this, where files differed because of embedded
> timestamps/hostnames/etc. but were not different in any meaningful way
> came up during testing of this feature.  As a result it was decided to
> not fail a build due to differences in file size, digest, or mtime
> because this would have resulted in a lot of false positives, and
> significantly reduced the usability and usefulness of this feature.
>
> The automated checks are not a replacement for diligent package review
> and testing, they are there to help package maintainers catch silly
> mistakes and oversights.  What we have now is a good balance between
> catching those oversights and not burdening maintainers with a huge
> number of false positives that (as in the python case above) they are
> unable to do anything about and thus unable to make use of this feature.
>
> Note that this feature is in no way more dangerous or prone to error
> than the existing method of creating noarch subpackages (extra-arches).
these automated checks give us something additional that is not available when 
making noarch packages of any other type. and are certainly not intended to be 
a complete and perfect replacement for package reviews.

Dennis