RFE: Running createrepo_c on individual update packages

[Tim Flink]

On Thu, 22 Oct 2015 17:16:03 +0200
Michael Schwendt <mschwendt at fedoraproject.org> wrote:

> On Thu, 22 Oct 2015 08:46:13 -0600, Tim Flink wrote:
> 
> > > How much effort would it be to not only run "rpmlint" on
> > > individual update packages added via bodhi but also
> > > "createrepo_c"?
> > 
> > Assuming that I understand what you're asking, it wouldn't be a lot
> > of effort but it would take enough to make me ask how often this
> > would catch something that wouldn't otherwise be caught.
> 
> Nobody knows. It has happened a few times over a period of years, but
> it has been news to me that the creatrepo* tools create repo metadata
> which make it impossible to detect the problem with tools like
> repoclosure.
> 
> It also depends on what other error conditions createrepo_c can detect
> (and how often it runs into its own bugs).
> 
> Primarily, I'd like to see createrepo_c reject packages it cannot
> handle instead of skipping invalid input data it fails to parse.
> 
> Tomas Mlcoch on devel@ list argues that it is not an option for
> createrepo_c to exit with error condition, because that might be
> exploited by packagers for a denial-of-service attack on the updates
> repo compose process and then would require admins to remove the
> broken packages.

If createrepo_c does/can not exit with an error condition when it runs
into problematic packages, how would we detect when one of these
packages was processed by createrepo_c?

From the devel@ thread, it sounds like createrepo_c should emit log
messages when it runs into invalid epochs but from my local testing
with blktap-3.0.0-3.fc22.git0.9.2, I don't see the error and even if I
did, I don't like the idea of scanning stdout/stderr for an error string
to determine success/failure - it tends to be too fragile. I'm not
saying that that I have a better solution to the problem with the
current constraints, though.

> > I wonder if rpmgrill [1] would end up being a better use of time -
> > I'd love to see that included as a regular task but haven't made
> > the time to get that working yet.
> > 
> > I'm not sure if this particular issue is included in the current
> > battery of things run through rpmgrill but AFAIK, more checks can be
> > added through extending or adding plugins.
> > 
> > Tim
> > 
> > [1] https://github.com/default-to-open/rpmgrill
> 
> Haven't heard about that before.
> 
> rpmlint cannot detect the problem. So far it doesn't "see" the bad
> value at all.
> 
> rpmbuild could detect the problem, but doesn't -> bug 1251453
> 
> repoclosure cannot detect the problem, because it's not copied into
> the repo metadat.
> 
> createrepo_c detects the problem (as part of converting RPM metadata
> into repo metadata), but hides the invalid input data instead of
> rejecting it.

The more I think about this, I'm not against the idea but I don't want
to take resources away from the stuff we're already committed to and
pretty far behind on, timewise.

If you write a task to run createrepo_c on new koji_builds and check
for errors in those runs, we'd be happy to run it in Taskotron after it
passes review by one of the core Taskotron devs.

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.fedoraproject.org/pipermail/qa-devel/attachments/20151022/c7de543c/attachment.sig>