Responsibility for rebuilding dependent components, was: F-16 Branched report: 20110920 changes

[Jesse Keating]

On Sep 20, 2011, at 12:19 PM, Doug Ledford wrote:
> ----- Original Message -----
>> This is essentially what we had a while ago, only with trac tickets
>> instead of bodhi requests.
> 
> Bodhi is definitely a better place to track this stuff, regardless of how decisions are made.
> 
>> There were a couple of problems with
>> this.
>> 
>> 1) Nowhere near enough releng folks to properly review all the
>> tickets.
> 
> Fair enough.
> 
>> 2) 9 times out of 10 there was very little data put into the ticket.
> 
> Multiple options here.  Kick back incomplete tickets, or the better option IMNSHO, run rpmdiff runs between the package currently in the compose and the one in testing to check for various failures and require the developer to justify failures.

There was supposed to be AutoQA filling this role here, catching obvious issues, sanity checking, and letting things through that looked OK.  We're still waiting for autoQA.  There certainly wasn't enough releng man power/volunteers to manually look through this for every potential update.

> 
>> 3) releng folks were often not the best people to decide whether a
>> change was "too risky"
> 
> The rpmdiff option above would help with this.
> 
>> 4) There was no easy way to get at the package and assess its
>> stability.
> 
> Using bodhi instead of trac solves this, no?

No, what was missing was a repository with yum metadata to get at the package and any sibling packages that needed to come with it for a comprehensive picture of what was going on.

> 
>> I think there were more issues, but those come to mind first.  We
>> decided it was best instead to make a repository out of proposed
>> changes,
> 
> But in practice that's not really what updates-testing on the early branched release really is.  It's a repo all right, but not of proposed changes, it's a repo of packages, and getting to the actual changes versus the final package would require installing the current source rpm, the new source rpm, then doing a manual inspection for changes.  An automated rpmdiff run would be a *far* superior means of presenting the proposed changes to the community.

It depends on what you're after.  If you just want a list of changes, sure, but if you want some idea as to whether or not those changes introduce instability then you need to look more comprehensively.  It is rather difficult to look at a small list of changes and gauge how well/ill it will effect the stability of the distribution going forward.  Either you're too liberal and we have rampant instability, or you're too draconian and we have very strict barriers to entry, which are suddenly removed once a product goes GA.  Using the same criteria prior to and post GA to assess the validity of an update makes sense to me.

> 
>> and let your packaging peers decide whether or not the
>> update was safe enough to go into the release,
> 
> But they can't, not really.  For instance, a proventester may install my mdadm package, but if they don't have a raid array, they can't decide anything.  Where as if they had access to the code diffs and could tell that all I did was change a typo in the udev rules file, and verify for themselves via code inspection that the code as it stands in the repo can't work and the fix should work, then they could make an educated contribution.

Sadly we have far far too few real developers who could do that comparison.

>  Because the testing repo doesn't really present changes, but only a final product, there is no possibility for my peers to look at my changes and say "Yeah, that's should be a safe change, let it in, and if it breaks then we'll fix it".  So the judgement call that I mentioned in my previous email is taken entirely out of the loop, and there is no ability for my peers to make any contribution to whether or not a package should be allowed in *except* unit testing, there is no ability for them to easily do an actual analysis of the changes and ma
> ke an engineering decision versus a QA decision.

I think that's largely because we don't have a community of engineers.  We have a community of /packagers/ who are able to cause packages to be built, and are able to do some measure of QA to see if those builds work, but do not have the skill set to look at a code diff and give a honest opinion as to whether or not the change is "safe".

If the makeup of our community were different, then you would have a case for your proposal.  I just don't believe we have the community it would take to accomplish it on a large scale.

> 
>> thus we have bodhi
>> and updates-testing as a gateway to get into the release.
> 
> It's a gateway, I just don't think it serves as useful a purpose as it was intended to.


The question though really is whether or not it is more useful than a few (like 4) release engineers looking at trac tickets and making best guesses depending on whatever the ticket reporter put in the ticket about the change.  We were trying to improve the workflow, not perfect it.  Has it been improved?

- jlk