On 04/14/2011 04:02 AM, Josef Skladanka wrote:
= Depcheck workflow =
After discussing some issues with Kamil, we decided, that it would be for the best, if we wrote down the intended Depcheck results usage & workflow.
The problem is tightly connected to #306. The problem described in #306 means this:
== Premise ==
"Updates can be changed in Bodhi." - the evidence is clear https://admin.fedoraproject.org/updates/control-center-3.0.0-1.fc15,gnome-settings-daemon-3.0.0.1-1.fc15, if you look at the bodhi comment from 2011-04-06 15:05:38 "hadess has edited this update..."
== Implications ==
=== Timeline ===
- All updates in -pending passed depcheck, and have "PASSED"
comment. 2) Update XYZ is changed by the developer
=== Consequences ===
Because one of the "already passed" updates is changed, we should discard the whole "passed" subset, because we can not be sure, that some errors were not introduced by the change.
== Level 1 Solution ==
#306 describes the 'simple' solution. When we search for the 'accepted' updates, we look for all the updates, which do have "PASSED depcheck" comment in bodhi.
We want to detect, whether the update was changed after the "PASSED depcheck" comment, and if so, then remove it from accepted set, and basicaly retest it.
I suppose that the easiest solution would be to parse the comments on an update as a whole (instead of one-by-one) and sort by date. If there was an update comment after the 'PASSED' comment, is_bodhi_commented() returns false.
I wonder how hard it would be to change the way depcheck and bodhi work a little bit.
Instead of just scraping comments, another option would be to have a text field in bodhi that held the list of passing tests. When a test passes, it adds its name to the field. When an update is changed, that field is reset and the tests are re-run.
It might make more sense to wait for resultsdb on this one, I'm not sure. Either one but might be more accurate and faster than trying to interpret comments.
Do we know how often this is happening?
== Level 2 Solution ==
Level 1 solution does not really solve the problem, as described, even though it is certainly needed to detect the 'passed but changed', and will sort out some of the problems.
The problem with the fact, that change in one update can possibly break some 'previously passed' dependency problems is still there.
Because it would be incredibly hard to detect just the updates influenced by changes in the respective update, we decided to define a workflow, which bypasses the overall need for this.
=== RelEng workflow ===
At the moment, rel-eng is pushing updates manualy few times a week [citation needed, this is informal information which has not been audited].
The thing is, that we can provide the _at the moment 'pushable'_ subset of packages. When rel-eng will want to push from -pending to {stable, updates, ...}, they'll just 'list' the pushable subset (yes, resultsdb will help ;-)).
"The tool" will detect if there were any changes in the pushable subset, and "the tool" will inform rel-eng that they either need to wait for the results of the next depcheck run, or (if we're able to do it) request new depcheck run.
Why not just make a tool that re-reruns all "*-pending" updates through a battery of tests (depcheck and upgradepath ATM), ignoring any "PASSED" results and not posting any new comments to bodhi unless there was a change for an update? That way we don't have to worry about detecting if there were changes or not and we can be more confidant of the to-be-pushed set as a whole (which is what our end goal is, IIRC)
One easy way to do this for the short term is to have some sort of CLI or webpage that does a "pre-push" test suite and runs upgradepath and depcheck (more later, maybe) on everything that is about to be pushed, regardless of what tags and comments already exist.
This could be done locally but might be more productive to push a job to autotest and have it run on the test machines so that whoever is doing the pushing doesn't have to download all the packages.
Tim