It seems like it would be useful if we could have a way to connect "I'm making an update in bodhi!" back to "This update is likely to affect a given release criteria". For example, it would be nice to have something show up in bodhi noting that updates to Firefox might affect https://fedoraproject.org/wiki/QA:Testcase_desktop_browser and even better, if that would also show up for updates to Firefox's dependencies.
The idea here is to help developers and packagers be more mindful of the release criteria that their updates might impact. Does this sound in theory desirable? Can you think of a way to do it other than listing specific packages in the test cases? I know that there's some tension between this and the generic broad expression used for many test cases.
On Fri, 2017-07-07 at 12:24 -0400, Matthew Miller wrote:
It seems like it would be useful if we could have a way to connect "I'm making an update in bodhi!" back to "This update is likely to affect a given release criteria". For example, it would be nice to have something show up in bodhi noting that updates to Firefox might affect https://fedoraproject.org/wiki/QA:Testcase_desktop_browser
We can already link packages and test cases such that updates that include those packages will always list those test cases:
https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation
I believe there's possibly some kind of bug with the process of syncing the wiki categories into Bodhi at present, I'm not totally sure of the current status of that, but the mechanism is already there.
and even better, if that would also show up for updates to Firefox's dependencies.
This isn't done yet. (And honestly, doing it in a simple-minded way would be a bit tricky; do we really want, say, every update to glibc listing pretty much every package-linked test case that exists?)
The idea here is to help developers and packagers be more mindful of the release criteria that their updates might impact. Does this sound in theory desirable? Can you think of a way to do it other than listing specific packages in the test cases? I know that there's some tension between this and the generic broad expression used for many test cases.
There really wouldn't be any 'tension' in adding the browser test case to the firefox package categories, so we could just go ahead and do that. Establishing links to *release criteria* - which is what you keep saying, but then your example involves a test case and not a release criterion, which confuses me a bit - is harder, because we don't really have any representation of the release criteria which is easy to interact with programmatically...
On Mon, Jul 10, 2017 at 01:58:10PM -0700, Adam Williamson wrote:
It seems like it would be useful if we could have a way to connect "I'm making an update in bodhi!" back to "This update is likely to affect a given release criteria". For example, it would be nice to have something show up in bodhi noting that updates to Firefox might affect https://fedoraproject.org/wiki/QA:Testcase_desktop_browser
We can already link packages and test cases such that updates that include those packages will always list those test cases:
Yes. This is cool.
https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation I believe there's possibly some kind of bug with the process of syncing the wiki categories into Bodhi at present, I'm not totally sure of the current status of that, but the mechanism is already there.
Also I have a request to make a drop-in-files thing which I hope will be easier for packagers than dealing with wiki categories, which seems to be a barrier.
and even better, if that would also show up for updates to Firefox's dependencies.
This isn't done yet. (And honestly, doing it in a simple-minded way would be a bit tricky; do we really want, say, every update to glibc listing pretty much every package-linked test case that exists?)
No. :) I don't really mean test cases; I mean release criteria and I just followed links too far when making an example above.
There really wouldn't be any 'tension' in adding the browser test case to the firefox package categories, so we could just go ahead and do that. Establishing links to *release criteria* - which is what you keep saying, but then your example involves a test case and not a release criterion, which confuses me a bit - is harder, because we don't really have any representation of the release criteria which is easy to interact with programmatically...
More wiki categories? :)
On Tue, 2017-07-11 at 13:59 -0400, Matthew Miller wrote:
On Mon, Jul 10, 2017 at 01:58:10PM -0700, Adam Williamson wrote:
It seems like it would be useful if we could have a way to connect "I'm making an update in bodhi!" back to "This update is likely to affect a given release criteria". For example, it would be nice to have something show up in bodhi noting that updates to Firefox might affect https://fedoraproject.org/wiki/QA:Testcase_desktop_browser
We can already link packages and test cases such that updates that include those packages will always list those test cases:
Yes. This is cool.
https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation I believe there's possibly some kind of bug with the process of syncing the wiki categories into Bodhi at present, I'm not totally sure of the current status of that, but the mechanism is already there.
Also I have a request to make a drop-in-files thing which I hope will be easier for packagers than dealing with wiki categories, which seems to be a barrier.
Erf. At some point I have to push back against all the wiki hate. The wiki basically *is* a bunch of text files, after all. I'm frankly refusing to believe that typing this into a test case page:
[[Category:Package_foobar_test_cases]]
is too difficult for people to handle, and that some random text file somewhere (yet *another* process to document and for people to forget about keeping working) would be better.
and even better, if that would also show up for updates to Firefox's dependencies.
This isn't done yet. (And honestly, doing it in a simple-minded way would be a bit tricky; do we really want, say, every update to glibc listing pretty much every package-linked test case that exists?)
No. :) I don't really mean test cases; I mean release criteria and I just followed links too far when making an example above.
There really wouldn't be any 'tension' in adding the browser test case to the firefox package categories, so we could just go ahead and do that. Establishing links to *release criteria* - which is what you keep saying, but then your example involves a test case and not a release criterion, which confuses me a bit - is harder, because we don't really have any representation of the release criteria which is easy to interact with programmatically...
More wiki categories? :)
Well, no, that wouldn't work, as we don't have one wiki page per criterion. I did sort of intend that the names of the anchor links in the criteria pages (each criterion has an anchor link) should never change and can reliably be used to refer to 'the same criterion' (even if we change the wording of the criterion), but we've never really made that an official thing, it's mostly just in my head so far. tflink has long had ideas about somehow handling the release criteria through a different system (possibly blockerbugs), but never had time for it, I don't think.
On Tue, Jul 11, 2017 at 11:17:02AM -0700, Adam Williamson wrote:
Also I have a request to make a drop-in-files thing which I hope will be easier for packagers than dealing with wiki categories, which seems to be a barrier.
Erf. At some point I have to push back against all the wiki hate. The wiki basically *is* a bunch of text files, after all. I'm frankly refusing to believe that typing this into a test case page:
[[Category:Package_foobar_test_cases]]
is too difficult for people to handle, and that some random text file somewhere (yet *another* process to document and for people to forget about keeping working) would be better.
I'll put the relative dearth of test cases using the wiki model as exhibit A. And, the category thing isn't _really_ that simple: https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation#Simple_.28r... is *definitely* the level to which someone who isn't working with mediawiki categories and probably our SOP for this in general might just say... never mind. And I'm not even talking about the "Advanced (optional) categorization".
More wiki categories? :)
Well, no, that wouldn't work, as we don't have one wiki page per criterion. I did sort of intend that the names of the anchor links in the criteria pages (each criterion has an anchor link) should never change and can reliably be used to refer to 'the same criterion' (even if we change the wording of the criterion), but we've never really made that an official thing, it's mostly just in my head so far. tflink has long had ideas about somehow handling the release criteria through a different system (possibly blockerbugs), but never had time for it, I don't think.
blockerbugs as in living in (or at least exposed through) that app, or actually in bugzilla as trees of dependencies?
But anyway, the anchor links are a *little* fragile, but maybe could work. But why _not_ have one page per criterion? (Transcluded in one big page, if that's important for convenience?)
On Tue, 2017-07-11 at 14:46 -0400, Matthew Miller wrote:
On Tue, Jul 11, 2017 at 11:17:02AM -0700, Adam Williamson wrote:
Also I have a request to make a drop-in-files thing which I hope will be easier for packagers than dealing with wiki categories, which seems to be a barrier.
Erf. At some point I have to push back against all the wiki hate. The wiki basically *is* a bunch of text files, after all. I'm frankly refusing to believe that typing this into a test case page:
[[Category:Package_foobar_test_cases]]
is too difficult for people to handle, and that some random text file somewhere (yet *another* process to document and for people to forget about keeping working) would be better.
I'll put the relative dearth of test cases using the wiki model as exhibit A. And, the category thing isn't _really_ that simple: https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation#Simple_.28r... is *definitely* the level to which someone who isn't working with mediawiki categories and probably our SOP for this in general might just say... never mind. And I'm not even talking about the "Advanced (optional) categorization".
I suppose I could just put a tl;dr infobox at the top, as the executive summary *is* pretty simple.
More wiki categories? :)
Well, no, that wouldn't work, as we don't have one wiki page per criterion. I did sort of intend that the names of the anchor links in the criteria pages (each criterion has an anchor link) should never change and can reliably be used to refer to 'the same criterion' (even if we change the wording of the criterion), but we've never really made that an official thing, it's mostly just in my head so far. tflink has long had ideas about somehow handling the release criteria through a different system (possibly blockerbugs), but never had time for it, I don't think.
blockerbugs as in living in (or at least exposed through) that app, or actually in bugzilla as trees of dependencies?
As in the app.
But anyway, the anchor links are a *little* fragile, but maybe could work. But why _not_ have one page per criterion? (Transcluded in one big page, if that's important for convenience?)
It's more work than writing one page, I guess? I don't think anyone ever made a specific decision to do it that way, it's just how it naturally happened.
-
Adam Williamson -
Matthew Miller