On Thu, 2 Jun 2016 07:46:33 -0400 (EDT)
Kamil Paral <kparal(a)redhat.com> wrote:
> Hi,
>
> we have deployed task result namespaces support a while ago and put
> our checks (depcheck, upgradepath, rpmlint) into the 'qa' namespace.
> With newly added tasks like task-abicheck and task-dockerautotest,
> we weren't sure where to put them and so we used the scratch
> namespace which is supposed to be used for testing purposes. Now
> with those "3rd party" tasks deployed to production, we need to
> decide what namespaces should they and other future tasks belong in.
>
> The starting namespaces, and maintainers of tasks belonging to that
> namespace, follows:
> qa.* - Fedora QA
> pkgs.<pkgname>.* - <pkgname> maintainers
> fas.<username>.* - <username>
> fasgroup.<groupname>.* - fas members belonging to <groupname>
> scratch.* - anyone
>
> Now, since we maintain task-dockerautotest, it should go into
> qa.*,
Agreed. Once we have dist-git checks implemented, we can ask docker
maintainers to keep it and maintain it in their repo. If they don't
want, we can keep it in our domain (same as anyone will be able to
run any test against any package in their personal space).
It seems really weird to have one effectively package-specific in qa.*
but there are worse things.
> I am not sure though where does task-abicheck belong. I see
three
> options here:
> 1. we can create fas group and put it there,
More on this below.
> 2. create additional dist.* namespace where tasks like abicheck
> and/or rpmgrill would be, or
What is the use case for "dist."? Which tasks would go there and
which would not? Is this just about "scratch" not sounding that good?
I'm pretty anti-fas-name on the release blocking checks (will detail
below) so in my mind, dist.* is a good place to put things which need
to be run on all the packages/updates/composes etc. - something that is
pretty much global. Other ideas on how to split that stuff up are
welcome, though.
Having scratch.* be the catch-all also allows us to say "scratch.*
probably isn't important" while having another option for where to put
results. I think of it as kinda similar to the concept of a scratch
build in koji.
> 3. change semantics of qa.* to 'anything Fedora QA maintains
or
> important, not package-specific, tasks".
Personally, I always considered our namespaces to have two functions:
1. show ownership - it's clear who owns qa.depcheck, or
fas.ksinny.abicheck, or group.workstation.gnome-startup
Why is it important to show ownership? In some cases (mostly
non-release-blocking stuff), I can kind of see a case for this but for
abicheck in particular, I don't see the point. We don't encode the
maintainer(s) into package names for fedora, why should we for
checks/tests/tasks?
From a more (perhaps overly) cynical PoV, I suspect that we're going to
get the lion's share of complaints about failing tasks, no matter what
the namespace happens to be.
I don't really have an issue with the group names. If that name changes
or the task is handed off to some other group, I think that there will
be bigger changes afoot where changing filters etc. aren't the biggest
concern.
FWIW, I don't remember the concept of ownership coming up in the
original discussions of namespaces - mostly "security" and being able
to tell which results are more important in terms of release blocking
vs. package blocking.
2. increase
security and eliminate mistakes - by allowing certain people or
groups to write results only into their own namespace, we reduce
attack vectors (random people can't send fake results for
qa.depcheck) and we eliminate mistakes (people will not create a
thousand test cases in "qa." by accident)
I think this is the primary use case for namespaces, honestly.
I didn't intend namespaces to reflect task importance in any
way,
e.g. to put all gating tasks into "qa.". It sounds tempting, but it's
likely to violate the two functions mentioned above. For release
critical tasks, it's possible that we will take their ownership and
maintain them, but I don't think it's going to happen for all of
them. I don't see a problem in Bodhi or some releng tools monitoring
qa.depcheck, qa.upgradepath and fas.ksinny.abicheck. It's all about
trust, and the exact testcase name doesn't matter. You can't trust a
random person that he/she will maintain the task and quickly fix all
issues, but once you know who you're dealing with and that he/she is
willing and capable of doing that work, it doesn't matter whether the
namespace is "qa." or "fas.".
Let's pretend that everyone who maintains task-abicheck decides to go on
vacation for a month. One week into that month, task-abicheck breaks
horribly and is no longer producing results that we can believe. If
we're relying on results from task-abicheck to do things like gate
non-rawhide builds moving into stable tags, I really don't think that
the attitude will be "oh well, none of the owners are around, I guess
we'll just turn off this check for a couple of weeks or not allow
anything to go stable until the maintainers return".
For abicheck in particular, I think that once it's integrated into
various Fedora workflows, it's going to stay there unless libabigail
stops being maintained for some reason and breaks horribly. We're going
to be on the hook for that task, regardless of whether its in the qa.*
namespace or not - I'm not saying that I think that something will
happen, just using it as an example.
I don't much care what the namespace ends up being, so long as it's not
tied to a fas username and makes sense. Any suggestions?
Of course there's the issue of people coming and going, and it
would
be nice to have the testcase name changing as little as possible. For
that reason, I imagine that really important tasks will move to some
group ownership, where people can maintain it and the namespace
doesn't need to change. Let's imagine "group.workstation" or
"group.dnf" (which can match fas groups, or be created manually, or
both - the implementation is up to discussion).
The primary issue I have with fas usernames as namespaces is the
churn that can represent. If task-abicheck reported to
fas.ksinny.abicheck, what happens if ksinny hands of maintenance to
someone else for whatever reason? Do we change the namespace for that
check just because the maintainer changes? That would have cascading
effects in notifications, filtering of fedmsgs/notifications by all
users and any systems which query resultsdb for abicheck results.
Once there's a namespace set for a certain task, that namespace
shouldn't change unless there's a _really_ good reason and I don't
think that change-of-maintainer is a good enough reason.
If we do go the group/area route, we're going to have to figure out how
the group names/membership will be maintained.
So, personally, what I would *not* do is to place task-abicheck into
"qa." namespace while having Dodji or Sinny still maintain it. That
breaks the two primary rules of namespaces (as I see it). We should
either move it to "qa." and start maintaining it ourselves, or put it
into Sinny's/Dodji's personal namespace and let them maintain it.
This is nothing personal against Sinny or Dodji (you two are doing
great work), and it's not that I don't trust them, I'm using a
concrete example but trying to show the principle in general.
The primary issue I have with fas usernames as namespaces is the
churn that can represent. If task-abicheck reported to
fas.ksinny.abicheck, what happens if ksinny hands of maintenance to
someone else for whatever reason? Do we change the namespace for that
check just because the maintainer changes? That would have cascading
effects in notifications, filtering of fedmsgs/notifications by all
users and any systems which query resultsdb for abicheck results. Once
there's a namespace set for a certain task, that namespace shouldn't
change unless there's a really good reason and change-of-maintainer
isn't a good enough reason.
I don't really care if abicheck is in the qa.* namespace or not,
though. It just seemed like a decent place and it's not like we
wouldn't be on the hook if the proverbial poo hit the fan.
The only thing is that "fas." is not implemented yet and
that's why
it is in "scratch." now. I don't see a problem with it, and we can
move it once we implement "fas.". Once abicheck proves to be useful
and reliable (or before, once we agree on what we really want), we
can suggest them to come up with some maintenance fas group so that
the namespace doesn't need to change, in case they want somebody else
to take it over.
I could be wrong, but I have a hard time seeing a huge use case for fas
username namespaces. If contributors write tasks, I really think
they're going to be for a group/package/image/etc. and using scratch.*
for things that don't have a group or other functional namespace works
well enough.
Of course you might have a different opinion on what the namespaces
are useful for, and what are their important a unimportant features.
Please feel free to disagree with me and present your views. I do not
feel strongly about this, I was just trying to describe what I
currently see as the best (safest, least error-prone, least
maintenance) way forward. Thanks.
It feels like we're starting to have another of those huge discussions
that isn't worth a huge discussion but hopefully it's just a lot of
words around a similar goal :)
From my PoV, the things I'd fight for are:
- fas username namespaced tasks cannot ever be release blocking
- anything release-blocking should have a pretty much immutable
namespace and name
- making sure that enough people can change release blocking tasks
that we don't hit a bus number problem on task ACL
- keep things as consistent as we can so we can avoid future
incidents of "oh, by the way, if you have any filters for
results ... you'll want to change those"
The things that I'd like to see but don't care about so much:
- no fas username namespacing at all
- namespace checks by functional use/area rather than "owner" even
though that would be similar in many cases
PS: There's one case where I can imagine we should do a
different
solution. If we wanted to show abicheck results in Bodhi *asap* and
were concerned about the risks of displaying results from "scratch."
namespace, then it would make sense to create some separate temporary
namespace, place abicheck into in (and set up permissions so that
it's not publicly accessible) and have it there until we implement
"fas.".
This involves changing the namespace and I don't think that's a good
idea. Let's change it once and (hopefully) only once.
Tim