> > 3. Report changes in build/update status on a
per-build/per-update
> > basis at every depcheck run
>
> I don't understand this. Is it somehow different from what we already
> do?
Not really. The emphasis was on reporting (to resultsdb, not to bodhi)
everything, every time instead of just reporting what changes may have
triggered the depcheck run.
OK, in that case we understand each other.
> I'm not completely familiar with 'builder' term. I
read the docs [1],
> and I see we have three builds on stage - all, i386 and x86_64 - but
> I'm not sure exactly why. Autotest allowed us to select machines
> depending on arbitrary tags (arch, distribution, virt capabilities,
> etc). I suppose we will need the same with Taskotron. Will we add a
> new builder for every former tag and their combinations, or why
> exactly do we need to solve this on builder level?
>
> [1]
http://docs.buildbot.net/0.8.1/Builder.html#Builder
We will probably need a method for selecting the type of client used
for tasks at some point, yes. At the moment, I don't think we need to
worry about it, though. None of the current tasks require specific
fedora releases and unless I'm mistaken, there are no runtime arch
requirements, either - every task could be run on x86_64 regardless of
what we're checking against.
There is no direct equivalent to autotest tags in buildbot but it is
possible to have builders accomplish most of the same things. A
buildslave can belong to multiple builders and that's how it's
currently set up - i386 slaves are assigned to both the 'i386' and
'all' builders and the x86_64 slaves are assigned to both the 'x86_64'
and 'all' builders.
The reason that I was thinking about it from a builder level was to
handle the incoming requests differently than the "non-fused" builders
if we did the implementation in buildbot. After talking with ralph over
IRC, it sounds like implementing the fuse in fedmsg-hub would be quite
a bit easier and cleaner but a little less transparent for now.
We can also implement it in the trigger itself. Schedule the command with cron and create
some temporary description file/lock file; if other signals come and the command still
hasn't executed, just ignore them.
> Yes, this sounds good. This is the simple concept. This
advanced
> concept would be even better:
> * an incoming signal starts the fuse
> * the fuse runs for X minutes
> * after the job is executed (or finished), another fuse can't be
> started for Y minutes (the next fuse timer X is ignited but frozen
> until Y expires)
>
> With this, we can wait a short time (X) for additional signals (to
> mitigate a problem of several signals coming shortly after each
> other), and then wait a long time (Y) until a new job can be started
> (therefore we mandate some minimum period of time between jobs, to
> lower the load).
>
> But I guess that would be more difficult to implement and the simple
> fuse is good enough for now.
I don't see the advantage of waiting a short time (X) for additional
signals and then waiting a longer time (Y) before scheduling another
job. Wouldn't that be pretty much equivalent to waiting a longer X
between initial signal and actual scheduling (with the exception of the
"hasn't run in a while" case)? I'm not sure we'd get much out of
the
added complexity.
If the system is utilized 100%, then there's no difference. If there are some quiet
times occasionally, then the XY fuse will execute the job faster than the X fuse.
> Just a note - currently, upgradepath is triggered on any Bodhi
update
> stable or testing request. That is not optimal. Optimal is: a) Don't
> trigger on update testing request (until [2] is implemented) b) _Do_
> trigger on any build tagged in Rawhide (i.e. Koji notification)
With the number of rawhide builds that happen, wouldn't that increase
the number of upgradepath runs by at least an order of magnatude?
I was trying to get some numbers (an average number of builds tagged into Rawhide daily),
but it wasn't really simple to do so I haven't invested the time into it. But if
it's important for us to know, I can write a script to query the datagrepper and
filter the results.
I don't think it would increase by an order of magnitude, but it would likely
increase, yes. That is my concern. On the other hand, we already run a couple of tests on
every single Koji build (not just Rawhide) in AutoQA, and we don't seem to suffer a
large performance hit from it.
> For upgradepath, I was thinking about implementing a proper
> per-update checking (rather than whole tag checking). So, if there
> was a new update foo-1.2, I would check just this update and nothing
> else. The execution would be (much) faster, but we would spawn (much)
> more jobs.
>
> It would require some changes in the trigger code (see paragraph
> above) and also we would need to spawn upgradepath for _every single
> new build in Rawhide_ (because that Rawhide build could fix some
> Bodhi update issue in stable releases).
>
> I'm not really sure this is worth it. It's a lot of work and the
> necessity to run upgradepath on every new Rawhide build deters me a
> bit. The test infra load is probably comparable or even higher than
> the fuse-based solution. But the check results would be available
> sooner. For this moment, I would see this as a high priority, even if
> we decide to do it. But I wanted to mention that for upgradepath, a
> different approach is possible (based purely on notifications and
> targeted testing, not testing the whole tag), it's just not a clear
> winner when compared to tag-based testing.
Just to make sure I'm understanding you, it sounds like you're OK with
running upgradepath on an entire tag for now? I'm not against the idea
of changing things up later but I'd like to keep things simple while we
get the initial system and tasks deployed.
Yes, I prefer to keep the current tag-based upgradepath implementation at the moment.