This a status report from our current 0.5.0 testing effort:
1. The tests seem to work properly (except for one Koji timeout and one NoMoreMirrorsRepoError yum error).
2. The number of items in the mailing list increased substantially, because we send one email for the full.log and one email for every (pretty) log created. Rpmlint equals two emails, upgradepath may equal to 10-20 emails. The mailing list is not much human readable now. We can make adjustments if you want. Personally I consider the list as another kind of storage place and I don't really mind.
3. We have found an issue with koji/bodhi watcher that caused every test to be scheduled multiple times. When asking Koji/Bodhi takes really long time, another watcher can be run in parallel. But the subsequent watcher still operates with the old data, thus resulting in scheduling the same tests as the first one. We have seen this issue happening also on our current production instance:
https://fedorahosted.org/pipermail/autoqa-results/2011-June/thread.html - search for q4wine
Jskladan already sent an example patch to tflink to examination, I believe a proper patch will appear on the ML/in git soon if we confirm it fixes the issue.
4. We have identified an issue where our email reduction does not work, it does not send email when it is supposed to (when both depchecks pass for an -updates-testing update). The root cause if not known yet.
I suppose it will take a few days until the problems are identified and fixed.
On Thu, 2011-06-16 at 10:59 -0400, Kamil Paral wrote:
This a status report from our current 0.5.0 testing effort:
The tests seem to work properly (except for one Koji timeout and one NoMoreMirrorsRepoError yum error).
The number of items in the mailing list increased substantially, because we send one email for the full.log and one email for every (pretty) log created. Rpmlint equals two emails, upgradepath may equal to 10-20 emails. The mailing list is not much human readable now. We can make adjustments if you want. Personally I consider the list as another kind of storage place and I don't really mind.
Do you know which emails are sent to maintainers who opt-in for emailed test results? Will they get the overall full.log, or the individual emails for their package?
- We have found an issue with koji/bodhi watcher that caused every test to be scheduled multiple times. When asking Koji/Bodhi takes really long time, another watcher can be run in parallel. But the subsequent watcher still operates with the old data, thus resulting in scheduling the same tests as the first one. We have seen this issue happening also on our current production instance:
https://fedorahosted.org/pipermail/autoqa-results/2011-June/thread.html - search for q4wine
Jskladan already sent an example patch to tflink to examination, I believe a proper patch will appear on the ML/in git soon if we confirm it fixes the issue.
Excellent, nice turn-around!
- We have identified an issue where our email reduction does not work, it does not send email when it is supposed to (when both depchecks pass for an -updates-testing update). The root cause if not known yet.
I suppose it will take a few days until the problems are identified and fixed.
Thanks for the update Kamil. James
On 06/16/2011 09:43 AM, James Laska wrote:
On Thu, 2011-06-16 at 10:59 -0400, Kamil Paral wrote:
This a status report from our current 0.5.0 testing effort:
- The tests seem to work properly (except for one Koji timeout and
one NoMoreMirrorsRepoError yum error).
I see these every once in a while, too and usually attribute them to network glitches. Do you think it would be worth looking into?
- We have found an issue with koji/bodhi watcher that caused every
test to be scheduled multiple times. When asking Koji/Bodhi takes really long time, another watcher can be run in parallel. But the subsequent watcher still operates with the old data, thus resulting in scheduling the same tests as the first one. We have seen this issue happening also on our current production instance:
https://fedorahosted.org/pipermail/autoqa-results/2011-June/thread.html
- search for q4wine
Jskladan already sent an example patch to tflink to examination, I believe a proper patch will appear on the ML/in git soon if we confirm it fixes the issue.
Excellent, nice turn-around!
I applied the code to the test instance and it has been running all day.
- We have identified an issue where our email reduction does not
work, it does not send email when it is supposed to (when both depchecks pass for an -updates-testing update). The root cause if not known yet.
I still haven't been able to reproduce this one so I'm not sure where its coming from. Hopefully I will be able to do so tomorrow.
Tim
On 06/16/2011 09:43 AM, James Laska wrote:
On Thu, 2011-06-16 at 10:59 -0400, Kamil Paral wrote:
This a status report from our current 0.5.0 testing effort:
- The tests seem to work properly (except for one Koji timeout and
one NoMoreMirrorsRepoError yum error).
I see these every once in a while, too and usually attribute them to network glitches. Do you think it would be worth looking into?
We have seen them in the past, it certainly not a regression. Unless they appear really frequently I wouldn't deal with it right now.
- The number of items in the mailing list increased substantially,
because we send one email for the full.log and one email for every (pretty) log created. Rpmlint equals two emails, upgradepath may equal to 10-20 emails. The mailing list is not much human readable now. We can make adjustments if you want. Personally I consider the list as another kind of storage place and I don't really mind.
Do you know which emails are sent to maintainers who opt-in for emailed test results? Will they get the overall full.log, or the individual emails for their package?
Only individual results (pretty logs) are sent to the public, either to Bodhi or to opted-in maintainers. Full.log is used for autoqa-results only.
On 06/16/2011 08:59 AM, Kamil Paral wrote:
- We have found an issue with koji/bodhi watcher that caused every
test to be scheduled multiple times. When asking Koji/Bodhi takes really long time, another watcher can be run in parallel. But the subsequent watcher still operates with the old data, thus resulting in scheduling the same tests as the first one. We have seen this issue happening also on our current production instance:
https://fedorahosted.org/pipermail/autoqa-results/2011-June/thread.html
- search for q4wine
Jskladan already sent an example patch to tflink to examination, I believe a proper patch will appear on the ML/in git soon if we confirm it fixes the issue.
I think that the patch for #342 is pretty much ready to be applied to master. Since this is mostly formatting changes, I did not apply it to the test instance.
- We have identified an issue where our email reduction does not
work, it does not send email when it is supposed to (when both depchecks pass for an -updates-testing update). The root cause if not known yet.
This should be fixed with a little more detail added to the logging in case we run into more issues here (and I assume that we will). I haven't applied this code to the test instance since we don't have comments turned on and it wouldn't be run anyways.
Overall, I think that we're pretty much ready to ship. I'm sure that we're going to hit bugs that we didn't see before but hopefully nothing big.
Thoughts?
Tim
autoqa-devel@lists.fedorahosted.org
-
James Laska -
Kamil Paral -
Tim Flink