#228: Rewrite post-bodhi-update watch to rely on koji -pending tags
----------------------------+-----------------------------------------------
Reporter: jlaska | Owner:
Type: task | Status: new
Priority: major | Milestone:
Component: infrastructure | Version: 1.0
Keywords: |
----------------------------+-----------------------------------------------
Per discussion from [https://fedorahosted.org/pipermail/autoqa-
devel/2010-September/001175.html autoqa-devel meeting], the ''post-bodhi-
update'' watcher will eventually need to be replaced by a watcher that
monitors koji ''-pending'' tags instead.
The current implementation is deemed fragile, and may no longer function
properly after bodhi-2.0 upgrade.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/228>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
#205: make depcheck submit karma/comments to bodhi
--------------------+-------------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
Modify the depcheck test so that it sends status info - karma, comments,
whatever - to bodhi.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/205>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
#201: Add mash support to depcheck
-----------------------+----------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck
Component: docs/wiki | Version: 1.0
Keywords: |
-----------------------+----------------------------------------------------
To properly handle possible multilib problems (e.g. the
[https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000703.html
nss-softokn problem] in June and the recent gfortran problem) we need to
run mash on the set of proposed packages, to generate a proper
multilibified repo to test against.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/201>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
#153: rpmguard: Separate checks into individual python files
-------------------------+--------------------------------------------------
Reporter: kparal | Owner:
Type: enhancement | Status: new
Priority: major | Milestone:
Component: tests | Version: 1.0
Keywords: rpmguard |
-------------------------+--------------------------------------------------
It would be great to have individual checks in rpmguard available as
separate files, very similarly to what rpmlint or yum (-plugins) use. It
would allow easy enabling/disabling of specific check, it would allow easy
contribution of a new check.
More ideas and information here: https://fedorahosted.org/pipermail
/autoqa-devel/2010-May/000496.html
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/153>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
#150: "artificial ignorance": suppress errors and warnings that aren't important
-------------------------+--------------------------------------------------
Reporter: dmalcolm | Owner:
Type: enhancement | Status: new
Priority: major | Milestone: Package Sanity Tests
Component: tests | Version: 1.0
Keywords: |
-------------------------+--------------------------------------------------
There are some interesting ideas in here about suppressing a logfile to
show up the information that's truly important:
http://www.ranum.com/security/computer_security/papers/ai/index.html
I think that the rpmlint test needs to have a set of filters, and should
suppress output matching certain patterns. For example
https://fedorahosted.org/pipermail/autoqa-results/2010-April/013903.html
shows:
{{{
gnome-python2-gnome.i686: W: spelling-error Summary(en_US) libgnome -> lib
gnome, lib-gnome, cognomen
}}}
i.e. it's complaining about a reference to "libgnome" within gnome-
python2-gnome, which is clearly a false-positive.
For this case we might have a python filtering process that says something
like this (caveat: untested):
{{{
patterns = [
": W: spelling-error \S+ libgnome .*",
]
def should_ignore(line):
for pat in patterns
# May want to precompile the patterns:
m = re.match(pat, line)
if m:
return True
# No match:
return False
for line in rpmlint_output
if not should_ignore(line):
print(line)
}}}
and then gradually grow patterns. The idea is to make it nice and
lightweight, and Fedora QA can then easily take ownership of the
suppresssions, without having to get patches into rpmlint.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/150>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
#138: ResultsDB: media wiki as a storage for metadata about tests
----------------------------+-----------------------------------------------
Reporter: jskladan | Owner:
Type: task | Status: new
Priority: major | Milestone: Resultdb
Component: infrastructure | Version: 1.0
Keywords: |
----------------------------+-----------------------------------------------
Once #135 #136 and #137 are finished, we should create a provider/middle
man (probably a library) which will allow us to get information about the
respective test and use it for test execution/storing results/displaying
results via frontends...
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/138>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
#137: ResultsDB: propose structure for storing metadata about tests on wiki
-----------------------+----------------------------------------------------
Reporter: jskladan | Owner:
Type: task | Status: new
Priority: major | Milestone: Resultdb
Component: docs/wiki | Version: 1.0
Keywords: |
-----------------------+----------------------------------------------------
We want to use mediawiki as a storage for tests metadata. We should agree
on the fields we want to store (e.g. test owner, destructive/non-
destructive, average time to complete test etc.) and the format we'll use
to store it (probably JSON though).
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/137>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
#231: upgradepath: ask FESCo how to deal with updates-testing
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: task | Status: new
Priority: major | Milestone: Package Update Acceptance Test Plan
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
This is a summary of problems with upgradepath constraint when applied to
updates-testing repos:
https://fedorahosted.org/pipermail/autoqa-devel/2010-September/001189.html
We should ask FESCo what the desired solution is.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/231>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project