[AutoQA] #125: initscripts: Choosing the right repo for dependencies
by fedora-badges
#125: initscripts: Choosing the right repo for dependencies
-------------------------+--------------------------------------------------
Reporter: kparal | Owner: jskladan
Type: task | Status: new
Priority: major | Milestone:
Component: tests | Version: 1.0
Keywords: initscripts |
-------------------------+--------------------------------------------------
Currently after new Koji build the initscripts test downloads all relevant
binary packages (related to the particular source package) and installs
them. But the RPM dependencies are installed from stable/updates repos,
not from Koji. Also the dependencies parsed out of Makefile are installed
from stable/updates repos.
We have to inquire a little more into this situation. Is it
sufficient/desirable? Or is it desirable to download dependencies from
updates-testing or bleeding edge Koji? May the initscripts test sometimes
fail because of insufficient dependencies are in stable/updates repos?
This also depends on cases when we are going to use initscripts test. If
it will be used with Bodhi hook, it will be probably easier to solve it
than when used with Koji hook.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/125>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 6 months
[AutoQA] #102: post-iso-build hook
by fedora-badges
#102: post-iso-build hook
----------------------+-----------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: Automate installation test plan
Component: watchers | Version: 1.0
Keywords: |
----------------------+-----------------------------------------------------
We need one (or more) standard locations where new iso images get posted
for testing, and a watcher that can monitor for new iso images and launch
appropriate installation/sanity tests.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/102>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
[AutoQA] #149: rpmlint: Provide whitelisting of messages
by fedora-badges
#149: rpmlint: Provide whitelisting of messages
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: task | Status: new
Priority: major | Milestone: Package update tests
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
We need to provide some whitelisting capability of rpmlint packages, so
the package maintainers can mark some messages as invalid. We will also
probably want to mark some messages as invalid by default for the whole
Fedora project.
As an interesting inspiration, look at lintian project (validation of deb
packages). It provides some "overrides" on a package basis, you just
create a definition file in /etc/lintian/package_name or kind of. I have
the feeling that rpmlint provides something similar, but we need to
discover it more properly.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/149>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
[AutoQA] #104: virtguest.py: accept iso image(s) as install location
by fedora-badges
#104: virtguest.py: accept iso image(s) as install location
--------------------+-------------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: Automate installation test plan
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
allow {{{VirtGuest.create()}}} to use an iso image (or images) for its
{{{location}}} arg, and automatically choose the appropriate {{{--location
URL}}} or {{{--cdrom IMAGE}}} flag.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/104>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
[AutoQA] #150: "artificial ignorance": suppress errors and warnings that aren't important
by fedora-badges
#150: "artificial ignorance": suppress errors and warnings that aren't important
-------------------------+--------------------------------------------------
Reporter: dmalcolm | Owner:
Type: enhancement | Status: new
Priority: major | Milestone: Package Sanity Tests
Component: tests | Version: 1.0
Keywords: |
-------------------------+--------------------------------------------------
There are some interesting ideas in here about suppressing a logfile to
show up the information that's truly important:
http://www.ranum.com/security/computer_security/papers/ai/index.html
I think that the rpmlint test needs to have a set of filters, and should
suppress output matching certain patterns. For example
https://fedorahosted.org/pipermail/autoqa-results/2010-April/013903.html
shows:
{{{
gnome-python2-gnome.i686: W: spelling-error Summary(en_US) libgnome -> lib
gnome, lib-gnome, cognomen
}}}
i.e. it's complaining about a reference to "libgnome" within gnome-
python2-gnome, which is clearly a false-positive.
For this case we might have a python filtering process that says something
like this (caveat: untested):
{{{
patterns = [
": W: spelling-error \S+ libgnome .*",
]
def should_ignore(line):
for pat in patterns
# May want to precompile the patterns:
m = re.match(pat, line)
if m:
return True
# No match:
return False
for line in rpmlint_output
if not should_ignore(line):
print(line)
}}}
and then gradually grow patterns. The idea is to make it nice and
lightweight, and Fedora QA can then easily take ownership of the
suppresssions, without having to get patches into rpmlint.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/150>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 5 months
[AutoQA] #138: ResultsDB: media wiki as a storage for metadata about tests
by fedora-badges
#138: ResultsDB: media wiki as a storage for metadata about tests
----------------------------+-----------------------------------------------
Reporter: jskladan | Owner:
Type: task | Status: new
Priority: major | Milestone: Resultdb
Component: infrastructure | Version: 1.0
Keywords: |
----------------------------+-----------------------------------------------
Once #135 #136 and #137 are finished, we should create a provider/middle
man (probably a library) which will allow us to get information about the
respective test and use it for test execution/storing results/displaying
results via frontends...
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/138>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 6 months
[AutoQA] #137: ResultsDB: propose structure for storing metadata about tests on wiki
by fedora-badges
#137: ResultsDB: propose structure for storing metadata about tests on wiki
-----------------------+----------------------------------------------------
Reporter: jskladan | Owner:
Type: task | Status: new
Priority: major | Milestone: Resultdb
Component: docs/wiki | Version: 1.0
Keywords: |
-----------------------+----------------------------------------------------
We want to use mediawiki as a storage for tests metadata. We should agree
on the fields we want to store (e.g. test owner, destructive/non-
destructive, average time to complete test etc.) and the format we'll use
to store it (probably JSON though).
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/137>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 6 months
[AutoQA] #134: ResultsDB: create resultsdb prototype
by fedora-badges
#134: ResultsDB: create resultsdb prototype
----------------------+-----------------------------------------------------
Reporter: jskladan | Owner:
Type: task | Status: new
Priority: major | Milestone: Resultdb
Component: tests | Version: 1.0
Keywords: |
----------------------+-----------------------------------------------------
Create prototype of database and software which will implement dbSchema
from #133 and API from #131
This prototype should also include some test suite, which will be used to
determine, if all the functionality is OK.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/134>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 6 months
[AutoQA] #123: rpmguard: check upgrade paths between releases
by fedora-badges
#123: rpmguard: check upgrade paths between releases
----------------------+-----------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone:
Component: tests | Version: 1.0
Keywords: rpmguard |
----------------------+-----------------------------------------------------
(As pointed out by spot on the devel list:
http://lists.fedoraproject.org/pipermail/devel/2010-March/131746.html)
We should be emitting warning/error messages if the new package(s) have a
higher ENVR than the currently-released package(s) in any newer release.
For example, right now a new F11 package should be checked against the
corresponding package(s) in the current F12, branched F13, and rawhide
repos.
This involves a lot of extra repos and metadata, though. It's not a
trivial thing to add.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/123>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 8 months
[AutoQA] #111: depcheck test
by fedora-badges
#111: depcheck test
--------------------+-------------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: autoqa depcheck
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
Write an actual depcheck test. This test should take, as inputs:
* one or more new package builds, and
* the name of a target repository for the new build(s)
The test should examine the PRCO (provides/requires/conflicts/obsoletes)
data for the new package(s) and the PRCO data for the target repository
(and all its parent repos).
The test should fail if the new builds would cause missing/broken
dependencies or unresolveable conflicts in the target repo(s).
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/111>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 8 months