[AutoQA] #245: Add support for test re-scheduling
by fedora-badges
#245: Add support for test re-scheduling
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: task | Status: new
Priority: major | Milestone: 0.5.0
Component: core | Keywords:
--------------------+-------------------------------------------------------
After we start sending results to Bodhi/ResultDB there will immediately
emerge a need for test re-scheduling. There may be different reasons for
that:
1. Something went wrong in the test execution, we need to run it again.
2. The environment has changed and test needs to be executed once again.
For example the upgradepath test for some update failed because of invalid
package versions in higher Fedora releases. That problem has been
corrected and we need now a new execution of the test.
The important thing is that the maintainer should be able to trigger the
re-scheduling, not just us.
Find a way how to properly reschedule tests (we will probably utilize
ResultDB for that) and add that functionality.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/245>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 2 months
[AutoQA] #290: Provide concurrency-safe way of executing tests
by fedora-badges
#290: Provide concurrency-safe way of executing tests
-------------------------+--------------------------------------------------
Reporter: kparal | Owner:
Type: enhancement | Status: new
Priority: major | Milestone: 0.5.0
Component: core | Keywords:
-------------------------+--------------------------------------------------
There is already ticket #248 that talks about problems when two depcheck
tests are running at the same time. But this problem is not just about
depcheck, it applies generally to all tests that handle multiple packages
(probably whole repo or similar) at once. We have to provide some option
for these tests to specify that either only one such test can run at a
time or the older run results can't beat the newer run results (will
resultdb solve it effortlessly?).
I'm creating new ticket for this in case we do some kind of depcheck hack
first (issue #248) and provide proper solution for general tests later on.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/290>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 2 months
[AutoQA] #212: fix check_opt_in to not use collection_name
by fedora-badges
#212: fix check_opt_in to not use collection_name
-----------------------+----------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: minor | Milestone:
Component: docs/wiki | Version: 1.0
Keywords: |
-----------------------+----------------------------------------------------
collection_name was used to line up with the dist-cvs branch layout; now
that we have switched to dist-git/fedpkg the branch names match the repo
names used in repoinfo, so we have no need for collection_name.
This will require modifying the existing autoqa-optin file structure to
match the git layout. We can leave symlinks for the old ('F-14') distro
names but use the new ones ('f14') as the canonical ones.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/212>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 2 months
[AutoQA] #247: upgradepath: possible broken-updates scenario for new packages
by fedora-badges
#247: upgradepath: possible broken-updates scenario for new packages
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: tests | Keywords:
--------------------+-------------------------------------------------------
Here is a scenario that will create broken upgrade path and our current
tests will not catch it:
1. Package foo is available for F12 (foo-1.0-1.fc12), but not for F13.
2. foo-1.0-1.fc13 is sent to f13-updates-pending as a new package update.
upgradepath passes.
3. foo-2.0-1.fc12 is sent to f12-updates-pending. Since there is still no
foo in F13 (AutoQA passed but it has not been pushed by RelEng team),
upgradepath passes.
4. foo-2.0-1.fc12 is pushed to f12-updates.
5. foo-1.0-1.fc13 is pushed to f13-updates.
6. Broken upgrade path.
What can we do about it?
We can break the scenario between steps 4 and 5. But we can't probably do
that just by watching -pending tags. This requires cooperation from the
updates manager (i.e. Bodhi). Bodhi must know that if some foo build
enters f12-updates, and there is also another foo build in f13-updates-
pending, it must notify us (over Fedora Message Bus) and ask us to retest
the latter build.
We can also break the scenario between steps 2 and 3. When we run
upgradepath on foo in f12-updates-pending, we can take into account not
just f13-updates, but also f13-updates-pending. But this is quite
controversial. Some builds in -updates-pending may be denied by RelEng
team, but we could have failed some other builds because of them.
We will probably solve this ticket only when we have advanced
infrastructure ready (ResultDB, Fedora Message Bus). Without it this seems
pretty hard to be solved properly.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/247>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 2 months
[AutoQA] #266: Organize the 'autoqa' script into functions
by fedora-badges
#266: Organize the 'autoqa' script into functions
-------------------------+--------------------------------------------------
Reporter: jskladan | Owner:
Type: enhancement | Status: new
Priority: minor | Milestone: Finger Food
Component: core | Keywords:
-------------------------+--------------------------------------------------
It would be great to take the autoqa script, identify the 'functionally
standalone' parts (e.g. parsing command line options), and encapsulate
these into separate functions.
This would definitely make the script not even more readable, but also
understandable.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/266>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 2 months
[AutoQA] #265: Create function to download RPMs from koji for given NVR
by fedora-badges
#265: Create function to download RPMs from koji for given NVR
-------------------------+--------------------------------------------------
Reporter: jskladan | Owner:
Type: enhancement | Status: new
Priority: minor | Milestone: Bite Food
Component: core | Keywords:
-------------------------+--------------------------------------------------
Through most (if not all) the tests, we use a set of commands like this:
#install packages from koji
koji = autoqa.koji_utils.SimpleKojiClientSession()
pkgurls = koji.nvr_to_urls(envr, arches = os.uname()[-1])
rpms = []
print "Saving RPMs to %s" % self.rpmdir
#download packages
for p in pkgurls:
# fetch package to rpmdir
print "Grabbing %s" % p
localfile = os.path.join(self.rpmdir, os.path.basename(p))
autoqa.util.grabber.urlgrab(p, localfile)
rpms.append(localfile)
Which takes NVR and downloads all the relevant RPM files.
I'd like to have this enclosed in a method in the autoqa lib, which takes
NVR (or a list of NVRs), downloads packages to the tmpdir, and returns
list os.path.realpath() of the respective RPM files.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/265>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 2 months