merge of clumens branch onto master
by Chris Lumens
As previously discussed, I'm ready to merge my storage (plus other stuff
jlaska's been working on) tests onto master. Doing so will result in
the following being pushed:
52 files changed, 3622 insertions(+), 0 deletions(-)
create mode 100644 hooks/git-post-receive/README
create mode 100644 hooks/git-post-receive/autoqa-git-hook.conf
create mode 100755 hooks/git-post-receive/autoqa-git-hook.wsgi
create mode 100644 hooks/git-post-receive/hook.py
create mode 100755 hooks/git-post-receive/post-receive.sample
create mode 100644 tests/anaconda_checkbot/anaconda_checkbot.py
create mode 100755 tests/anaconda_checkbot/checkbot.sh
create mode 100644 tests/anaconda_checkbot/control
create mode 100644 tests/anaconda_checkbot/control.autoqa
create mode 100644 tests/anaconda_checkbot/mock.cfg.in
create mode 100644 tests/anaconda_storage/README
create mode 100644 tests/anaconda_storage/TODO
create mode 100755 tests/anaconda_storage/anaconda_storage.py
create mode 100644 tests/anaconda_storage/control
create mode 100644 tests/anaconda_storage/control.autoqa
create mode 100644 tests/anaconda_storage/framework/__init__.py
create mode 100644 tests/anaconda_storage/framework/cases/Autopart.py
create mode 100644 tests/anaconda_storage/framework/cases/AutopartFreespace.py
create mode 100644 tests/anaconda_storage/framework/cases/EncryptedAutopart.py
create mode 100644 tests/anaconda_storage/framework/cases/FSOnPartition.py
create mode 100644 tests/anaconda_storage/framework/cases/LVMonRAID0.py
create mode 100644 tests/anaconda_storage/framework/cases/NoSwapLVM.py
create mode 100644 tests/anaconda_storage/framework/cases/RAID0.py
create mode 100644 tests/anaconda_storage/framework/cases/RAID1.py
create mode 100644 tests/anaconda_storage/framework/cases/RAID5.py
create mode 100644 tests/anaconda_storage/framework/cases/RAID6.py
create mode 100644 tests/anaconda_storage/framework/cases/__init__.py
create mode 100755 tests/anaconda_storage/framework/driver.py.in
create mode 100644 tests/anaconda_storage/framework/exception.py
create mode 100644 tests/anaconda_storage/framework/intf.py
create mode 100755 tests/anaconda_storage/framework/runtest.py
create mode 100644 tests/anaconda_storage/framework/storage-test.ks
create mode 100644 tests/anaconda_storage/framework/suites/Autopart.py
create mode 100644 tests/anaconda_storage/framework/suites/AutopartFreespace.py
create mode 100644 tests/anaconda_storage/framework/suites/AutopartPreexistAutopart.py
create mode 100644 tests/anaconda_storage/framework/suites/ConflictingLVMAutopart.py
create mode 100644 tests/anaconda_storage/framework/suites/EncryptedAutopart.py
create mode 100644 tests/anaconda_storage/framework/suites/Ext2OnPartition.py
create mode 100644 tests/anaconda_storage/framework/suites/Ext3OnPartition.py
create mode 100644 tests/anaconda_storage/framework/suites/Ext4OnPartition.py
create mode 100644 tests/anaconda_storage/framework/suites/LVMonRAID0.py
create mode 100644 tests/anaconda_storage/framework/suites/NoSwapLVM.py
create mode 100644 tests/anaconda_storage/framework/suites/RAID0.py
create mode 100644 tests/anaconda_storage/framework/suites/RAID1.py
create mode 100644 tests/anaconda_storage/framework/suites/RAID5.py
create mode 100644 tests/anaconda_storage/framework/suites/RAID6.py
create mode 100644 tests/anaconda_storage/framework/suites/__init__.py
create mode 100644 tests/compose_tree/compose_tree.py
create mode 100755 tests/compose_tree/compose_tree.sh
create mode 100644 tests/compose_tree/control
create mode 100644 tests/compose_tree/control.autoqa
clumens@exeter:~/src/autoqa$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 87 commits.
Everyone's still okay with this, yeah? Because that's a big pile of stuff
to push on everyone all at once.
- Chris
12 years, 8 months
[AutoQA] #241: Add support for a staging server
by fedora-badges
#241: Add support for a staging server
--------------------+-------------------------------------------------------
Reporter: kparal | Owner:
Type: task | Status: new
Priority: major | Milestone: 0.4.4
Component: core | Keywords:
--------------------+-------------------------------------------------------
We want to be able to deploy AutoQA onto a staging server. Mainly it means
we must be able to disable or redirect the produced test output. We want
to be able to disable or redirect email sending (that includes standard
results and also opt-in emails). We want to be able to disable posting
Bodhi comments (or logging it somewhere else instead). We want to be able
to use a different ResultDB instance.
Make sure all the bits are in place and add required functionality if
necessary.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/241>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
[PATCH] use self.__class__ in super() calls
by Kamil Paral
This simplifies the test object template and our tests, it is no longer
necessary to update class names in super() calls in run_once and
initialize methods.
---
doc/test_class.py.template | 4 ++--
tests/conflicts/conflicts.py | 2 +-
tests/helloworld/helloworld.py | 2 +-
tests/initscripts/initscripts.py | 4 ++--
tests/rats_install/rats_install.py | 4 ++--
tests/rats_sanity/rats_sanity.py | 4 ++--
tests/repoclosure/repoclosure.py | 2 +-
tests/rpmguard/rpmguard.py | 4 ++--
tests/rpmlint/rpmlint.py | 4 ++--
tests/upgradepath/upgradepath.py | 4 ++--
10 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/doc/test_class.py.template b/doc/test_class.py.template
index c19074a..e5823ac 100644
--- a/doc/test_class.py.template
+++ b/doc/test_class.py.template
@@ -53,7 +53,7 @@ class testclassname(AutoQATest): # <-- UPDATE class name
# method - if you don't need to initialize anything, delete this block.
#@ExceptionCatcher()
#def initialize(self, config, **kwargs): #**kwargs needs to stay
- # super(testclassname, self).initialize(config) # <-- UPDATE class name
+ # super(self.__class__, self).initialize(config)
# #your extra initialization code goes here
# This is where the test code actually gets run. It's the only required
@@ -65,7 +65,7 @@ class testclassname(AutoQATest): # <-- UPDATE class name
# self.highlights: important lines to notice (string or list of strings)
@ExceptionCatcher()
def run_once(self, some_params, **kwargs): #**kwargs needs to stay
- super(testclassname, self).run_once() # <-- UPDATE class name
+ super(self.__class__, self).run_once()
cmd = 'test_binary --param %s' % some_params
self.outputs = utils.system_output(cmd, retain_output=True)
diff --git a/tests/conflicts/conflicts.py b/tests/conflicts/conflicts.py
index c9a4667..d77f507 100644
--- a/tests/conflicts/conflicts.py
+++ b/tests/conflicts/conflicts.py
@@ -34,7 +34,7 @@ class conflicts(AutoQATest):
@ExceptionCatcher()
def run_once(self, baseurl, parents, name, **kwargs):
- super(conflicts, self).run_once()
+ super(self.__class__, self).run_once()
if name:
name = "%s-%s" % (name, autoqa.util.get_basearch())
else:
diff --git a/tests/helloworld/helloworld.py b/tests/helloworld/helloworld.py
index 1a96ef5..5422ae3 100644
--- a/tests/helloworld/helloworld.py
+++ b/tests/helloworld/helloworld.py
@@ -25,7 +25,7 @@ class helloworld(AutoQATest):
@ExceptionCatcher()
def run_once(self, *args, **kwargs):
- super(helloworld, self).run_once()
+ super(self.__class__, self).run_once()
self.summary = 'Hello, World!'
self.outputs = "===Printing passed params===\n"
for arg in args:
diff --git a/tests/initscripts/initscripts.py b/tests/initscripts/initscripts.py
index a5d22f3..51462c9 100644
--- a/tests/initscripts/initscripts.py
+++ b/tests/initscripts/initscripts.py
@@ -48,7 +48,7 @@ class initscripts(AutoQATest):
@ExceptionCatcher()
def initialize(self, config, **kwargs):
- super(initscripts, self).initialize(config)
+ super(self.__class__, self).initialize(config)
self.rpmdir = os.path.join(self.tmpdir, 'rpms')
os.makedirs(self.rpmdir)
@@ -102,7 +102,7 @@ class initscripts(AutoQATest):
@ExceptionCatcher()
def run_once(self, kojitag, **kwargs):
- super(initscripts, self).run_once()
+ super(self.__class__, self).run_once()
if kwargs['hook'] == 'post-koji-build':
envrs = [kwargs['envr']]
update_id = kwargs['envr']
diff --git a/tests/rats_install/rats_install.py b/tests/rats_install/rats_install.py
index 23c2015..4ed010f 100644
--- a/tests/rats_install/rats_install.py
+++ b/tests/rats_install/rats_install.py
@@ -31,7 +31,7 @@ class rats_install(AutoQATest):
@ExceptionCatcher()
def initialize(self, config, **kwargs):
- super(rats_install, self).initialize(config)
+ super(self.__class__, self).initialize(config)
self.irb_conf = os.path.join(self.bindir, 'irb.cfg')
@ExceptionCatcher()
@@ -52,7 +52,7 @@ class rats_install(AutoQATest):
@ExceptionCatcher()
def run_once(self, baseurl, name, image_url="", boot_args="", **kwargs):
- super(rats_install, self).run_once()
+ super(self.__class__, self).run_once()
if name:
name = "%s-%s" % (name, util.get_basearch())
else:
diff --git a/tests/rats_sanity/rats_sanity.py b/tests/rats_sanity/rats_sanity.py
index 19116ef..de704b8 100644
--- a/tests/rats_sanity/rats_sanity.py
+++ b/tests/rats_sanity/rats_sanity.py
@@ -30,7 +30,7 @@ class rats_sanity(AutoQATest):
@ExceptionCatcher()
def initialize(self, config, **kwargs):
- super(rats_sanity, self).initialize(config)
+ super(self.__class__, self).initialize(config)
self.irb_conf = os.path.join(self.bindir, 'irb.cfg')
@ExceptionCatcher()
@@ -40,7 +40,7 @@ class rats_sanity(AutoQATest):
@ExceptionCatcher()
def run_once(self, baseurl, parents, name, **kwargs):
- super(rats_install, self).run_once()
+ super(self.__class__, self).run_once()
if name:
name = "%s-%s" % (name, util.get_basearch())
else:
diff --git a/tests/repoclosure/repoclosure.py b/tests/repoclosure/repoclosure.py
index f167722..eb5a914 100644
--- a/tests/repoclosure/repoclosure.py
+++ b/tests/repoclosure/repoclosure.py
@@ -31,7 +31,7 @@ class repoclosure(AutoQATest):
@ExceptionCatcher()
def run_once(self, baseurl, parents='', name='', **kwargs):
- super(repoclosure, self).run_once()
+ super(self.__class__, self).run_once()
if name:
name = "%s-%s" % (name, autoqa.util.get_basearch())
else:
diff --git a/tests/rpmguard/rpmguard.py b/tests/rpmguard/rpmguard.py
index afc4ca3..faba8e0 100644
--- a/tests/rpmguard/rpmguard.py
+++ b/tests/rpmguard/rpmguard.py
@@ -37,14 +37,14 @@ class rpmguard(AutoQATest):
@ExceptionCatcher()
def initialize(self, config, **kwargs):
- super(rpmguard, self).initialize(config)
+ super(self.__class__, self).initialize(config)
self.rpmguard = os.path.join(self.bindir, 'rpmguard')
self.rpmdir = os.path.join(self.tmpdir, 'rpms')
os.makedirs(self.rpmdir)
@ExceptionCatcher()
def run_once(self, kojitag, **kwargs):
- super(rpmguard, self).run_once()
+ super(self.__class__, self).run_once()
if kwargs['hook'] == 'post-koji-build':
envrs = [kwargs['envr']]
update_id = kwargs['envr']
diff --git a/tests/rpmlint/rpmlint.py b/tests/rpmlint/rpmlint.py
index b2f5515..b1ebb35 100644
--- a/tests/rpmlint/rpmlint.py
+++ b/tests/rpmlint/rpmlint.py
@@ -38,13 +38,13 @@ class rpmlint(AutoQATest):
@ExceptionCatcher()
def initialize(self, config, **kwargs):
- super(rpmlint, self).initialize(config)
+ super(self.__class__, self).initialize(config)
self.rpmdir = os.path.join(self.tmpdir, 'rpms')
os.makedirs(self.rpmdir)
@ExceptionCatcher()
def run_once(self, kojitag, **kwargs):
- super(rpmlint, self).run_once()
+ super(self.__class__, self).run_once()
if kwargs['hook'] == 'post-koji-build':
envrs = [kwargs['envr']]
update_id = kwargs['envr']
diff --git a/tests/upgradepath/upgradepath.py b/tests/upgradepath/upgradepath.py
index a99ac2f..15e5c38 100755
--- a/tests/upgradepath/upgradepath.py
+++ b/tests/upgradepath/upgradepath.py
@@ -30,7 +30,7 @@ class upgradepath(AutoQATest):
@ExceptionCatcher()
def initialize(self, config, **kwargs):
- super(upgradepath, self).initialize(config)
+ super(self.__class__, self).initialize(config)
self.result = 'PASSED'
# order for evaluation of final result; higher index means preference
self.result_order = ('PASSED','INFO','FAILED','ABORTED')
@@ -86,7 +86,7 @@ class upgradepath(AutoQATest):
@ExceptionCatcher()
def run_once(self, envrs, kojitag, **kwargs):
- super(upgradepath, self).run_once()
+ super(self.__class__, self).run_once()
update_id = kwargs['name'] or kwargs['id']
# Get a list of all repos we monitor (currently not -testing)
--
1.7.3.4
12 years, 9 months
[AutoQA] #205: make depcheck submit karma/comments to bodhi
by fedora-badges
#205: make depcheck submit karma/comments to bodhi
--------------------+-------------------------------------------------------
Reporter: wwoods | Owner:
Type: task | Status: new
Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck
Component: tests | Version: 1.0
Keywords: |
--------------------+-------------------------------------------------------
Modify the depcheck test so that it sends status info - karma, comments,
whatever - to bodhi.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/205>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
12 years, 9 months
'rpm' make target failing
by James Laska
Greetings,
I was just testing the latest 'master' branch, and it seems that the
'rpms' Makefile target is broken. Along with the change to support
posting bodhi comments (5d37e59128c1913946c02e6a1d899a5576f37801), we
set the group of fas.conf to the user 'autotest'. On my primary system,
I don't have autotest installed, so the 'make rpms' target fails as
follows:
make[1]: Leaving directory `/tmp/jlaska-rpm/BUILD/autoqa-0.4.3'
+ install -m 644 autoqa.conf repoinfo.conf /tmp/jlaska-rpm/BUILDROOT/autoqa-0.4.3-1.fc14.x86_64/etc/autoqa/
+ install -m 640 -g autotest fas.conf /tmp/jlaska-rpm/BUILDROOT/autoqa-0.4.3-1.fc14.x86_64/etc/autoqa/
install: invalid group `autotest'
error: Bad exit status from /tmp/jlaska-rpm/TMP/rpm-tmp.Hj4RAu (%install)
An easy fix would be to add "BuildRequires: autotest" to the spec file,
but for some reason that doesn't seem right. I can't think of any, but
should we consider alternatives to adding a BuildRequires for autotest?
Thanks,
James
12 years, 9 months
[PATCH] two bugfixes
by Kamil Paral
Shame on me. Already pushed to master. Thanks Josef for spotting this.
Kamil
commit a09a4276ddbef08ff21067efc4d95a33da634410
Author: Kamil Páral <kparal(a)redhat.com>
Date: Fri Jan 7 12:23:31 2011 +0100
bugfix: getbool() crashes for empty string
Revert to expected behavior where False is returned for empty string
input.
diff --git a/lib/python/config.py b/lib/python/config.py
index c9a59ad..1d7bc73 100644
--- a/lib/python/config.py
+++ b/lib/python/config.py
@@ -89,14 +89,14 @@ def getbool(value):
Uses default bool conversion for non-string values.
'''
boolean_states = {'1': True, 'yes': True, 'true': True, 'on': True,
- '0': False, 'no': False, 'false': False, 'off': False}
+ '0': False, 'no': False, 'false': False, 'off': False, '': False}
if not isinstance(value, str):
return bool(value)
value = value.lower()
if value not in boolean_states:
- raise ValueError, 'Not a boolean: %s' % value
+ raise ValueError, 'Not a boolean: %r %s' % (value, type(value))
return boolean_states[value]
commit ae43a319b7154904194a05ddbf0c72671fa1204f
Author: Kamil Páral <kparal(a)redhat.com>
Date: Fri Jan 7 12:39:47 2011 +0100
repoinfo.conf: set 'composes = no' as default for correct retyping
ConfigParser.getboolean() can't handle 'composes = (empty)'.
diff --git a/repoinfo.conf b/repoinfo.conf
index 6f5c279..97dceca 100644
--- a/repoinfo.conf
+++ b/repoinfo.conf
@@ -4,7 +4,7 @@ parents =
# list of supported architectures
arches = i386, x86_64
# whether installation composes are created for this repo or not
-composes =
+composes = no
# tag defaults to dist-[section_name]
tag = dist-%(__name__)s
baseurl = http://download.fedoraproject.org/pub/fedora/linux
12 years, 9 months
[PATCH] Load config files from current directory by default
by Kamil Paral
This patch ensures that the last remaining config file repoinfo.conf is
also loaded from current directory by default and only when it's not
present the system-wide location is used. It means that now all our
config files use CWD location by default and we don't have to maintain
/etc directory on AutoQA clients anymore.
It also adds a few comments to RepoinfoConfig object and repoinfo.conf.
Post-repo-update and post-tree-compose watchers have slightly modified
placeholder for architecture in repo URLs.
getbool() method now does not crash for non-string inputs.
Fixes ticket #253.
---
hooks/post-repo-update/watch-repos.py | 12 ++++++------
hooks/post-tree-compose/watch-composes.py | 10 +++++-----
lib/python/config.py | 16 +++++++++-------
lib/python/repoinfo.py | 29 +++++++++++++++++++++++------
repoinfo.conf | 4 ++++
5 files changed, 47 insertions(+), 24 deletions(-)
diff --git a/hooks/post-repo-update/watch-repos.py b/hooks/post-repo-update/watch-repos.py
index 0d3ce8f..46bbb13 100755
--- a/hooks/post-repo-update/watch-repos.py
+++ b/hooks/post-repo-update/watch-repos.py
@@ -36,8 +36,8 @@ parser.add_option('--dryrun', '--dry-run', action='store_true',
help='Do not actually execute commands, just show what would be done')
(opts, args) = parser.parse_args()
-# Set the default arch to our placeholder, '%a'
-repoinfo.setarch('%%a') # two %% because of ConfigParser interpolation
+# Set the default arch to our placeholder '$ARCH'
+repoinfo.setarch('$ARCH')
# Setup a cache path
cachedir = '/var/cache/autoqa/watch-repos'
@@ -65,7 +65,7 @@ for reponame in repoinfo.repos():
cache = ''
# grab repomd.xml from remote
try:
- urlpath = os.path.join(repo['url'].replace('%a',arch),
+ urlpath = os.path.join(repo['url'].replace('$ARCH',arch),
'repodata', 'repomd.xml')
remote = urlgrabber.urlread(urlpath)
except Exception, e:
@@ -85,8 +85,8 @@ for reponame, arches in sorted(testable.items()):
'--arch', arch]
# Add parent repos to the arglist
for preponame in repo['parents']:
- harnesscall += ['--parent', repoinfo.get(preponame, 'url').replace('%a', arch)]
- harnesscall.append(repo['url'].replace('%a', arch))
+ harnesscall += ['--parent', repoinfo.get(preponame, 'url').replace('$ARCH', arch)]
+ harnesscall.append(repo['url'].replace('$ARCH', arch))
if opts.dryrun:
print ' '.join(harnesscall)
@@ -96,7 +96,7 @@ for reponame, arches in sorted(testable.items()):
cachepath = os.path.join(cachedir, reponame, arch)
if not os.path.exists(cachepath):
os.makedirs(cachepath)
- urlpath = os.path.join(repo['url'].replace('%a',arch),
+ urlpath = os.path.join(repo['url'].replace('$ARCH',arch),
'repodata',
'repomd.xml')
output = os.path.join(cachepath, 'repomd.xml')
diff --git a/hooks/post-tree-compose/watch-composes.py b/hooks/post-tree-compose/watch-composes.py
index a9d4e23..ca9c0d1 100755
--- a/hooks/post-tree-compose/watch-composes.py
+++ b/hooks/post-tree-compose/watch-composes.py
@@ -37,8 +37,8 @@ parser.add_option('--dryrun', '--dry-run', action='store_true',
help='Do not actually execute commands, just show what would be done')
(opts, args) = parser.parse_args()
-# Set the default arch to our placeholder, '%a'
-repoinfo.setarch('%%a') # two %% because of ConfigParser interpolation
+# Set the default arch to our placeholder '$ARCH'
+repoinfo.setarch('$ARCH')
# Get the list of repos to watch from repoinfo (see /etc/autoqa/repoinfo.conf)
watchcomposes = [r for r in repoinfo.repos() if getbool(repoinfo.get(r,'composes'))]
@@ -70,7 +70,7 @@ for reponame in watchcomposes:
# check .treeinfo
try:
# XXX use urljoin?
- urlpath = os.path.join(repo['url'].replace('%a',arch),
+ urlpath = os.path.join(repo['url'].replace('$ARCH',arch),
'.treeinfo')
remote = urlgrabber.urlread(urlpath)
# XXX maybe less broad exception?
@@ -89,7 +89,7 @@ for reponame, arches in sorted(testable.items()):
# construct arguments for call to test launcher
harnesscall = ['autoqa', 'post-tree-compose', '--name', repo['name'],
'--arch', arch]
- harnesscall.append(repo['url'].replace('%a', arch))
+ harnesscall.append(repo['url'].replace('$ARCH', arch))
if opts.dryrun:
print ' '.join(harnesscall)
continue
@@ -98,7 +98,7 @@ for reponame, arches in sorted(testable.items()):
cachepath = os.path.join(cachedir, repo['path'], arch)
if not os.path.exists(cachepath):
os.makedirs(cachepath)
- composepath = repo['url'].replace('%a', arch)
+ composepath = repo['url'].replace('$ARCH', arch)
urlpath = os.path.join(composepath, '.treeinfo')
output = os.path.join(cachepath, 'treeinfo')
urlgrabber.urlgrab(urlpath, filename=output)
diff --git a/lib/python/config.py b/lib/python/config.py
index 955f7a0..c9a59ad 100644
--- a/lib/python/config.py
+++ b/lib/python/config.py
@@ -81,22 +81,24 @@ class SingleConfigParser(ConfigParser.SafeConfigParser):
retval.update(items)
return retval
-def getbool(string):
+def getbool(value):
'''Converts a string into a boolean.
Recognizes many usual strings: 1/0, yes/no, true/false, on/off; also
empty strings.
Throws ValueError if the string can't be recognized.
+ Uses default bool conversion for non-string values.
'''
boolean_states = {'1': True, 'yes': True, 'true': True, 'on': True,
'0': False, 'no': False, 'false': False, 'off': False}
- if not string:
- return False
- string = string.lower()
- if string not in boolean_states:
- raise ValueError, 'Not a boolean: %s' % string
+ if not isinstance(value, str):
+ return bool(value)
- return boolean_states[string]
+ value = value.lower()
+ if value not in boolean_states:
+ raise ValueError, 'Not a boolean: %s' % value
+
+ return boolean_states[value]
autoqa_conf = SingleConfigParser()
diff --git a/lib/python/repoinfo.py b/lib/python/repoinfo.py
index 3a8c1f8..de9f2de 100644
--- a/lib/python/repoinfo.py
+++ b/lib/python/repoinfo.py
@@ -20,24 +20,35 @@
# Author: Will Woods <wwoods(a)redhat.com>
from ConfigParser import SafeConfigParser
+from config import SingleConfigParser
import warnings
-configpath = ('/etc/autoqa/repoinfo.conf',
- 'repoinfo.conf')
+configpath = ('repoinfo.conf',
+ '/etc/autoqa/repoinfo.conf',)
class RepoinfoConfig(object):
def __init__(self, arch="$basearch", filelist=configpath, defaults={}):
+ '''Create RepoinfoConfig object.
+ Attributes:
+ * arch - architecture for which to adjust repo URLs
+ * filelist - list of config files to read information from, just the
+ first available file will be used
+ * defaults - dictionary of default values
+ '''
self.defaults = defaults
self.filelist = filelist
self.setarch(arch) # implies read()
def _getlist(self, name, item):
+ '''Transform a list of comma separated values into a Python list
+ Returns empty list for non-existent item'''
itemlist = self.config.get(name, item)
if not itemlist:
return []
else:
return [t.strip() for t in itemlist.split(',')]
def getarches(self, name):
+ '''Get the list of all supported architectures for a repo'''
return self._getlist(name, 'arches')
def getreleases(self):
'''Return the list of known, non-EOL releases.'''
@@ -51,7 +62,8 @@ class RepoinfoConfig(object):
return self._getlist(name, 'parents')
def getrepo(self, name):
'''Given a repo name, return the repoinfo dict, with the keys:
- 'arches', 'parents', 'tag', 'url', 'path', 'name' '''
+ 'arches', 'parents', 'tag', 'url', 'path', 'name', 'collection_name'
+ and 'composes' '''
repo = {'arches': self.getarches(name),
'parents': self.getparents(name),
'tag': self.config.get(name,'tag'),
@@ -59,7 +71,7 @@ class RepoinfoConfig(object):
'path': self.config.get(name,'path'),
'name': name, # kind of unnecessary
'collection_name': self.config.get(name, 'collection_name'),
- 'composes': self.config.get(name,'composes'),
+ 'composes': self.config.getboolean(name,'composes'),
}
return repo
def getrepo_by_tag(self, tag):
@@ -76,15 +88,20 @@ class RepoinfoConfig(object):
'''Returns a list of all known repo names'''
return self.config.sections()
def setarch(self, arch):
+ '''Set architecture for which to adjust repo URLs'''
self.defaults['arch'] = arch
self.read()
def read(self):
- self.config = SafeConfigParser(self.defaults)
- return self.config.read(self.filelist)
+ '''Apply the defaults and then read first available config file from
+ the list of provided config files.'''
+ self.config = SingleConfigParser(defaults=self.defaults)
+ self.config.read_single(self.filelist)
def get(self, repo, key):
+ '''Get a specific key from a repo'''
return self.config.get(repo, key)
repoinfo = RepoinfoConfig()
+
if not repoinfo.repos():
warnings.warn("No repos found in (%s)" % ', '.join(configpath),
RuntimeWarning)
diff --git a/repoinfo.conf b/repoinfo.conf
index 4873302..6f5c279 100644
--- a/repoinfo.conf
+++ b/repoinfo.conf
@@ -1,6 +1,9 @@
[DEFAULT]
+# list of parent repos to this repo
parents =
+# list of supported architectures
arches = i386, x86_64
+# whether installation composes are created for this repo or not
composes =
# tag defaults to dist-[section_name]
tag = dist-%(__name__)s
@@ -8,6 +11,7 @@ baseurl = http://download.fedoraproject.org/pub/fedora/linux
goldurl = %(baseurl)s/releases/%(path)s/Everything/%(arch)s/os
updatesurl = %(baseurl)s/updates/%(path)s/%(arch)s
rawhideurl = %(baseurl)s/%(path)s/%(arch)s/os
+# true for "main" repos corresponding to currently supported Fedora releases
isactiverelease =
# Rawhide
--
1.7.3.3
12 years, 9 months
[PATCH] Add support for a staging server.
by Martin Krizek
Hey all,
this patch solves ticket #241. It adds options in autoqa.conf to turn on/off
optin, notifications and results e-mails. There are options containing urls of
koji and bodhi instances that will be used.
Also there are some alterations in bodhi_utils, there is only one creation of
BodhiClient instance and one config files reading to simplify the code.
You can find the patch in mkrizek-staging-ready branch.
---
diff --git a/autoqa b/autoqa
index fd41503..190e63b 100755
--- a/autoqa
+++ b/autoqa
@@ -40,12 +40,14 @@ conf = {
'testdir': '/usr/share/autotest/client/site_tests',
'hookdir': '/usr/share/autoqa',
'notification_email': '',
+ 'send_notification_email': 'false',
'autotest_server': socket.gethostname(),
}
cfg_parser = SingleConfigParser()
cfg_parser.read(cfgfile)
-conf = cfg_parser.get_section('general', conf) # used by prep_controlfile
+conf = cfg_parser.get_section('general', conf)
+conf = cfg_parser.get_section('notifications', conf)
# we don't need to catch errors here, bcz we want autoqa to crash for invalid config
def prep_controlfile(controlfile, extradata):
@@ -283,7 +285,10 @@ for test in testlist:
for arch in test_vars[test]['archs']:
testname='%s:%s.%s' % (hookname, test, arch)
- email = conf['notification_email']
+ if conf['send_notification_email']:
+ email = conf['notification_email']
+ else:
+ email = ''
if run_local:
retval = run_test_locally(control, name=testname,
diff --git a/autoqa.conf b/autoqa.conf
index 69278fe..886536e 100644
--- a/autoqa.conf
+++ b/autoqa.conf
@@ -6,29 +6,62 @@
hookdir = /usr/share/autoqa
# Path containing the actual autoqa tests to be run
testdir = /usr/share/autotest/client/site_tests
-# comma-separated email addresses that will get notification of test completion
-# (leave blank to disable)
-notification_email =
# Set to "true" if you want autoqa to always run tests locally using 'autotest'.
# Default is to schedule jobs with the autotest server using 'atest'.
# This is the same as using 'autoqa --local'.
-#local = true
+local = false
+# Hostname of this autotest server. Used for constructing URLs for accessing
+# test results. By default current hostname is used.
+autotest_server =
-# The 'test' section is used for systemwide configuration of autoqa tests
-[test]
-# comma-separated list of email addresses that will receive full test results
-# (leave blank to disable)
-result_email =
-# email address for From: line of emails sent by tests
-mail_from = autoqa(a)fedoraproject.org
-# hostname or hostname:port of smtp server / mailhub to use for sending email
+# The 'notifications' section controls which notifications about which events
+# you want to send
+[notifications]
+# Sender email address used for all emails sent from AutoQA.
+mail_from =
+# Hostname or hostname:port of SMTP server used for sending emails.
+# Please note that the emails are sent from the autotest clients, not the
+# server.
smtpserver = localhost
-[bodhi]
-# If "true", test results (for tests utilizing this feature) will be sent
-# as comments to Fedora Update System (Bodhi). This requires that you have
-# Bodhi credentials filled in in fas.conf.
+# Comma-separated list of email addresses that will receive full test results
+# of every completed test.
+result_email =
+# If "true", result email will be sent. Otherwise "false".
+send_result_email = false
+
+# Opt-in emails are sent (for selected tests) to the maintainers of a package
+# that has just been tested and the maintainer opted in. Read more at:
+# http://jlaska.wordpress.com/2010/06/01/fedora-package-maintainers-want-te...
+# If "true", opt-in email will be sent. Otherwise "false".
+send_optin_email = false
+
+# Comma-separated email addresses that will get notification of every job
+# completion or every job state change as defined in 'notify_email_statuses'
+# in ~autotest/global_config.ini.
+# Important: These emails are handled and sent by autotest server, the
+# 'mail_from' and 'smtpserver' options don't apply for it.
+# Read more at http://autotest.kernel.org/wiki/GlobalConfig (search for
+# 'notify_email').
+notification_email =
+# If "true", notification email will be sent. Otherwise "false".
+send_notification_email = false
+
+# If "true", test results (for selected tests) will be sent as comments to
+# Fedora Update System (Bodhi). This requires that you have Bodhi credentials
+# filled in in fas.conf.
send_bodhi_comments = false
-# how long (minutes) should we wait before posting the same comment to bodhi
-# by default 3 days (3*24*60 = 4320)
+# How long (in minutes) should we wait before allowing consequent test to
+# re-post a 'FAILED' comment into Bodhi once again.
+# By default 3 days (3*24*60 = 4320).
bodhi_posting_comments_span = 4320
+
+# The 'resources' section specify access details to various external services
+[resources]
+# URL of Koji instance used for querying about new builds
+koji_url = http://koji.fedoraproject.org/kojihub
+# URL of repository of all the RPM packages built in Koji
+pkg_url = http://koji.fedoraproject.org/packages
+# URL of Bodhi instance used for communication about package updates
+# (leave blank for default)
+bodhi_server =
diff --git a/hooks/post-koji-build/watch-koji-builds.py b/hooks/post-koji-build/watch-koji-builds.py
index 4cc2d7f..57f0aee 100755
--- a/hooks/post-koji-build/watch-koji-builds.py
+++ b/hooks/post-koji-build/watch-koji-builds.py
@@ -41,9 +41,6 @@ try:
except OSError, e:
if e.errno != 17: # already exists
raise
-# XXX configparser? /etc/koji.conf, section 'koji', item 'server'
-# alternately: read from e.g. /etc/autoqa/autoqa.conf
-kojiserver = 'http://koji.fedoraproject.org/kojihub'
def get_prevtime():
'''Returns the prevtime to use when looking for new builds. Value will be
@@ -126,7 +123,7 @@ tags for new builds and kick off tests when new builds/packages are found.')
# Set up the koji connection
kojiopts = {} # Possible items: user, password, debug_xmlrpc, debug..
- session = koji_utils.SimpleKojiClientSession(kojiserver, kojiopts)
+ session = koji_utils.SimpleKojiClientSession(kojiopts)
untagged_builds = []
tagged_builds = []
exit_status = 0
diff --git a/lib/python/bodhi_utils.py b/lib/python/bodhi_utils.py
index 94e1a7d..95d18a5 100644
--- a/lib/python/bodhi_utils.py
+++ b/lib/python/bodhi_utils.py
@@ -28,6 +28,32 @@ from datetime import datetime
from util import SingleConfigParser, getbool
import ConfigParser
+try:
+ autoqa_conf = SingleConfigParser()
+ autoqa_conf.read_single(['autoqa.conf', '/etc/autoqa/autoqa.conf'])
+
+ server = autoqa_conf.get('resources', 'bodhi_server')
+except ConfigParser.Error, e:
+ server = ''
+
+try:
+ fas_conf = SingleConfigParser()
+ fas_conf.read_single(['fas.conf', '/etc/autoqa/fas.conf'])
+ fas = fas_conf.get_section('fas')
+ user = fas['username']
+ pswd = fas['password']
+ if not user or not pswd:
+ raise IOError
+except (IOError, ConfigParser.Error), e:
+ # if there's no fas.conf or it's incomplete, we just log in anonymously.
+ user = ''
+ pswd = ''
+
+if server:
+ bodhi = fedora.client.bodhi.BodhiClient(base_url=server, username=user, password=pswd)
+else:
+ bodhi = fedora.client.bodhi.BodhiClient(username=user, password=pswd)
+
def bodhitime(timestamp):
'''Convert timestamp (seconds since Epoch, assumed to be local time) to a
Bodhi-approved time string ('%Y-%m-%d %H:%M:%S')'''
@@ -40,7 +66,6 @@ def parse_bodhitime(bodhitimestr):
def bodhi_list(params, limit=100):
'''Perform a bodhi 'list' method call, with pagination handling'''
- bodhi = fedora.client.bodhi.BodhiClient()
params['tg_paginate_limit'] = limit
params['tg_paginate_no'] = 1
@@ -136,7 +161,7 @@ def _is_bodhi_testresult_needed(old_result, comment_time, result, time_span):
return True
-def bodhi_post_testresult(update, testname, result, url, config, arch = 'noarch', karma = 0):
+def bodhi_post_testresult(update, testname, result, url, arch = 'noarch', karma = 0):
'''Post comment and karma to bodhi
Args:
@@ -144,7 +169,6 @@ def bodhi_post_testresult(update, testname, result, url, config, arch = 'noarch'
testname -- the name of the test
result -- the result of the test
url -- url of the result of the test
- config -- autoqa config
arch -- tested architecture (default 'noarch')
karma -- karma points (default 0)
@@ -153,37 +177,25 @@ def bodhi_post_testresult(update, testname, result, url, config, arch = 'noarch'
posted (either posting is turned off or comment was already posted),
False otherwise.
'''
+
+ # TODO when new bodhi releases, add update identification by UPDATEID support
+
err_msg = 'Could not post a comment to bodhi'
- conffiles = ['fas.conf', '/etc/autoqa/fas.conf']
- try:
- cfg_parser = SingleConfigParser()
- cfg_parser.read_single(conffiles)
- fas = cfg_parser.get_section('fas')
- except (IOError, ConfigParser.Error), e:
- print >> sys.stderr, "ERROR: Couldn't read none of config files: %s" % conffiles
- print >> sys.stderr, e
- return False
if not update or not testname or not result or url == None:
sys.stderr.write('Incomplete arguments!\n%s\n' % err_msg)
return False
try:
- if not getbool(config.get('bodhi', 'send_bodhi_comments')):
- print 'Sending bodhi comments is turned off. Test result will NOT be sent.'
- return True
- except KeyError:
+ if not getbool(autoqa_conf.get('notifications', 'send_bodhi_comments')):
+ raise ValueError
+ except ValueError:
print 'Sending bodhi comments is turned off. Test result will NOT be sent.'
- # option missing -> it's false, do not send it (but return True since
+ # it's either False or not a bool -> it's false, do not send it (but return True since
# it's intentional, not an error)
return True
- try:
- user = fas['username']
- pswd = fas['password']
- if not user or not pswd:
- raise KeyError
- except KeyError:
+ if not user or not pswd:
sys.stderr.write('Conf file containing FAS credentials is incomplete!\n%s\n' % err_msg)
return False
@@ -191,14 +203,12 @@ def bodhi_post_testresult(update, testname, result, url, config, arch = 'noarch'
% (testname, result, arch, url)
try:
(old_result, comment_time) = _bodhi_already_commented(update, user, testname, arch)
- time_span = int(config.get('bodhi', 'bodhi_posting_comments_span'))
+ time_span = int(autoqa_conf.get('notifications', 'bodhi_posting_comments_span'))
if not _is_bodhi_testresult_needed(old_result, comment_time, result, time_span):
print 'The test result already posted to bodhi.'
return True
- bodhi = fedora.client.BodhiClient(username=user, password=pswd)
-
if not bodhi.comment(update, comment, karma):
sys.stderr.write('%s\n') % err_msg
return False
@@ -216,10 +226,7 @@ def _self_test():
Simple self test.
'''
from datetime import timedelta
- from ConfigParser import SafeConfigParser
- cfg_parser = SafeConfigParser()
- cfg_parser.read('/etc/autoqa/autoqa.conf')
- time_span = int(cfg_parser.get('bodhi', 'bodhi_posting_comments_span'))
+ time_span = int(autoqa_conf.get('notifications', 'bodhi_posting_comments_span'))
try:
print '1. Test:',
assert _is_bodhi_testresult_needed('PASSED', datetime.now, 'PASSED', time_span) == False
diff --git a/lib/python/koji_utils.py b/lib/python/koji_utils.py
index 87b32e9..648412b 100644
--- a/lib/python/koji_utils.py
+++ b/lib/python/koji_utils.py
@@ -24,15 +24,16 @@ import koji
from repoinfo import repoinfo
import rpmUtils.miscutils
import sys
-
-# XXX fetch from /etc/koji.conf, section 'koji'
-kojiurl = 'http://koji.fedoraproject.org/kojihub'
-pkgurl = 'http://koji.fedoraproject.org/packages'
+from autoqa.bodhi_utils import SingleConfigParser
class SimpleKojiClientSession(koji.ClientSession):
'''Convenience wrapper class for interacting with koji'''
- def __init__(self, server=kojiurl, opts=None):
- return koji.ClientSession.__init__(self, server, opts)
+ def __init__(self, opts=None):
+ cfg_parser = SingleConfigParser()
+ cfg_parser.read_single(['autoqa.conf', '/etc/autoqa/autoqa.conf'])
+ self.server = cfg_parser.get('resources', 'koji_url')
+ self.pkgurl = cfg_parser.get('resources', 'pkg_url')
+ return koji.ClientSession.__init__(self, self.server, opts)
def list_builds_since(self, timestamp):
'''Return a list of new builds since the given timestamp'''
@@ -147,7 +148,7 @@ class SimpleKojiClientSession(koji.ClientSession):
def nvr_to_urls(self, nvr, arches=None, debuginfo=False):
info = self.getBuild(nvr)
- baseurl = '/'.join((pkgurl, info['package_name'],
+ baseurl = '/'.join((self.pkgurl, info['package_name'],
info['version'], info['release']))
rpms = self.listRPMs(buildID=info['id'], arches=arches)
@@ -162,7 +163,7 @@ class SimpleKojiClientSession(koji.ClientSession):
key added with URL where to download the RPM as the value.'''
info = self.getBuild(nvr)
- baseurl = '/'.join((pkgurl, info['package_name'],
+ baseurl = '/'.join((self.pkgurl, info['package_name'],
info['version'], info['release']))
rpms = self.listRPMs(buildID=info['id'])
diff --git a/lib/python/test.py b/lib/python/test.py
index 29bb96c..73efc80 100644
--- a/lib/python/test.py
+++ b/lib/python/test.py
@@ -21,7 +21,7 @@ from autotest_lib.client.bin import test, utils
from autotest_lib.client.bin.test_config import config_loader
from decorators import ExceptionCatcher
-from util import make_autotest_url
+from util import make_autotest_url, getbool
import os, sys, traceback
class AutoQATest(test.test, object):
@@ -80,7 +80,7 @@ class AutoQATest(test.test, object):
# prepare email
send_to = []
- result_email = self.config.get('test','result_email')
+ result_email = self.config.get('notifications','result_email')
if result_email and isinstance(result_email, str):
send_to = result_email.split(',')
try:
@@ -114,12 +114,12 @@ class AutoQATest(test.test, object):
log.close()
# send email
- if send_to:
+ if getbool(self.config.get('notifications', 'send_result_email')) and send_to:
utils.send_email(mail_to=send_to,
- mail_from=self.config.get('test','mail_from'),
- mail_server=self.config.get('test','smtpserver'),
- subject=self.summary,
- body = mail_body)
+ mail_from=self.config.get('notifications','mail_from'),
+ mail_server=self.config.get('notifications','smtpserver'),
+ subject=self.summary,
+ body = mail_body)
def _convert_list_variables(self):
'''
diff --git a/tests/initscripts/initscripts.py b/tests/initscripts/initscripts.py
index 8e35e35..097fbc8 100644
--- a/tests/initscripts/initscripts.py
+++ b/tests/initscripts/initscripts.py
@@ -43,7 +43,7 @@ class initscripts(AutoQATest):
f = open("/etc/yum.repos.d/beakerlib.repo", "w")
f.write("[beakerlib-testing]\nname=Testing repo for BeakerLib\nbaseurl=http://afri.fedorapeople.org/beakerlib/\nenabled=1\ngp...")
f.close()
- utils.system('yum -y install beakerlib')
+ utils.system('yum -y install beakerlib')
@ExceptionCatcher()
def initialize(self, config, **kwargs):
@@ -174,7 +174,9 @@ class initscripts(AutoQATest):
# email results to mailing list and to pkg owner if they optin
repo = repoinfo.getrepo_by_tag(kojitag)
- if repo is not None and autoqa.util.check_opt_in(name, repo['collection_name']):
+ send_optin_email = getbool(self.config.get('notifications', 'send_optin_email'))
+ if repo is not None and send_optin_email and \
+ autoqa.util.check_opt_in(pkg_name, repo['collection_name']):
#FIXME - hardcoded partial email address here - obviously sub-par
self.mail_to.append('%s-owner(a)fedoraproject.org' % name)
diff --git a/tests/rpmguard/rpmguard.py b/tests/rpmguard/rpmguard.py
index bf973f0..4a20d53 100644
--- a/tests/rpmguard/rpmguard.py
+++ b/tests/rpmguard/rpmguard.py
@@ -82,7 +82,9 @@ class rpmguard(AutoQATest):
# email results to mailing list and to pkg owner if they optin
repo = repoinfo.getrepo_by_tag(kojitag)
pkg_name = rpmUtils.miscutils.splitFilename(envr + '.noarch')[0]
- if repo is not None and autoqa.util.check_opt_in(pkg_name, repo['collection_name']):
+ send_optin_email = getbool(self.config.get('notifications', 'send_optin_email'))
+ if repo is not None and send_optin_email and \
+ autoqa.util.check_opt_in(pkg_name, repo['collection_name']):
#FIXME - hardcoded partial email address here - obviously sub-par
self.mail_to.append('%s-owner(a)fedoraproject.org' % pkg_name)
diff --git a/tests/rpmlint/rpmlint.py b/tests/rpmlint/rpmlint.py
index 1c06a8b..cac8a48 100644
--- a/tests/rpmlint/rpmlint.py
+++ b/tests/rpmlint/rpmlint.py
@@ -116,7 +116,9 @@ class rpmlint(AutoQATest):
# email results to mailing list and to pkg owner if they optin
repo = repoinfo.getrepo_by_tag(kojitag)
pkg_name = rpmUtils.miscutils.splitFilename(envr + '.noarch')[0]
- if repo is not None and autoqa.util.check_opt_in(pkg_name, repo['collection_name']):
+ send_optin_email = getbool(self.config.get('notifications', 'send_optin_email'))
+ if repo is not None and send_optin_email and \
+ autoqa.util.check_opt_in(pkg_name, repo['collection_name']):
#FIXME - hardcoded partial email address here - obviously sub-par
self.mail_to.append('%s-owner(a)fedoraproject.org' % pkg_name)
--
1.7.3.2
---
Thanks,
Martin
12 years, 9 months
