August 2010 - autoqa-devel - Fedora mailing-lists

[AutoQA] #213: remove collection_name from repoinfo.conf
by fedora-badges 29 Mar '11

29 Mar '11

#213: remove collection_name from repoinfo.conf ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- Once ticket #212 is complete we have no real need for collection_name anymore; remove it from repoinfo.conf. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/213> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #212: fix check_opt_in to not use collection_name
by fedora-badges 29 Mar '11

29 Mar '11

#212: fix check_opt_in to not use collection_name -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- collection_name was used to line up with the dist-cvs branch layout; now that we have switched to dist-git/fedpkg the branch names match the repo names used in repoinfo, so we have no need for collection_name. This will require modifying the existing autoqa-optin file structure to match the git layout. We can leave symlinks for the old ('F-14') distro names but use the new ones ('f14') as the canonical ones. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/212> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #204: add depcheck to post-bodhi-update hook
by fedora-badges 25 Jan '11

25 Jan '11

#204: add depcheck to post-bodhi-update hook --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Actually get the depcheck test running on post-bodhi-update. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/204> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #205: make depcheck submit karma/comments to bodhi
by fedora-badges 10 Jan '11

10 Jan '11

#205: make depcheck submit karma/comments to bodhi --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Modify the depcheck test so that it sends status info - karma, comments, whatever - to bodhi. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/205> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 10

[AutoQA] #201: Add mash support to depcheck
by fedora-badges 29 Nov '10

29 Nov '10

#201: Add mash support to depcheck -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- To properly handle possible multilib problems (e.g. the [https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000703.html nss-softokn problem] in June and the recent gfortran problem) we need to run mash on the set of proposed packages, to generate a proper multilibified repo to test against. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/201> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #153: rpmguard: Separate checks into individual python files
by fedora-badges 19 Nov '10

19 Nov '10

#153: rpmguard: Separate checks into individual python files -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: rpmguard | -------------------------+-------------------------------------------------- It would be great to have individual checks in rpmguard available as separate files, very similarly to what rpmlint or yum (-plugins) use. It would allow easy enabling/disabling of specific check, it would allow easy contribution of a new check. More ideas and information here: https://fedorahosted.org/pipermail /autoqa-devel/2010-May/000496.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/153> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #150: "artificial ignorance": suppress errors and warnings that aren't important
by fedora-badges 16 Nov '10

16 Nov '10

#150: "artificial ignorance": suppress errors and warnings that aren't important -------------------------+-------------------------------------------------- Reporter: dmalcolm | Owner: Type: enhancement | Status: new Priority: major | Milestone: Package Sanity Tests Component: tests | Version: 1.0 Keywords: | -------------------------+-------------------------------------------------- There are some interesting ideas in here about suppressing a logfile to show up the information that's truly important: http://www.ranum.com/security/computer_security/papers/ai/index.html I think that the rpmlint test needs to have a set of filters, and should suppress output matching certain patterns. For example https://fedorahosted.org/pipermail/autoqa-results/2010-April/013903.html shows: {{{ gnome-python2-gnome.i686: W: spelling-error Summary(en_US) libgnome -> lib gnome, lib-gnome, cognomen }}} i.e. it's complaining about a reference to "libgnome" within gnome- python2-gnome, which is clearly a false-positive. For this case we might have a python filtering process that says something like this (caveat: untested): {{{ patterns = [ ": W: spelling-error \S+ libgnome .*", ] def should_ignore(line): for pat in patterns # May want to precompile the patterns: m = re.match(pat, line) if m: return True # No match: return False for line in rpmlint_output if not should_ignore(line): print(line) }}} and then gradually grow patterns. The idea is to make it nice and lightweight, and Fedora QA can then easily take ownership of the suppresssions, without having to get patches into rpmlint. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/150> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #179: Initscript review: tigervnc
by fedora-badges 13 Oct '10

13 Oct '10

#179: Initscript review: tigervnc ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/179> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

[AutoQA] #167: Initscript review: mcstrans
by fedora-badges 13 Oct '10

13 Oct '10

#167: Initscript review: mcstrans ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/167> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #138: ResultsDB: media wiki as a storage for metadata about tests
by fedora-badges 13 Oct '10

13 Oct '10

#138: ResultsDB: media wiki as a storage for metadata about tests ----------------------------+----------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- Once #135 #136 and #137 are finished, we should create a provider/middle man (probably a library) which will allow us to get information about the respective test and use it for test execution/storing results/displaying results via frontends... -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/138> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #137: ResultsDB: propose structure for storing metadata about tests on wiki
by fedora-badges 13 Oct '10

13 Oct '10

#137: ResultsDB: propose structure for storing metadata about tests on wiki -----------------------+---------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- We want to use mediawiki as a storage for tests metadata. We should agree on the fields we want to store (e.g. test owner, destructive/non- destructive, average time to complete test etc.) and the format we'll use to store it (probably JSON though). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/137> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #170: Initscript review: net-tools
by fedora-badges 11 Oct '10

11 Oct '10

#170: Initscript review: net-tools ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/170> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #166: Initscript review: iputils
by fedora-badges 11 Oct '10

11 Oct '10

#166: Initscript review: iputils ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/166> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #134: ResultsDB: create resultsdb prototype
by fedora-badges 06 Oct '10

06 Oct '10

#134: ResultsDB: create resultsdb prototype ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Create prototype of database and software which will implement dbSchema from #133 and API from #131 This prototype should also include some test suite, which will be used to determine, if all the functionality is OK. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/134> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #202: testcases for depcheck
by fedora-badges 25 Sep '10

25 Sep '10

#202: testcases for depcheck --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- To be sure that depcheck is operating as expected, we should have a library of test cases for it. These test cases could take the form of a set (or sets) of trivial RPMs that illustrate various possible dependency problems - or a script that generates RPMs like that. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/202> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #221: Convert package repository to repos.fedorapeople.org/autoqa
by fedora-badges 16 Sep '10

16 Sep '10

#221: Convert package repository to repos.fedorapeople.org/autoqa -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: minor | Milestone: Packaging, Review, & Deployment Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- It would make me feel more comfortable having the repositories less tied to my name. It seems to transient that way, and for some reason, feels one step closer to permanent if the packages are housed at http://repos.fedorapeople.org -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/221> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #215: Rewrite the "how to write autoqa tests" wiki page
by fedora-badges 08 Sep '10

08 Sep '10

#215: Rewrite the "how to write autoqa tests" wiki page -----------------------+---------------------------------------------------- Reporter: jskladan | Owner: jskladan Type: task | Status: new Priority: major | Milestone: Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- Rewrite the "how to write autoqa tests" wiki page so it reflects the current features and concepts. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/215> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #203: make rpmguard run as a post-bodhi-update test too
by fedora-badges 06 Sep '10

06 Sep '10

#203: make rpmguard run as a post-bodhi-update test too --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Multi-hook tests Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Using the multihook capability described elsewhere, make rpmguard run for the post-bodhi-update hook. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/203> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

[PATCH] Polished upgradepath code - minor fixes.
by Josef Skladanka 31 Aug '10

31 Aug '10

--- tests/upgradepath/upgradepath.py | 44 ++++++++++++------------------------- 1 files changed, 14 insertions(+), 30 deletions(-) diff --git a/tests/upgradepath/upgradepath.py b/tests/upgradepath/upgradepath.py index 257eddc..a49fa07 100755 --- a/tests/upgradepath/upgradepath.py +++ b/tests/upgradepath/upgradepath.py @@ -36,17 +36,11 @@ from autotest_lib.client.common_lib import error class upgradepath(AutoQATest): version = 1 # increment if setup() changes - test_result = 'PASS' - log = [] - envr_list = set() - def initialize(self, config): - self.config = config_loader(config, self.tmpdir) - #URL of logs/results stored on autotest-server - self.autotest_url = autoqa.util.make_autotest_url(self.config) - - def setup(self): - pass + def __init__(self, *args, **kwargs): + super(upgradepath, self).__init__(*args, **kwargs) + self.log = [] + self.envr_list = set() def compare(self, matching_build, tags, op): koji = autoqa.koji_utils.SimpleKojiClientSession() @@ -68,7 +62,7 @@ class upgradepath(AutoQATest): print msg self.log.append(msg) else: - self.test_result = 'FAIL' + self.result = 'FAILED' msg ='{0:<7}{1}'.format('[FAIL]', tag) print msg self.log.append(msg) @@ -85,11 +79,11 @@ class upgradepath(AutoQATest): try: current_tag = repotags.index(kojitag) except ValueError: - self.test_result = 'FAIL' - msg = "ERROR: Entered bad kojitag" - print msg - self.log.append(msg) - self.envr_list.add(matching_build['envr']) + self.result = 'FAILED' + self.summary = "ERROR: Entered bad kojitag" + print self.summary + self.log.append(self.summary) + self.outputs = "\n".join(self.log) raise error.TestFail hi_tags = repotags[(current_tag + 1):] # tags higher than current @@ -136,20 +130,10 @@ class upgradepath(AutoQATest): self.log.append(msg) print msg - self.result = self.test_result - self.outputs = "" - for i in self.log: - self.outputs += i + '\n' - - self.outputs += '\n \n' - - packages_fail = "" - for i in self.envr_list: - packages_fail += i + ', ' - packages_fail = packages_fail[:-2] - - self.summary = packages_fail + self.outputs = "\n".join(self.log) + self.summary = ", ".join(self.envr_list) - if self.test_result == 'FAIL': + if self.result == 'FAILED': raise error.TestFail + self.result = "PASSED" -- 1.6.6.1

2 1

[AutoQA] #123: rpmguard: check upgrade paths between releases
by fedora-badges 31 Aug '10

31 Aug '10

#123: rpmguard: check upgrade paths between releases ----------------------+----------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: rpmguard | ----------------------+----------------------------------------------------- (As pointed out by spot on the devel list: http://lists.fedoraproject.org/pipermail/devel/2010-March/131746.html) We should be emitting warning/error messages if the new package(s) have a higher ENVR than the currently-released package(s) in any newer release. For example, right now a new F11 package should be checked against the corresponding package(s) in the current F12, branched F13, and rawhide repos. This involves a lot of extra repos and metadata, though. It's not a trivial thing to add. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/123> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #220: Check domains of running services after installation
by fedora-badges 30 Aug '10

30 Aug '10

#220: Check domains of running services after installation ---------------------+------------------------------------------------------ Reporter: adelton | Owner: Type: task | Status: new Priority: minor | Milestone: Component: tests | Version: 1.0 Keywords: | ---------------------+------------------------------------------------------ Recently I've came across issue https://bugzilla.redhat.com/show_bug.cgi?id=628495 when daemons run as initrc_t. In this case it was in spite of the fact that there was a SELinux policy module loaded, simply because the file context was pointing to a file that no longer existed in that rpm and which was no longer run during service startup. I believe that with older Fedoras and with older versions of that package, the file context was actually correct and the daemons run confined. So avoid any such regressions, it would be nice when doing some post- installation tests and starting services to also check the SELinux domain of the processes that are started by those services (ps Z). Thanks, Jan -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/220> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #111: depcheck test
by fedora-badges 27 Aug '10

27 Aug '10

#111: depcheck test --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: autoqa depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Write an actual depcheck test. This test should take, as inputs: * one or more new package builds, and * the name of a target repository for the new build(s) The test should examine the PRCO (provides/requires/conflicts/obsoletes) data for the new package(s) and the PRCO data for the target repository (and all its parent repos). The test should fail if the new builds would cause missing/broken dependencies or unresolveable conflicts in the target repo(s). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/111> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 8

Writing AutoQA Tests wiki page
by Josef Skladanka 27 Aug '10

27 Aug '10

Hello gang, I have a first spin of the new wiki page https://fedoraproject.org/wiki/User:Jskladan/Sandbox:Writing_autoqa_tests and I'd be glad if you could skim through the page, and post some feedback. Thanks Joza

2 1

Re: depcheck - next steps!
by Kamil Paral 26 Aug '10

26 Aug '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > > == write control/control.autoqa/test wrapper object == > > I took a stab at this in the depcheck branch but I'm pretty sure I > did > it wrong / left some stuff out. Kamil/Joza, if you guys can give me > some > help/pointers here I'd appreciate it... but don't worry, I'll send > the > patch to the list for a final review before I merge anything. > I'm a little confused by all those files in tests/depcheck. There is * depcheck * depcheck.py * depcheck_test.py By our conventions the AutoQA test object should be called "depcheck.py". Therefore we need to rename the actual script to something else, like "run_depcheck.py" or similar. It's not clear to me what the purpose of the current 'depcheck' file is, I think its functionality can be easily integrated into the main depcheck script itself (the one with proposed name "run_depcheck.py"), can't it?

2 1

[AutoQA] #219: depcheck: NameError: global name 'rpmfluff' is not defined
by fedora-badges 24 Aug '10

24 Aug '10

#219: depcheck: NameError: global name 'rpmfluff' is not defined --------------------+------------------------------------------------------- Reporter: kparal | Owner: wwoods Type: defect | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- {{{ # ~autotest/client/site_tests/depcheck/depcheck --selftest /usr/lib/python2.6/site-packages/rpmfluff.py:58: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2 EEEEEEEE ====================================================================== ERROR: test_createrepo (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 277, in test_createrepo rpmlist = simple_rpmlist() File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 249, in simple_rpmlist m = rpmfluff.SimpleRpmBuild('mint', '1.0', '1') NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: test_rpmfluff (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 260, in test_rpmfluff p = rpmfluff.SimpleRpmBuild(n, v, r) NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: test_simple_yum_setup (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 290, in test_simple_yum_setup arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Test a changed Provides: ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Make sure we accept a consistent repo, and no updates at all ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Make sure depcheck accepts a good update as expected ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: make sure depcheck catches updates with missing requires ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Test with one good and one bad update ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ---------------------------------------------------------------------- Ran 8 tests in 0.002s FAILED (errors=8) }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/219> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel August 2010