autoqa-devel July 2011

autoqa-devel@lists.fedorahosted.org

12 participants
91 discussions

[AutoQA] #352: Determine AutoQA Functional Self Test Cases
by fedora-badges 05 Aug '11

05 Aug '11

#352: Determine AutoQA Functional Self Test Cases --------------------+------------------------------------------------------- Reporter: tflink | Owner: Type: task | Status: new Priority: major | Milestone: Self Test Component: core | Keywords: --------------------+------------------------------------------------------- In order to start forming a better self test suite for AutoQA, we need to start cataloging situations that will make for good functional regression tests. Instead of focusing on how to get them all run, the idea is to describe the situations that would make for good tests. Formatting and categorizing the tests will be left for another ticket. Examples of jobs that have caused problems in the past or continue to cause strange problems would be helpful. An extremely simplistic example for depcheck might look like: === Package Descriptions === || '''Package''' || '''Depends On''' || || foo-1.0-1 || bar-1.0-1 || || foo-2.0-1 || bar-2.0-1 || === Repo Contents === || '''Name''' || '''Packages''' || || stable || foo-1.0-1, bar-1.0-1 || || pending || foo-2.0-1 || === Expected Results === Failure because foo-2.0-1 depends on bar-2.0-1 which is not available. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/352> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #330: upgradepath: improve behavior when pushing update to multiple releases
by fedora-badges 04 Aug '11

04 Aug '11

#330: upgradepath: improve behavior when pushing update to multiple releases -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: 0.5.0 Component: tests | Keywords: -------------------------+-------------------------------------------------- This ticket stems from my talk to mcepl and our following discussion: https://fedorahosted.org/pipermail/autoqa-devel/2011-May/002215.html Package maintainers usually push fixes to many Fedora releases at once. Upgradepath usually fails for all but the highest one. We should improve the output. 1. If upgradepath passes, no changes. 2. If upgradepath fails, check update-pending repos for the failed sections and determine whether the same proposed update is available there. If it is, check upgradepath again and determine whether test would pass if all those proposed updates for all releases were pushed at the same time. Inform the user about it. 3. If the condition in 2) is valid, change FAIL result to NEEDS_INSPECTION. I'll gladly take this ticket when I finish my current work. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/330> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #348: anaconda upgrade test
by fedora-badges 04 Aug '11

04 Aug '11

#348: anaconda upgrade test ----------------------+----------------------------------------------------- Reporter: hongqing | Owner: hongqing Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: core | Keywords: ----------------------+----------------------------------------------------- To implement test case: https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_New_Bootloader . we have already implemented Fedora installation automation from DVD, next step we will upgrade the previous release. "virsh' will be used to operate on the guest. [[BR]] The tasks include:[[BR]] * define upgrade steps [[BR]] * install previous release [[BR]] * modify guest xml to boot from new release[[BR]] * upgrade -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/348> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #354: Custom 404 page for autoqa.fp.o
by fedora-badges 03 Aug '11

03 Aug '11

#354: Custom 404 page for autoqa.fp.o ----------------------+----------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: minor | Milestone: 0.6.0 Component: autotest | Keywords: ----------------------+----------------------------------------------------- Quoting tflink: For some reason, this came to mind when I was thinking about the bodhi link issue but how do we feel about putting a custom 404 on autoqa.fp.o? Since we don't keep logs forever, anything that links to old results goes to a generic 404 without more explanation than "could not find the file you were looking for". Instead, we could do a custom 404 that contains: - Explanation of log retention policy - Relevant AutoQA or FedoraQA links It would involve a small change to the apache config and some static HTML on the autotest server. Pretty easy finger food :) https://fedorahosted.org/pipermail/autoqa-devel/2011-July/002546.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/354> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 11

[AutoQA] #102: post-iso-build hook
by fedora-badges 03 Aug '11

03 Aug '11

#102: post-iso-build hook ----------------------+----------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: watchers | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- We need one (or more) standard locations where new iso images get posted for testing, and a watcher that can monitor for new iso images and launch appropriate installation/sanity tests. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/102> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #209: complete auto CD install test driver
by fedora-badges 02 Aug '11

02 Aug '11

#209: complete auto CD install test driver -------------------+-------------------------------------------------------- Reporter: liam | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | -------------------+-------------------------------------------------------- complete the cd_install.py as dvd_install.py to support CD install, with some functions like automatically swap CD when CD1 install was completed. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/209> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #365: update mediakit_sanity.py to accept different images, DVD, Live, netinst.iso
by fedora-badges 02 Aug '11

02 Aug '11

#365: update mediakit_sanity.py to accept different images, DVD, Live, netinst.iso -------------------------+-------------------------------------------------- Reporter: hongqing | Owner: hongqing Type: enhancement | Status: new Priority: minor | Milestone: Automate installation test plan Component: tests | Keywords: -------------------------+-------------------------------------------------- mediakit_sanity test covers the DVD.iso sanity test cases. But the Live.iso and netinst.iso are different with DVD.iso, they cannot be done repoclosure and fileconficts tests. mediakit_sanity should be updated to accept Live.iso and netinst.iso sanity tests. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/365> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

Very Rough Code for Mock Fedora Infrastructure
by Tim Flink 01 Aug '11

01 Aug '11

This kept falling off my TODO list, but I finally got what I have for initial mock Fedora infrastructure code up on github:

4 6

[AutoQA] #364: test.py: 'Unknown result keyword' handling in cmp_result()
by fedora-badges 01 Aug '11

01 Aug '11

#364: test.py: 'Unknown result keyword' handling in cmp_result() ----------------------+----------------------------------------------------- Reporter: pschindl | Owner: Type: defect | Status: new Priority: minor | Milestone: Finger Food Component: core | Keywords: ----------------------+----------------------------------------------------- Test.py raise exception when unknown result keyword is used in PrettyLog.summary_detail dictionary: {{{ Traceback (most recent call last): File "/usr/share/autotest/client/bin/parallel.py", line 18, in fork_start l() File "/usr/share/autotest/client/bin/job.py", line 522, in <lambda> l = lambda : test.runtest(self, url, tag, args, dargs) File "/usr/share/autotest/client/bin/test.py", line 115, in runtest job.sysinfo.log_after_each_iteration) File "/usr/share/autotest/client/common_lib/test.py", line 710, in runtest mytest._exec(args, dargs) File "/usr/share/autotest/client/common_lib/test.py", line 419, in _exec _call_test_function(self.execute, *p_args, **p_dargs) File "/usr/share/autotest/client/common_lib/test.py", line 630, in _call_test_function raise error.UnhandledTestFail(e) UnhandledTestFail: Unhandled ValueError: tuple.index(x): x not in tuple Traceback (most recent call last): File "/usr/share/autotest/client/common_lib/test.py", line 623, in _call_test_function return func(*args, **dargs) File "/usr/share/autotest/client/common_lib/test.py", line 291, in execute postprocess_profiled_run, args, dargs) File "/usr/share/autotest/client/common_lib/test.py", line 215, in _call_run_once self.postprocess_iteration() File "/usr/lib/python2.7/site-packages/autoqa/test.py", line 135, in postprocess_iteration self.post_results() File "/usr/lib/python2.7/site-packages/autoqa/test.py", line 183, in post_results logname = self._write_log(test_detail) File "/usr/lib/python2.7/site-packages/autoqa/test.py", line 313, in _write_log log.write(test_detail.create_output(full_log=full_log)) File "/usr/lib/python2.7/site-packages/autoqa/test.py", line 561, in create_output return self.log.create() File "/usr/lib/python2.7/site-packages/autoqa/pretty_log.py", line 245, in create summary_detail = self._make_summary_detail() File "/usr/lib/python2.7/site-packages/autoqa/pretty_log.py", line 87, in _make_summary_detail cmp=lambda x,y: td.cmp_result(x,y)) File "/usr/lib/python2.7/site-packages/autoqa/pretty_log.py", line 87, in <lambda> cmp=lambda x,y: td.cmp_result(x,y)) File "/usr/lib/python2.7/site-packages/autoqa/test.py", line 499, in cmp_result index1 = self.result_order.index(result1) if result1 is not None else -1 ValueError: tuple.index(x): x not in tuple }}} Problem is in cmp_result() from test.py. It compares only results which are in result_order tuple. We can write some reasonable error message or we can rewrite cmp_result to handle this problem (then it is needed to check if it won't break something). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/364> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

[AutoQA] #349: Depcheck - prefilter the list of packages passed to yum instance's rpmdb
by fedora-badges 01 Aug '11

01 Aug '11

#349: Depcheck - prefilter the list of packages passed to yum instance's rpmdb ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: 0.6.0 Component: core | Keywords: ----------------------+----------------------------------------------------- Recent problem with libreport <http://autoqa.fedoraproject.org/results/140482-autotest/172.16.0.20/depchec…> is most probably caused by depcheck selecting too old packages (i.e. ignoring the newest packages in repos) due to the fact that YUM selects packages alphabeticaly on fist-comes-first served basis. Because YUM is hacked into believing that all packages are installed by passing the content of _whole_ pkgsack into the rpmdb, it contains multiple versions of the same package. I belive, that this can be easily solved by pre-filtering the list, so it contains only the newest from each package. <code> # This YumBase object will act like all repo packages are already installed y = YumDepcheck() y.tsInfo = yum.transactioninfo.TransactionData() y.tsInfo.debug = 1 # this line is causing the problems y.rpmdb = MetaSackRPMDB(yum_repos.pkgSack) y.xsack = yum.packageSack.PackageSack() # Hacky way to set up the databases (this is copied from yum's testbase.py) y.tsInfo.setDatabases(y.rpmdb, y.xsack) </code> the yum_repos.pkgSack is a list of tuples NAEVR <code>[('libreport', 'x86_64', '0', '2.0.2', '5.fc15'), ('libreport', 'i686', '0', '2.0.2', '5.fc15'), ('libreport', 'x86_64', '0', '2.0.4', '1.fc15'), ('libreport', 'i686', '0', '2.0.4', '1.fc15')]</code> After the filtering, only the newest package for each architecture should be left. Newest = the first while sorting using E,V,R as keys with decreasing importance in this order. I.e. <code> [('libreport', 'x86_64', '0', '2.0.4', '1.fc15'), ('libreport', 'i686', '0', '2.0.4', '1.fc15')]</code>. koji_utils.compareEVR() can be used to compare EVRs. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/349> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel July 2011