August 2010 - autoqa-devel - Fedora mailing-lists

[AutoQA] #191: Submit autotest for Fedora package approval
by fedora-badges 26 Sep '12

26 Sep '12

#191: Submit autotest for Fedora package approval -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Packaging and Review Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- * https://bugzilla.redhat.com/show_bug.cgi?id=548522 Currently stalled, pending packaging of the gwt package. This includes packaging all bundled .jar files included with gwt. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/191> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #157: conflicts: requested arch not respected
by fedora-badges 06 Aug '12

06 Aug '12

#157: conflicts: requested arch not respected --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This is a bug in the potential_conflict.py script in tests/conflicts/. Usage suggests using --arch options for specifying requested architecture to be tested. But if you look at the attached log, the first and second output match (that's correct), but the third doesn't match, even though it should. Everything was tested on x86_64 machine. Also from the usage line it is not clear whether --arch is related only to particular yum repository selection (replacing $basearch keyword), or whether it also relates to packages built for one architecture but being in different repo (e.g. i686 packages are very often in x86_64 repo). I have looked into the source code of the script and it seems that the --arch option is not used at all, just printed out in the usage help?? Seth, you're listed as script author. Any comments on that? Thanks. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/157> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #206: Update bodhi to enforce Package Update Acceptance policy
by fedora-badges 26 Mar '12

26 Mar '12

#206: Update bodhi to enforce Package Update Acceptance policy ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- In order to enforce the Package Update Acceptance Test Plan, Bodhi will need be modified to reject the push of any package that fails acceptance testing. The acceptance tests may need to send status/data to bodhi in order for it to make decisions about policy (see e.g. ticket #205). Later bodhi might just get data from resultdb (see that milestone for details). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/206> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #186: Automate media kit sanity tests
by fedora-badges 05 Mar '12

05 Mar '12

#186: Automate media kit sanity tests --------------------+------------------------------------------------------- Reporter: jlaska | Owner: lili Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This ticket is intended to track automating the mediakit sanity tests (http://fedoraproject.org/wiki/Category:Installer_Image_Sanity_Test_Cases) Discussion already underway, along with code in Liam's private branch. For details see https://fedorahosted.org/pipermail/autoqa- devel/2010-June/000595.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/186> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

[AutoQA] #194: Incorporate automated anaconda storage suite
by fedora-badges 05 Mar '12

05 Mar '12

#194: Incorporate automated anaconda storage suite --------------------+------------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Chris Lumens has developed a virt-based test suite to automate different disk/storage install scenarios. I've discussed this briefly with Chris, but I'd be interested in seeing his test suite incorporated within autoqa and run on a regular interval (perhaps along-side Liam's install tests). The tests are currently available at http://clumens.fedorapeople.org /anaconda-storage-test/ -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/194> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #182: Write virt management script
by fedora-badges 22 Nov '11

22 Nov '11

#182: Write virt management script ----------------------------+----------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- We must write a virt management script that will be able to the basic tasks for us: * query for available virt systems that we can use * switch between "real" disk and a disk snapshot for a virt system * revert a disk snapshot for a virt system * perform some command in a virt system - e.g. "yum update" for keeping it periodically up-to-date * install/reinstall a virt-system for us with specified distro * (some more to come?) We may want to use some pre-defined names syntax to recognize what virt systems do we have. For example: {{{ /dev/vg_autoqa/F12_i686_1 /dev/vg_autoqa/F12_i686_1-snap }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/182> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

2 4

[AutoQA] #196: Log failed job scheduling
by fedora-badges 11 Nov '11

11 Nov '11

#196: Log failed job scheduling -------------------------+-------------------------------------------------- Reporter: kparal | Owner: kparal Type: enhancement | Status: new Priority: major | Milestone: Component: harness | Version: 1.0 Keywords: | -------------------------+-------------------------------------------------- Currently when we schedule a job and it fails, we just print it out and forget about it: {{{ retval = schedule_job(control, email=email, name=testname, required_arch=arch, dryrun=opts.dryrun, labels=aq_labels) if retval != 0: print "ERROR: failed to schedule job %s" % testname }}} That should not be the case. We should: * use syslog and log important stuff to /var/log/autoqa.log or similar * send an email about failed scheduling to autoqa-results That way we will know something is wrong and will be able to fix it. Otherwise it just silently fails and we don't know about it. Feel free to reassign the ticket if you want to work on it. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/196> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #192: Submit autoqa for Fedora package approval
by fedora-badges 13 Oct '11

13 Oct '11

#192: Submit autoqa for Fedora package approval -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Packaging and Review Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- Currently stalled, pending packaging of the gwt and autotest package. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/192> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #180: Initscript review: vsftpd
by fedora-badges 10 Oct '11

10 Oct '11

#180: Initscript review: vsftpd ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/180> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #178: Initscript review: squid
by fedora-badges 10 Oct '11

10 Oct '11

#178: Initscript review: squid ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/178> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #177: Initscript review: setroubleshoot
by fedora-badges 10 Oct '11

10 Oct '11

#177: Initscript review: setroubleshoot ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/177> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #175: Initscript review: postgresql
by fedora-badges 10 Oct '11

10 Oct '11

#175: Initscript review: postgresql ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/175> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #174: Initscript review: openswan
by fedora-badges 10 Oct '11

10 Oct '11

#174: Initscript review: openswan ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/174> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #173: Initscript review: openldap
by fedora-badges 10 Oct '11

10 Oct '11

#173: Initscript review: openldap ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/173> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #171: Initscript review: nss-pam-ldapd
by fedora-badges 10 Oct '11

10 Oct '11

#171: Initscript review: nss-pam-ldapd ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/171> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #169: Initscript review: net-snmp
by fedora-badges 10 Oct '11

10 Oct '11

#169: Initscript review: net-snmp ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/169> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #125: initscripts: Choosing the right repo for dependencies
by fedora-badges 10 Oct '11

10 Oct '11

#125: initscripts: Choosing the right repo for dependencies -------------------------+-------------------------------------------------- Reporter: kparal | Owner: jskladan Type: task | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: initscripts | -------------------------+-------------------------------------------------- Currently after new Koji build the initscripts test downloads all relevant binary packages (related to the particular source package) and installs them. But the RPM dependencies are installed from stable/updates repos, not from Koji. Also the dependencies parsed out of Makefile are installed from stable/updates repos. We have to inquire a little more into this situation. Is it sufficient/desirable? Or is it desirable to download dependencies from updates-testing or bleeding edge Koji? May the initscripts test sometimes fail because of insufficient dependencies are in stable/updates repos? This also depends on cases when we are going to use initscripts test. If it will be used with Bodhi hook, it will be probably easier to solve it than when used with Koji hook. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/125> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #102: post-iso-build hook
by fedora-badges 02 Aug '11

02 Aug '11

#102: post-iso-build hook ----------------------+----------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: watchers | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- We need one (or more) standard locations where new iso images get posted for testing, and a watcher that can monitor for new iso images and launch appropriate installation/sanity tests. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/102> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #209: complete auto CD install test driver
by fedora-badges 02 Aug '11

02 Aug '11

#209: complete auto CD install test driver -------------------+-------------------------------------------------------- Reporter: liam | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | -------------------+-------------------------------------------------------- complete the cd_install.py as dvd_install.py to support CD install, with some functions like automatically swap CD when CD1 install was completed. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/209> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #149: rpmlint: Provide whitelisting of messages
by fedora-badges 28 Jul '11

28 Jul '11

#149: rpmlint: Provide whitelisting of messages --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Package update tests Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- We need to provide some whitelisting capability of rpmlint packages, so the package maintainers can mark some messages as invalid. We will also probably want to mark some messages as invalid by default for the whole Fedora project. As an interesting inspiration, look at lintian project (validation of deb packages). It provides some "overrides" on a package basis, you just create a definition file in /etc/lintian/package_name or kind of. I have the feeling that rpmlint provides something similar, but we need to discover it more properly. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/149> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #104: virtguest.py: accept iso image(s) as install location
by fedora-badges 21 Jul '11

21 Jul '11

#104: virtguest.py: accept iso image(s) as install location --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- allow {{{VirtGuest.create()}}} to use an iso image (or images) for its {{{location}}} arg, and automatically choose the appropriate {{{--location URL}}} or {{{--cdrom IMAGE}}} flag. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/104> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #183: Write a script and a daemon to send signals between virt machine and host
by fedora-badges 11 Jul '11

11 Jul '11

#183: Write a script and a daemon to send signals between virt machine and host ----------------------------+----------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- We need to exchange some signals between virt machine and a host. For example "autotest job finished, need to revert to previous state". We need to write simple script that will send this signal for virt machine and a daemon that will receive this signal in the host machine and act accordingly. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/183> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #208: update minimon to transport logs between guest and host via virtio
by fedora-badges 20 May '11

20 May '11

#208: update minimon to transport logs between guest and host via virtio -------------------+-------------------------------------------------------- Reporter: liam | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | -------------------+-------------------------------------------------------- Since minimon has to use network during test, some test cases will not activate network during test,in this case, minimon can not transport logs to host, test also be identified fail even it was successfully run. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/208> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #214: conflicts: fails with parent repositories
by fedora-badges 22 Apr '11

22 Apr '11

#214: conflicts: fails with parent repositories --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- These two commands work (although the package counts are somewhat weirdly same): {{{ # repoclosure --tempcache --newest --repofrompath=target,http://download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os --repoid=target Added target repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os Reading in repository metadata - please wait.... Checking Dependencies /usr/lib/python2.6/site-packages/yum/packages.py:401: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal if prcotuple in self.prco[prcotype]: Repos looked at: 1 target Num Packages in Repos: 20806 }}} {{{ # repoclosure --tempcache --newest --repofrompath=target,http://download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 --repoid=target Added target repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 Reading in repository metadata - please wait.... Checking Dependencies /usr/lib/python2.6/site-packages/yum/packages.py:401: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal if prcotuple in self.prco[prcotype]: Repos looked at: 1 target Num Packages in Repos: 20806 }}} But if you combine it: {{{ # repoclosure --tempcache --newest --repofrompath=target,http://download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 --repoid=target --repofrompath=parent1,http://download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os --repoid=parent1 Added target repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 Added parent1 repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os Reading in repository metadata - please wait.... Cannot retrieve repository metadata (repomd.xml) for repository: parent1. Please verify its path and try again Some dependencies may not be complete for this repository Run as root to get all dependencies or use -t to enable a user temp cache Checking Dependencies Cannot retrieve repository metadata (repomd.xml) for repository: parent1. Please verify its path and try again }}} Something is broken here. {{{ yum-utils-1.1.28-1.fc13.noarch yum-metadata-parser-1.1.4-1.fc13.x86_64 yum-3.2.27-4.fc13.noarch }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/214> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #218: re-use fedora-qa script for checking PackageReviewGuidelines
by fedora-badges 07 Apr '11

07 Apr '11

#218: re-use fedora-qa script for checking PackageReviewGuidelines --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Future test cases Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Aurélien Bompard has written a "fedora-qa" tool, which checks all the MUST items in !PackageReviewGuidelines that are not covered by rpmlint. http://fedoraproject.org/wiki/AurelienBompard#QA There's a high possibility that we could re-use parts of his code for several new AutoQA test cases. Worth investigating. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/218> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #213: remove collection_name from repoinfo.conf
by fedora-badges 29 Mar '11

29 Mar '11

#213: remove collection_name from repoinfo.conf ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- Once ticket #212 is complete we have no real need for collection_name anymore; remove it from repoinfo.conf. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/213> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #212: fix check_opt_in to not use collection_name
by fedora-badges 29 Mar '11

29 Mar '11

#212: fix check_opt_in to not use collection_name -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- collection_name was used to line up with the dist-cvs branch layout; now that we have switched to dist-git/fedpkg the branch names match the repo names used in repoinfo, so we have no need for collection_name. This will require modifying the existing autoqa-optin file structure to match the git layout. We can leave symlinks for the old ('F-14') distro names but use the new ones ('f14') as the canonical ones. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/212> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #204: add depcheck to post-bodhi-update hook
by fedora-badges 25 Jan '11

25 Jan '11

#204: add depcheck to post-bodhi-update hook --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Actually get the depcheck test running on post-bodhi-update. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/204> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #205: make depcheck submit karma/comments to bodhi
by fedora-badges 10 Jan '11

10 Jan '11

#205: make depcheck submit karma/comments to bodhi --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Modify the depcheck test so that it sends status info - karma, comments, whatever - to bodhi. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/205> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 10

[AutoQA] #201: Add mash support to depcheck
by fedora-badges 29 Nov '10

29 Nov '10

#201: Add mash support to depcheck -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- To properly handle possible multilib problems (e.g. the [https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000703.html nss-softokn problem] in June and the recent gfortran problem) we need to run mash on the set of proposed packages, to generate a proper multilibified repo to test against. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/201> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #153: rpmguard: Separate checks into individual python files
by fedora-badges 19 Nov '10

19 Nov '10

#153: rpmguard: Separate checks into individual python files -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: rpmguard | -------------------------+-------------------------------------------------- It would be great to have individual checks in rpmguard available as separate files, very similarly to what rpmlint or yum (-plugins) use. It would allow easy enabling/disabling of specific check, it would allow easy contribution of a new check. More ideas and information here: https://fedorahosted.org/pipermail /autoqa-devel/2010-May/000496.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/153> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #150: "artificial ignorance": suppress errors and warnings that aren't important
by fedora-badges 16 Nov '10

16 Nov '10

#150: "artificial ignorance": suppress errors and warnings that aren't important -------------------------+-------------------------------------------------- Reporter: dmalcolm | Owner: Type: enhancement | Status: new Priority: major | Milestone: Package Sanity Tests Component: tests | Version: 1.0 Keywords: | -------------------------+-------------------------------------------------- There are some interesting ideas in here about suppressing a logfile to show up the information that's truly important: http://www.ranum.com/security/computer_security/papers/ai/index.html I think that the rpmlint test needs to have a set of filters, and should suppress output matching certain patterns. For example https://fedorahosted.org/pipermail/autoqa-results/2010-April/013903.html shows: {{{ gnome-python2-gnome.i686: W: spelling-error Summary(en_US) libgnome -> lib gnome, lib-gnome, cognomen }}} i.e. it's complaining about a reference to "libgnome" within gnome- python2-gnome, which is clearly a false-positive. For this case we might have a python filtering process that says something like this (caveat: untested): {{{ patterns = [ ": W: spelling-error \S+ libgnome .*", ] def should_ignore(line): for pat in patterns # May want to precompile the patterns: m = re.match(pat, line) if m: return True # No match: return False for line in rpmlint_output if not should_ignore(line): print(line) }}} and then gradually grow patterns. The idea is to make it nice and lightweight, and Fedora QA can then easily take ownership of the suppresssions, without having to get patches into rpmlint. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/150> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #179: Initscript review: tigervnc
by fedora-badges 13 Oct '10

13 Oct '10

#179: Initscript review: tigervnc ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/179> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

[AutoQA] #167: Initscript review: mcstrans
by fedora-badges 13 Oct '10

13 Oct '10

#167: Initscript review: mcstrans ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/167> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #138: ResultsDB: media wiki as a storage for metadata about tests
by fedora-badges 13 Oct '10

13 Oct '10

#138: ResultsDB: media wiki as a storage for metadata about tests ----------------------------+----------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- Once #135 #136 and #137 are finished, we should create a provider/middle man (probably a library) which will allow us to get information about the respective test and use it for test execution/storing results/displaying results via frontends... -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/138> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #137: ResultsDB: propose structure for storing metadata about tests on wiki
by fedora-badges 13 Oct '10

13 Oct '10

#137: ResultsDB: propose structure for storing metadata about tests on wiki -----------------------+---------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- We want to use mediawiki as a storage for tests metadata. We should agree on the fields we want to store (e.g. test owner, destructive/non- destructive, average time to complete test etc.) and the format we'll use to store it (probably JSON though). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/137> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #170: Initscript review: net-tools
by fedora-badges 11 Oct '10

11 Oct '10

#170: Initscript review: net-tools ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/170> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #166: Initscript review: iputils
by fedora-badges 11 Oct '10

11 Oct '10

#166: Initscript review: iputils ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/166> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #134: ResultsDB: create resultsdb prototype
by fedora-badges 06 Oct '10

06 Oct '10

#134: ResultsDB: create resultsdb prototype ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Create prototype of database and software which will implement dbSchema from #133 and API from #131 This prototype should also include some test suite, which will be used to determine, if all the functionality is OK. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/134> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #202: testcases for depcheck
by fedora-badges 24 Sep '10

24 Sep '10

#202: testcases for depcheck --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- To be sure that depcheck is operating as expected, we should have a library of test cases for it. These test cases could take the form of a set (or sets) of trivial RPMs that illustrate various possible dependency problems - or a script that generates RPMs like that. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/202> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #221: Convert package repository to repos.fedorapeople.org/autoqa
by fedora-badges 16 Sep '10

16 Sep '10

#221: Convert package repository to repos.fedorapeople.org/autoqa -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: minor | Milestone: Packaging, Review, & Deployment Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- It would make me feel more comfortable having the repositories less tied to my name. It seems to transient that way, and for some reason, feels one step closer to permanent if the packages are housed at http://repos.fedorapeople.org -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/221> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #215: Rewrite the "how to write autoqa tests" wiki page
by fedora-badges 08 Sep '10

08 Sep '10

#215: Rewrite the "how to write autoqa tests" wiki page -----------------------+---------------------------------------------------- Reporter: jskladan | Owner: jskladan Type: task | Status: new Priority: major | Milestone: Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- Rewrite the "how to write autoqa tests" wiki page so it reflects the current features and concepts. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/215> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #203: make rpmguard run as a post-bodhi-update test too
by fedora-badges 06 Sep '10

06 Sep '10

#203: make rpmguard run as a post-bodhi-update test too --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Multi-hook tests Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Using the multihook capability described elsewhere, make rpmguard run for the post-bodhi-update hook. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/203> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

[PATCH] Polished upgradepath code - minor fixes.
by Josef Skladanka 31 Aug '10

31 Aug '10

--- tests/upgradepath/upgradepath.py | 44 ++++++++++++------------------------- 1 files changed, 14 insertions(+), 30 deletions(-) diff --git a/tests/upgradepath/upgradepath.py b/tests/upgradepath/upgradepath.py index 257eddc..a49fa07 100755 --- a/tests/upgradepath/upgradepath.py +++ b/tests/upgradepath/upgradepath.py @@ -36,17 +36,11 @@ from autotest_lib.client.common_lib import error class upgradepath(AutoQATest): version = 1 # increment if setup() changes - test_result = 'PASS' - log = [] - envr_list = set() - def initialize(self, config): - self.config = config_loader(config, self.tmpdir) - #URL of logs/results stored on autotest-server - self.autotest_url = autoqa.util.make_autotest_url(self.config) - - def setup(self): - pass + def __init__(self, *args, **kwargs): + super(upgradepath, self).__init__(*args, **kwargs) + self.log = [] + self.envr_list = set() def compare(self, matching_build, tags, op): koji = autoqa.koji_utils.SimpleKojiClientSession() @@ -68,7 +62,7 @@ class upgradepath(AutoQATest): print msg self.log.append(msg) else: - self.test_result = 'FAIL' + self.result = 'FAILED' msg ='{0:<7}{1}'.format('[FAIL]', tag) print msg self.log.append(msg) @@ -85,11 +79,11 @@ class upgradepath(AutoQATest): try: current_tag = repotags.index(kojitag) except ValueError: - self.test_result = 'FAIL' - msg = "ERROR: Entered bad kojitag" - print msg - self.log.append(msg) - self.envr_list.add(matching_build['envr']) + self.result = 'FAILED' + self.summary = "ERROR: Entered bad kojitag" + print self.summary + self.log.append(self.summary) + self.outputs = "\n".join(self.log) raise error.TestFail hi_tags = repotags[(current_tag + 1):] # tags higher than current @@ -136,20 +130,10 @@ class upgradepath(AutoQATest): self.log.append(msg) print msg - self.result = self.test_result - self.outputs = "" - for i in self.log: - self.outputs += i + '\n' - - self.outputs += '\n \n' - - packages_fail = "" - for i in self.envr_list: - packages_fail += i + ', ' - packages_fail = packages_fail[:-2] - - self.summary = packages_fail + self.outputs = "\n".join(self.log) + self.summary = ", ".join(self.envr_list) - if self.test_result == 'FAIL': + if self.result == 'FAILED': raise error.TestFail + self.result = "PASSED" -- 1.6.6.1

2 1

[AutoQA] #123: rpmguard: check upgrade paths between releases
by fedora-badges 31 Aug '10

31 Aug '10

#123: rpmguard: check upgrade paths between releases ----------------------+----------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: rpmguard | ----------------------+----------------------------------------------------- (As pointed out by spot on the devel list: http://lists.fedoraproject.org/pipermail/devel/2010-March/131746.html) We should be emitting warning/error messages if the new package(s) have a higher ENVR than the currently-released package(s) in any newer release. For example, right now a new F11 package should be checked against the corresponding package(s) in the current F12, branched F13, and rawhide repos. This involves a lot of extra repos and metadata, though. It's not a trivial thing to add. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/123> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #220: Check domains of running services after installation
by fedora-badges 30 Aug '10

30 Aug '10

#220: Check domains of running services after installation ---------------------+------------------------------------------------------ Reporter: adelton | Owner: Type: task | Status: new Priority: minor | Milestone: Component: tests | Version: 1.0 Keywords: | ---------------------+------------------------------------------------------ Recently I've came across issue https://bugzilla.redhat.com/show_bug.cgi?id=628495 when daemons run as initrc_t. In this case it was in spite of the fact that there was a SELinux policy module loaded, simply because the file context was pointing to a file that no longer existed in that rpm and which was no longer run during service startup. I believe that with older Fedoras and with older versions of that package, the file context was actually correct and the daemons run confined. So avoid any such regressions, it would be nice when doing some post- installation tests and starting services to also check the SELinux domain of the processes that are started by those services (ps Z). Thanks, Jan -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/220> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #111: depcheck test
by fedora-badges 27 Aug '10

27 Aug '10

#111: depcheck test --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: autoqa depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Write an actual depcheck test. This test should take, as inputs: * one or more new package builds, and * the name of a target repository for the new build(s) The test should examine the PRCO (provides/requires/conflicts/obsoletes) data for the new package(s) and the PRCO data for the target repository (and all its parent repos). The test should fail if the new builds would cause missing/broken dependencies or unresolveable conflicts in the target repo(s). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/111> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 8

Writing AutoQA Tests wiki page
by Josef Skladanka 27 Aug '10

27 Aug '10

Hello gang, I have a first spin of the new wiki page https://fedoraproject.org/wiki/User:Jskladan/Sandbox:Writing_autoqa_tests and I'd be glad if you could skim through the page, and post some feedback. Thanks Joza

2 1

Re: depcheck - next steps!
by Kamil Paral 26 Aug '10

26 Aug '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > > == write control/control.autoqa/test wrapper object == > > I took a stab at this in the depcheck branch but I'm pretty sure I > did > it wrong / left some stuff out. Kamil/Joza, if you guys can give me > some > help/pointers here I'd appreciate it... but don't worry, I'll send > the > patch to the list for a final review before I merge anything. > I'm a little confused by all those files in tests/depcheck. There is * depcheck * depcheck.py * depcheck_test.py By our conventions the AutoQA test object should be called "depcheck.py". Therefore we need to rename the actual script to something else, like "run_depcheck.py" or similar. It's not clear to me what the purpose of the current 'depcheck' file is, I think its functionality can be easily integrated into the main depcheck script itself (the one with proposed name "run_depcheck.py"), can't it?

2 1

[AutoQA] #219: depcheck: NameError: global name 'rpmfluff' is not defined
by fedora-badges 24 Aug '10

24 Aug '10

#219: depcheck: NameError: global name 'rpmfluff' is not defined --------------------+------------------------------------------------------- Reporter: kparal | Owner: wwoods Type: defect | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- {{{ # ~autotest/client/site_tests/depcheck/depcheck --selftest /usr/lib/python2.6/site-packages/rpmfluff.py:58: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2 EEEEEEEE ====================================================================== ERROR: test_createrepo (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 277, in test_createrepo rpmlist = simple_rpmlist() File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 249, in simple_rpmlist m = rpmfluff.SimpleRpmBuild('mint', '1.0', '1') NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: test_rpmfluff (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 260, in test_rpmfluff p = rpmfluff.SimpleRpmBuild(n, v, r) NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: test_simple_yum_setup (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 290, in test_simple_yum_setup arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Test a changed Provides: ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Make sure we accept a consistent repo, and no updates at all ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Make sure depcheck accepts a good update as expected ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: make sure depcheck catches updates with missing requires ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Test with one good and one bad update ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ---------------------------------------------------------------------- Ran 8 tests in 0.002s FAILED (errors=8) }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/219> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel August 2010