August 2010 - autoqa-devel - Fedora mailing-lists

[AutoQA] #191: Submit autotest for Fedora package approval
by fedora-badges 26 Sep '12

26 Sep '12

#191: Submit autotest for Fedora package approval -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Packaging and Review Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- * https://bugzilla.redhat.com/show_bug.cgi?id=548522 Currently stalled, pending packaging of the gwt package. This includes packaging all bundled .jar files included with gwt. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/191> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #157: conflicts: requested arch not respected
by fedora-badges 06 Aug '12

06 Aug '12

#157: conflicts: requested arch not respected --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This is a bug in the potential_conflict.py script in tests/conflicts/. Usage suggests using --arch options for specifying requested architecture to be tested. But if you look at the attached log, the first and second output match (that's correct), but the third doesn't match, even though it should. Everything was tested on x86_64 machine. Also from the usage line it is not clear whether --arch is related only to particular yum repository selection (replacing $basearch keyword), or whether it also relates to packages built for one architecture but being in different repo (e.g. i686 packages are very often in x86_64 repo). I have looked into the source code of the script and it seems that the --arch option is not used at all, just printed out in the usage help?? Seth, you're listed as script author. Any comments on that? Thanks. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/157> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #206: Update bodhi to enforce Package Update Acceptance policy
by fedora-badges 26 Mar '12

26 Mar '12

#206: Update bodhi to enforce Package Update Acceptance policy ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- In order to enforce the Package Update Acceptance Test Plan, Bodhi will need be modified to reject the push of any package that fails acceptance testing. The acceptance tests may need to send status/data to bodhi in order for it to make decisions about policy (see e.g. ticket #205). Later bodhi might just get data from resultdb (see that milestone for details). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/206> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #186: Automate media kit sanity tests
by fedora-badges 05 Mar '12

05 Mar '12

#186: Automate media kit sanity tests --------------------+------------------------------------------------------- Reporter: jlaska | Owner: lili Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This ticket is intended to track automating the mediakit sanity tests (http://fedoraproject.org/wiki/Category:Installer_Image_Sanity_Test_Cases) Discussion already underway, along with code in Liam's private branch. For details see https://fedorahosted.org/pipermail/autoqa- devel/2010-June/000595.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/186> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

[AutoQA] #194: Incorporate automated anaconda storage suite
by fedora-badges 05 Mar '12

05 Mar '12

#194: Incorporate automated anaconda storage suite --------------------+------------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Chris Lumens has developed a virt-based test suite to automate different disk/storage install scenarios. I've discussed this briefly with Chris, but I'd be interested in seeing his test suite incorporated within autoqa and run on a regular interval (perhaps along-side Liam's install tests). The tests are currently available at http://clumens.fedorapeople.org /anaconda-storage-test/ -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/194> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #182: Write virt management script
by fedora-badges 22 Nov '11

22 Nov '11

#182: Write virt management script ----------------------------+----------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- We must write a virt management script that will be able to the basic tasks for us: * query for available virt systems that we can use * switch between "real" disk and a disk snapshot for a virt system * revert a disk snapshot for a virt system * perform some command in a virt system - e.g. "yum update" for keeping it periodically up-to-date * install/reinstall a virt-system for us with specified distro * (some more to come?) We may want to use some pre-defined names syntax to recognize what virt systems do we have. For example: {{{ /dev/vg_autoqa/F12_i686_1 /dev/vg_autoqa/F12_i686_1-snap }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/182> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

2 4

[AutoQA] #196: Log failed job scheduling
by fedora-badges 11 Nov '11

11 Nov '11

#196: Log failed job scheduling -------------------------+-------------------------------------------------- Reporter: kparal | Owner: kparal Type: enhancement | Status: new Priority: major | Milestone: Component: harness | Version: 1.0 Keywords: | -------------------------+-------------------------------------------------- Currently when we schedule a job and it fails, we just print it out and forget about it: {{{ retval = schedule_job(control, email=email, name=testname, required_arch=arch, dryrun=opts.dryrun, labels=aq_labels) if retval != 0: print "ERROR: failed to schedule job %s" % testname }}} That should not be the case. We should: * use syslog and log important stuff to /var/log/autoqa.log or similar * send an email about failed scheduling to autoqa-results That way we will know something is wrong and will be able to fix it. Otherwise it just silently fails and we don't know about it. Feel free to reassign the ticket if you want to work on it. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/196> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #192: Submit autoqa for Fedora package approval
by fedora-badges 13 Oct '11

13 Oct '11

#192: Submit autoqa for Fedora package approval -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Packaging and Review Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- Currently stalled, pending packaging of the gwt and autotest package. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/192> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #180: Initscript review: vsftpd
by fedora-badges 10 Oct '11

10 Oct '11

#180: Initscript review: vsftpd ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/180> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #178: Initscript review: squid
by fedora-badges 10 Oct '11

10 Oct '11

#178: Initscript review: squid ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/178> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #177: Initscript review: setroubleshoot
by fedora-badges 10 Oct '11

10 Oct '11

#177: Initscript review: setroubleshoot ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/177> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #175: Initscript review: postgresql
by fedora-badges 10 Oct '11

10 Oct '11

#175: Initscript review: postgresql ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/175> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #174: Initscript review: openswan
by fedora-badges 10 Oct '11

10 Oct '11

#174: Initscript review: openswan ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/174> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #173: Initscript review: openldap
by fedora-badges 10 Oct '11

10 Oct '11

#173: Initscript review: openldap ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/173> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #171: Initscript review: nss-pam-ldapd
by fedora-badges 10 Oct '11

10 Oct '11

#171: Initscript review: nss-pam-ldapd ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/171> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #169: Initscript review: net-snmp
by fedora-badges 10 Oct '11

10 Oct '11

#169: Initscript review: net-snmp ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/169> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #125: initscripts: Choosing the right repo for dependencies
by fedora-badges 10 Oct '11

10 Oct '11

#125: initscripts: Choosing the right repo for dependencies -------------------------+-------------------------------------------------- Reporter: kparal | Owner: jskladan Type: task | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: initscripts | -------------------------+-------------------------------------------------- Currently after new Koji build the initscripts test downloads all relevant binary packages (related to the particular source package) and installs them. But the RPM dependencies are installed from stable/updates repos, not from Koji. Also the dependencies parsed out of Makefile are installed from stable/updates repos. We have to inquire a little more into this situation. Is it sufficient/desirable? Or is it desirable to download dependencies from updates-testing or bleeding edge Koji? May the initscripts test sometimes fail because of insufficient dependencies are in stable/updates repos? This also depends on cases when we are going to use initscripts test. If it will be used with Bodhi hook, it will be probably easier to solve it than when used with Koji hook. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/125> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #102: post-iso-build hook
by fedora-badges 03 Aug '11

03 Aug '11

#102: post-iso-build hook ----------------------+----------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: watchers | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- We need one (or more) standard locations where new iso images get posted for testing, and a watcher that can monitor for new iso images and launch appropriate installation/sanity tests. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/102> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #209: complete auto CD install test driver
by fedora-badges 02 Aug '11

02 Aug '11

#209: complete auto CD install test driver -------------------+-------------------------------------------------------- Reporter: liam | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | -------------------+-------------------------------------------------------- complete the cd_install.py as dvd_install.py to support CD install, with some functions like automatically swap CD when CD1 install was completed. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/209> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #149: rpmlint: Provide whitelisting of messages
by fedora-badges 28 Jul '11

28 Jul '11

#149: rpmlint: Provide whitelisting of messages --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Package update tests Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- We need to provide some whitelisting capability of rpmlint packages, so the package maintainers can mark some messages as invalid. We will also probably want to mark some messages as invalid by default for the whole Fedora project. As an interesting inspiration, look at lintian project (validation of deb packages). It provides some "overrides" on a package basis, you just create a definition file in /etc/lintian/package_name or kind of. I have the feeling that rpmlint provides something similar, but we need to discover it more properly. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/149> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #104: virtguest.py: accept iso image(s) as install location
by fedora-badges 21 Jul '11

21 Jul '11

#104: virtguest.py: accept iso image(s) as install location --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- allow {{{VirtGuest.create()}}} to use an iso image (or images) for its {{{location}}} arg, and automatically choose the appropriate {{{--location URL}}} or {{{--cdrom IMAGE}}} flag. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/104> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #183: Write a script and a daemon to send signals between virt machine and host
by fedora-badges 11 Jul '11

11 Jul '11

#183: Write a script and a daemon to send signals between virt machine and host ----------------------------+----------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- We need to exchange some signals between virt machine and a host. For example "autotest job finished, need to revert to previous state". We need to write simple script that will send this signal for virt machine and a daemon that will receive this signal in the host machine and act accordingly. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/183> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #208: update minimon to transport logs between guest and host via virtio
by fedora-badges 20 May '11

20 May '11

#208: update minimon to transport logs between guest and host via virtio -------------------+-------------------------------------------------------- Reporter: liam | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | -------------------+-------------------------------------------------------- Since minimon has to use network during test, some test cases will not activate network during test,in this case, minimon can not transport logs to host, test also be identified fail even it was successfully run. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/208> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #214: conflicts: fails with parent repositories
by fedora-badges 22 Apr '11

22 Apr '11

#214: conflicts: fails with parent repositories --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- These two commands work (although the package counts are somewhat weirdly same): {{{ # repoclosure --tempcache --newest --repofrompath=target,http://download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os --repoid=target Added target repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os Reading in repository metadata - please wait.... Checking Dependencies /usr/lib/python2.6/site-packages/yum/packages.py:401: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal if prcotuple in self.prco[prcotype]: Repos looked at: 1 target Num Packages in Repos: 20806 }}} {{{ # repoclosure --tempcache --newest --repofrompath=target,http://download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 --repoid=target Added target repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 Reading in repository metadata - please wait.... Checking Dependencies /usr/lib/python2.6/site-packages/yum/packages.py:401: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal if prcotuple in self.prco[prcotype]: Repos looked at: 1 target Num Packages in Repos: 20806 }}} But if you combine it: {{{ # repoclosure --tempcache --newest --repofrompath=target,http://download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 --repoid=target --repofrompath=parent1,http://download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os --repoid=parent1 Added target repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/updates/13/x86_64 Added parent1 repo from /root/http:/download.fedoraproject.org/pub/fedora/linux/releases/13/Everything/x86_64/os Reading in repository metadata - please wait.... Cannot retrieve repository metadata (repomd.xml) for repository: parent1. Please verify its path and try again Some dependencies may not be complete for this repository Run as root to get all dependencies or use -t to enable a user temp cache Checking Dependencies Cannot retrieve repository metadata (repomd.xml) for repository: parent1. Please verify its path and try again }}} Something is broken here. {{{ yum-utils-1.1.28-1.fc13.noarch yum-metadata-parser-1.1.4-1.fc13.x86_64 yum-3.2.27-4.fc13.noarch }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/214> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #218: re-use fedora-qa script for checking PackageReviewGuidelines
by fedora-badges 07 Apr '11

07 Apr '11

#218: re-use fedora-qa script for checking PackageReviewGuidelines --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Future test cases Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Aurélien Bompard has written a "fedora-qa" tool, which checks all the MUST items in !PackageReviewGuidelines that are not covered by rpmlint. http://fedoraproject.org/wiki/AurelienBompard#QA There's a high possibility that we could re-use parts of his code for several new AutoQA test cases. Worth investigating. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/218> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #213: remove collection_name from repoinfo.conf
by fedora-badges 29 Mar '11

29 Mar '11

#213: remove collection_name from repoinfo.conf ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- Once ticket #212 is complete we have no real need for collection_name anymore; remove it from repoinfo.conf. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/213> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #212: fix check_opt_in to not use collection_name
by fedora-badges 29 Mar '11

29 Mar '11

#212: fix check_opt_in to not use collection_name -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- collection_name was used to line up with the dist-cvs branch layout; now that we have switched to dist-git/fedpkg the branch names match the repo names used in repoinfo, so we have no need for collection_name. This will require modifying the existing autoqa-optin file structure to match the git layout. We can leave symlinks for the old ('F-14') distro names but use the new ones ('f14') as the canonical ones. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/212> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #204: add depcheck to post-bodhi-update hook
by fedora-badges 25 Jan '11

25 Jan '11

#204: add depcheck to post-bodhi-update hook --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Actually get the depcheck test running on post-bodhi-update. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/204> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #205: make depcheck submit karma/comments to bodhi
by fedora-badges 10 Jan '11

10 Jan '11

#205: make depcheck submit karma/comments to bodhi --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Modify the depcheck test so that it sends status info - karma, comments, whatever - to bodhi. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/205> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 10

[AutoQA] #201: Add mash support to depcheck
by fedora-badges 29 Nov '10

29 Nov '10

#201: Add mash support to depcheck -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- To properly handle possible multilib problems (e.g. the [https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000703.html nss-softokn problem] in June and the recent gfortran problem) we need to run mash on the set of proposed packages, to generate a proper multilibified repo to test against. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/201> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #153: rpmguard: Separate checks into individual python files
by fedora-badges 19 Nov '10

19 Nov '10

#153: rpmguard: Separate checks into individual python files -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: rpmguard | -------------------------+-------------------------------------------------- It would be great to have individual checks in rpmguard available as separate files, very similarly to what rpmlint or yum (-plugins) use. It would allow easy enabling/disabling of specific check, it would allow easy contribution of a new check. More ideas and information here: https://fedorahosted.org/pipermail /autoqa-devel/2010-May/000496.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/153> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #150: "artificial ignorance": suppress errors and warnings that aren't important
by fedora-badges 16 Nov '10

16 Nov '10

#150: "artificial ignorance": suppress errors and warnings that aren't important -------------------------+-------------------------------------------------- Reporter: dmalcolm | Owner: Type: enhancement | Status: new Priority: major | Milestone: Package Sanity Tests Component: tests | Version: 1.0 Keywords: | -------------------------+-------------------------------------------------- There are some interesting ideas in here about suppressing a logfile to show up the information that's truly important: http://www.ranum.com/security/computer_security/papers/ai/index.html I think that the rpmlint test needs to have a set of filters, and should suppress output matching certain patterns. For example https://fedorahosted.org/pipermail/autoqa-results/2010-April/013903.html shows: {{{ gnome-python2-gnome.i686: W: spelling-error Summary(en_US) libgnome -> lib gnome, lib-gnome, cognomen }}} i.e. it's complaining about a reference to "libgnome" within gnome- python2-gnome, which is clearly a false-positive. For this case we might have a python filtering process that says something like this (caveat: untested): {{{ patterns = [ ": W: spelling-error \S+ libgnome .*", ] def should_ignore(line): for pat in patterns # May want to precompile the patterns: m = re.match(pat, line) if m: return True # No match: return False for line in rpmlint_output if not should_ignore(line): print(line) }}} and then gradually grow patterns. The idea is to make it nice and lightweight, and Fedora QA can then easily take ownership of the suppresssions, without having to get patches into rpmlint. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/150> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #179: Initscript review: tigervnc
by fedora-badges 13 Oct '10

13 Oct '10

#179: Initscript review: tigervnc ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/179> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

[AutoQA] #167: Initscript review: mcstrans
by fedora-badges 13 Oct '10

13 Oct '10

#167: Initscript review: mcstrans ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/167> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #138: ResultsDB: media wiki as a storage for metadata about tests
by fedora-badges 13 Oct '10

13 Oct '10

#138: ResultsDB: media wiki as a storage for metadata about tests ----------------------------+----------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- Once #135 #136 and #137 are finished, we should create a provider/middle man (probably a library) which will allow us to get information about the respective test and use it for test execution/storing results/displaying results via frontends... -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/138> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #137: ResultsDB: propose structure for storing metadata about tests on wiki
by fedora-badges 13 Oct '10

13 Oct '10

#137: ResultsDB: propose structure for storing metadata about tests on wiki -----------------------+---------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- We want to use mediawiki as a storage for tests metadata. We should agree on the fields we want to store (e.g. test owner, destructive/non- destructive, average time to complete test etc.) and the format we'll use to store it (probably JSON though). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/137> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #170: Initscript review: net-tools
by fedora-badges 11 Oct '10

11 Oct '10

#170: Initscript review: net-tools ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/170> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #166: Initscript review: iputils
by fedora-badges 11 Oct '10

11 Oct '10

#166: Initscript review: iputils ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: minor | Milestone: Initscripts review Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Check the test according to the <https://fedorahosted.org/autoqa/wiki/initscripts> -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/166> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #134: ResultsDB: create resultsdb prototype
by fedora-badges 06 Oct '10

06 Oct '10

#134: ResultsDB: create resultsdb prototype ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: Type: task | Status: new Priority: major | Milestone: Resultdb Component: tests | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- Create prototype of database and software which will implement dbSchema from #133 and API from #131 This prototype should also include some test suite, which will be used to determine, if all the functionality is OK. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/134> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #202: testcases for depcheck
by fedora-badges 24 Sep '10

24 Sep '10

#202: testcases for depcheck --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- To be sure that depcheck is operating as expected, we should have a library of test cases for it. These test cases could take the form of a set (or sets) of trivial RPMs that illustrate various possible dependency problems - or a script that generates RPMs like that. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/202> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #221: Convert package repository to repos.fedorapeople.org/autoqa
by fedora-badges 16 Sep '10

16 Sep '10

#221: Convert package repository to repos.fedorapeople.org/autoqa -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: minor | Milestone: Packaging, Review, & Deployment Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- It would make me feel more comfortable having the repositories less tied to my name. It seems to transient that way, and for some reason, feels one step closer to permanent if the packages are housed at http://repos.fedorapeople.org -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/221> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #215: Rewrite the "how to write autoqa tests" wiki page
by fedora-badges 08 Sep '10

08 Sep '10

#215: Rewrite the "how to write autoqa tests" wiki page -----------------------+---------------------------------------------------- Reporter: jskladan | Owner: jskladan Type: task | Status: new Priority: major | Milestone: Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- Rewrite the "how to write autoqa tests" wiki page so it reflects the current features and concepts. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/215> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #203: make rpmguard run as a post-bodhi-update test too
by fedora-badges 06 Sep '10

06 Sep '10

#203: make rpmguard run as a post-bodhi-update test too --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Multi-hook tests Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Using the multihook capability described elsewhere, make rpmguard run for the post-bodhi-update hook. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/203> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

[PATCH] Polished upgradepath code - minor fixes.
by Josef Skladanka 31 Aug '10

31 Aug '10

--- tests/upgradepath/upgradepath.py | 44 ++++++++++++------------------------- 1 files changed, 14 insertions(+), 30 deletions(-) diff --git a/tests/upgradepath/upgradepath.py b/tests/upgradepath/upgradepath.py index 257eddc..a49fa07 100755 --- a/tests/upgradepath/upgradepath.py +++ b/tests/upgradepath/upgradepath.py @@ -36,17 +36,11 @@ from autotest_lib.client.common_lib import error class upgradepath(AutoQATest): version = 1 # increment if setup() changes - test_result = 'PASS' - log = [] - envr_list = set() - def initialize(self, config): - self.config = config_loader(config, self.tmpdir) - #URL of logs/results stored on autotest-server - self.autotest_url = autoqa.util.make_autotest_url(self.config) - - def setup(self): - pass + def __init__(self, *args, **kwargs): + super(upgradepath, self).__init__(*args, **kwargs) + self.log = [] + self.envr_list = set() def compare(self, matching_build, tags, op): koji = autoqa.koji_utils.SimpleKojiClientSession() @@ -68,7 +62,7 @@ class upgradepath(AutoQATest): print msg self.log.append(msg) else: - self.test_result = 'FAIL' + self.result = 'FAILED' msg ='{0:<7}{1}'.format('[FAIL]', tag) print msg self.log.append(msg) @@ -85,11 +79,11 @@ class upgradepath(AutoQATest): try: current_tag = repotags.index(kojitag) except ValueError: - self.test_result = 'FAIL' - msg = "ERROR: Entered bad kojitag" - print msg - self.log.append(msg) - self.envr_list.add(matching_build['envr']) + self.result = 'FAILED' + self.summary = "ERROR: Entered bad kojitag" + print self.summary + self.log.append(self.summary) + self.outputs = "\n".join(self.log) raise error.TestFail hi_tags = repotags[(current_tag + 1):] # tags higher than current @@ -136,20 +130,10 @@ class upgradepath(AutoQATest): self.log.append(msg) print msg - self.result = self.test_result - self.outputs = "" - for i in self.log: - self.outputs += i + '\n' - - self.outputs += '\n \n' - - packages_fail = "" - for i in self.envr_list: - packages_fail += i + ', ' - packages_fail = packages_fail[:-2] - - self.summary = packages_fail + self.outputs = "\n".join(self.log) + self.summary = ", ".join(self.envr_list) - if self.test_result == 'FAIL': + if self.result == 'FAILED': raise error.TestFail + self.result = "PASSED" -- 1.6.6.1

2 1

[AutoQA] #123: rpmguard: check upgrade paths between releases
by fedora-badges 31 Aug '10

31 Aug '10

#123: rpmguard: check upgrade paths between releases ----------------------+----------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: rpmguard | ----------------------+----------------------------------------------------- (As pointed out by spot on the devel list: http://lists.fedoraproject.org/pipermail/devel/2010-March/131746.html) We should be emitting warning/error messages if the new package(s) have a higher ENVR than the currently-released package(s) in any newer release. For example, right now a new F11 package should be checked against the corresponding package(s) in the current F12, branched F13, and rawhide repos. This involves a lot of extra repos and metadata, though. It's not a trivial thing to add. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/123> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 14

[AutoQA] #220: Check domains of running services after installation
by fedora-badges 30 Aug '10

30 Aug '10

#220: Check domains of running services after installation ---------------------+------------------------------------------------------ Reporter: adelton | Owner: Type: task | Status: new Priority: minor | Milestone: Component: tests | Version: 1.0 Keywords: | ---------------------+------------------------------------------------------ Recently I've came across issue https://bugzilla.redhat.com/show_bug.cgi?id=628495 when daemons run as initrc_t. In this case it was in spite of the fact that there was a SELinux policy module loaded, simply because the file context was pointing to a file that no longer existed in that rpm and which was no longer run during service startup. I believe that with older Fedoras and with older versions of that package, the file context was actually correct and the daemons run confined. So avoid any such regressions, it would be nice when doing some post- installation tests and starting services to also check the SELinux domain of the processes that are started by those services (ps Z). Thanks, Jan -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/220> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #111: depcheck test
by fedora-badges 27 Aug '10

27 Aug '10

#111: depcheck test --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: autoqa depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Write an actual depcheck test. This test should take, as inputs: * one or more new package builds, and * the name of a target repository for the new build(s) The test should examine the PRCO (provides/requires/conflicts/obsoletes) data for the new package(s) and the PRCO data for the target repository (and all its parent repos). The test should fail if the new builds would cause missing/broken dependencies or unresolveable conflicts in the target repo(s). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/111> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 8

Writing AutoQA Tests wiki page
by Josef Skladanka 27 Aug '10

27 Aug '10

Hello gang, I have a first spin of the new wiki page https://fedoraproject.org/wiki/User:Jskladan/Sandbox:Writing_autoqa_tests and I'd be glad if you could skim through the page, and post some feedback. Thanks Joza

2 1

Re: depcheck - next steps!
by Kamil Paral 26 Aug '10

26 Aug '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > > == write control/control.autoqa/test wrapper object == > > I took a stab at this in the depcheck branch but I'm pretty sure I > did > it wrong / left some stuff out. Kamil/Joza, if you guys can give me > some > help/pointers here I'd appreciate it... but don't worry, I'll send > the > patch to the list for a final review before I merge anything. > I'm a little confused by all those files in tests/depcheck. There is * depcheck * depcheck.py * depcheck_test.py By our conventions the AutoQA test object should be called "depcheck.py". Therefore we need to rename the actual script to something else, like "run_depcheck.py" or similar. It's not clear to me what the purpose of the current 'depcheck' file is, I think its functionality can be easily integrated into the main depcheck script itself (the one with proposed name "run_depcheck.py"), can't it?

2 1

[AutoQA] #219: depcheck: NameError: global name 'rpmfluff' is not defined
by fedora-badges 24 Aug '10

24 Aug '10

#219: depcheck: NameError: global name 'rpmfluff' is not defined --------------------+------------------------------------------------------- Reporter: kparal | Owner: wwoods Type: defect | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - depcheck Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- {{{ # ~autotest/client/site_tests/depcheck/depcheck --selftest /usr/lib/python2.6/site-packages/rpmfluff.py:58: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. import popen2 EEEEEEEE ====================================================================== ERROR: test_createrepo (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 277, in test_createrepo rpmlist = simple_rpmlist() File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 249, in simple_rpmlist m = rpmfluff.SimpleRpmBuild('mint', '1.0', '1') NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: test_rpmfluff (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 260, in test_rpmfluff p = rpmfluff.SimpleRpmBuild(n, v, r) NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: test_simple_yum_setup (depcheck.DepcheckPrereqTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 290, in test_simple_yum_setup arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Test a changed Provides: ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Make sure we accept a consistent repo, and no updates at all ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Make sure depcheck accepts a good update as expected ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: make sure depcheck catches updates with missing requires ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ====================================================================== ERROR: Test with one good and one bad update ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/share/autotest/client/site_tests/depcheck/depcheck.py", line 317, in setUp self.arch = rpmfluff.expectedArch NameError: global name 'rpmfluff' is not defined ---------------------------------------------------------------------- Ran 8 tests in 0.002s FAILED (errors=8) }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/219> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

Re: [PATCH] add upgradepath test
by Kamil Paral 24 Aug '10

24 Aug '10

----- "James Laska" <jlaska(a)redhat.com> wrote: > > See attached patch to convey the stand-alone idea noted above. Ok, I see your point now. Yes, we can do it like that. We will create "stand-alone" script check_upgradepath.py, that will depend on autoqa (therefore be able to use autoqa library and repoinfo.conf), but can be run alone like: $ ./check_upgradepath.py --envrs blobby-0.9b-1.fc13 --kojitag dist-f13-updates I'm not sure how large the benefit is over calling it through the hook, but surely the CLI interface is a little simpler than: $ autoqa post-bodhi-update --title blobby-0.9b-1.fc13 --target-tag dist-f13-updates blobby-0.9b-1.fc13 --local -t upgradepath Both should achieve the same functionality.

4 5

[AutoQA] #217: autoqa job failures
by fedora-badges 24 Aug '10

24 Aug '10

#217: autoqa job failures --------------------+------------------------------------------------------- Reporter: jlaska | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- After Kamil mentioned this earlier in the week, I noticed not many post- koji-build jobs were processing. Inspecting the logs, it appears they are failing .... For example, the following traceback comes from http://test1250.test.redhat.com/results/44857-autotest/brutus.test.redhat.c… {{{ 08/19 13:01:22 INFO | autotest:0913| FAIL rpmlint rpmlint timestamp=1282249190 localtime=Aug 19 16:19:50 Unhandled str: Unhandled TypeError: 'NoneType' object is unsubscriptable 08/19 13:01:22 INFO | autotest:0913| Traceback (most recent call last): 08/19 13:01:22 INFO | autotest:0913| File "/usr/share/autotest/common_lib/test.py", line 570, in _call_test_function 08/19 13:01:22 INFO | autotest:0913| return func(*args, **dargs) 08/19 13:01:22 INFO | autotest:0913| File "/usr/share/autotest/common_lib/test.py", line 279, in execute 08/19 13:01:22 INFO | autotest:0913| postprocess_profiled_run, args, dargs) 08/19 13:01:22 INFO | autotest:0913| File "/usr/share/autotest/common_lib/test.py", line 201, in _call_run_once 08/19 13:01:22 INFO | autotest:0913| self.run_once(*args, **dargs) 08/19 13:01:22 INFO | autotest:0913| File "/usr/share/autotest/site_tests/rpmlint/rpmlint.py", line 70, in run_once 08/19 13:01:22 INFO | autotest:0913| collection_name = repoinfo.getrepo_by_tag(kojitag)['collection_name'] 08/19 13:01:22 INFO | autotest:0913| TypeError: 'NoneType' object is unsubscriptable 08/19 13:01:22 INFO | autotest:0913| 08/19 13:01:22 INFO | autotest:0913| None 08/19 13:01:22 INFO | autotest:0913| END FAIL rpmlint rpmlint timestamp=1282249190 localtime=Aug 19 16:19:50 }}} This AutoQA instance is running the older 0.3 branch. I gather the tests have changed significantly since. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/217> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

Re: [PATCH] check for previous releases even in the requested repo, not just in parents
by Kamil Paral 24 Aug '10

24 Aug '10

----- "James Laska" <jlaska(a)redhat.com> wrote: > On Fri, 2010-08-20 at 17:42 +0200, Kamil Páral wrote: > > --- > > lib/python/koji_utils.py | 51 > ++++++++++++++++++++------------------------- > > 1 files changed, 23 insertions(+), 28 deletions(-) > > > > diff --git a/lib/python/koji_utils.py b/lib/python/koji_utils.py > > index f8ac077..87b32e9 100644 > > --- a/lib/python/koji_utils.py > > +++ b/lib/python/koji_utils.py > > <snip> > > Thanks Kamil, > > As we discussed in the Fedora QA meeting today. I have applied this > patch to the running AutoQA instance and rescheduled several > previously > failed rpmguard jobs. I consider the results a PASS, but welcome any > additional tests you'd like me to run. > > First, a package that does *not* have previous builds ... > gtk-nodoka-engine-0.7.5-1.fc15 > * Without the patch, the job (#45485) failed (see > http://fpaste.org/cOJh/) > * With the patch, the job (#45506) runs to completion, but is > not > able to find a previous build to compare against in dist-f15. > This is accurate, looking in 'koji' I cannot find another > dist-f15 build to compare against > > (http://koji.fedoraproject.org/koji/packageinfo?packageID=4871) > > N: There is no previous build of > gtk-nodoka-engine-0.7.5-1.fc15 in parents of dist-f15 > tag. > > Next, a package that does have previous builds ... > schroot-1.4.2-5.fc14 > * Without patch - rpmguard result - > > https://fedorahosted.org/pipermail/autoqa-results/2010-July/028449.html > * With patch - rpmguard result - > > https://fedorahosted.org/pipermail/autoqa-results/2010-August/030026.html > > The results match, and the confirm what I'd expect to see based on > koji > builds > (http://koji.fedoraproject.org/koji/packageinfo?packageID=8929) Yes, I also believe that the behaviour is correct now. Thanks for testing it, James, and thanks for pushing it, Will. Closing ticket #217.

1 0

[PATCH] check for previous releases even in the requested repo, not just in parents
by Kamil Paral 23 Aug '10

23 Aug '10

--- lib/python/koji_utils.py | 51 ++++++++++++++++++++------------------------- 1 files changed, 23 insertions(+), 28 deletions(-) diff --git a/lib/python/koji_utils.py b/lib/python/koji_utils.py index f8ac077..87b32e9 100644 --- a/lib/python/koji_utils.py +++ b/lib/python/koji_utils.py @@ -23,6 +23,7 @@ import time import koji from repoinfo import repoinfo import rpmUtils.miscutils +import sys # XXX fetch from /etc/koji.conf, section 'koji' kojiurl = 'http://koji.fedoraproject.org/kojihub' @@ -101,40 +102,34 @@ class SimpleKojiClientSession(koji.ClientSession): '''Get most recent releases of package in specified tag (inheritence enabled). Returns dictionary in the format {tag: NVR object}. Optional max_evr says that all returned release versions must be lesser then - it. If you enable unstable_tags, then also 'testing' and 'candidate' - tags are considered. + it. If you enable unstable_tags, then also '-testing' tags are considered. + max_evr = (epoch, version, release) ''' prev_rlss = {} # minor hack - assume -candidate parents = stable+updates+testing # TODO: provide this info in repoinfo instead + # since there's no repo for candidate builds, just look at -testing and below if tag.endswith('-updates-candidate'): - testtag = tag.replace('-candidate','-testing') - testrepo = repoinfo.getrepo_by_tag(testtag) - # -testing parents = stable+updates. add -testing and we're good. - parents = testrepo['parents'] + [testrepo['name']] - else: - parents = repoinfo.getrepo_by_tag(tag)['parents'] - if parents: - for p in parents: - ptag = repoinfo.get(p,'tag') - if ('-testing' in ptag or '-candidate' in ptag) and \ - not unstable_tags: - # skip unstable tags - continue - prev_rlss[ptag] = self.latest_by_tag(ptag, name, max_evr=max_evr) - else: - # NOTE: for tags with no parents (e.g. rawhide) we obviously can't - # check the parent repos' tags to get the previously released - # package NVRs. Instead we use koji's tagHistory command to find - # the previous version of this package that had this tag. - hist = self.tagHistory(tag=tag, package=name) - # remove untagged builds - hist = filter(lambda h: h['revoke_event'] is None, hist) - # ensure sorting by time - hist.sort(key=lambda h: h['create_ts']) - # hist[-2] = not the currently-tagged build, the one before that - prev_rlss[tag] = self.getBuild(hist[-2]['build_id']) + tag = tag.replace('-candidate','-testing') + + testrepo = repoinfo.getrepo_by_tag(tag) + if testrepo is None: + print >> sys.stderr, "Error: There's no repo definition for tag '%s'" % tag + return {} + parents = testrepo['parents'] + repos = parents + [testrepo['name']] + + if not unstable_tags: + for repo in repos[:]: + rtag = repoinfo.get(repo, 'tag') + if ('-testing' in rtag or '-candidate' in rtag): + # remove repo with unstable tag + repos.remove(repo) + + for repo in repos: + rtag = repoinfo.get(repo, 'tag') + prev_rlss[rtag] = self.latest_by_tag(rtag, name, max_evr=max_evr) return prev_rlss def list_previous_release(self, name, tag, max_evr=None, -- 1.7.2.1

2 1

[PATCH] don't crash when repoinfo.getrepo_by_tag returns None
by Kamil Paral 20 Aug '10

20 Aug '10

--- lib/python/util.py | 3 +++ tests/initscripts/initscripts.py | 6 +++--- tests/rpmguard/rpmguard.py | 5 +++-- tests/rpmlint/rpmlint.py | 4 ++-- 4 files changed, 11 insertions(+), 7 deletions(-) diff --git a/lib/python/util.py b/lib/python/util.py index ed90ba4..b6d0013 100644 --- a/lib/python/util.py +++ b/lib/python/util.py @@ -96,6 +96,9 @@ def checkhash(filename, algo, hexdigest): def check_opt_in(pkgname, dist_release): """take a package name and distribution release and check for an autoqa-optin file""" + if not pkgname or not dist_release: + return False + url = 'http://fedorapeople.org/packages/' fullurl = url + '%s/%s/%s/autoqa-optin' % (pkgname[0].lower(), pkgname, dist_release) tmout = socket.getdefaulttimeout() diff --git a/tests/initscripts/initscripts.py b/tests/initscripts/initscripts.py index 0806107..1885003 100644 --- a/tests/initscripts/initscripts.py +++ b/tests/initscripts/initscripts.py @@ -152,9 +152,9 @@ class initscripts(AutoQATest): self.highlights = "Following tests returned zero exit code: %s\n" % repr(tests_exit_status_zero) self.highlights += "Following tests returned NON-zero exit code: %s\n" % repr(tests_exit_status_nonzero) - #add subscribed package owners to mail-to - collection_name = repoinfo.getrepo_by_tag(kojitag)['collection_name'] - if collection_name and autoqa.util.check_opt_in(name, collection_name): + # email results to mailing list and to pkg owner if they optin + repo = repoinfo.getrepo_by_tag(kojitag) + if repo is not None and autoqa.util.check_opt_in(name, repo['collection_name']): #FIXME - hardcoded partial email address here - obviously sub-par self.mail_to.append('%s-owner(a)fedoraproject.org' % name) diff --git a/tests/rpmguard/rpmguard.py b/tests/rpmguard/rpmguard.py index a0e7e74..b317089 100644 --- a/tests/rpmguard/rpmguard.py +++ b/tests/rpmguard/rpmguard.py @@ -188,8 +188,9 @@ rpmlint | cut -d ' ' -f 1", retain_output=True) self.summary = '%u warnings for package %s' % (warning_count, envr) self.outputs = log - collection_name = repoinfo.getrepo_by_tag(kojitag)['collection_name'] - if collection_name and autoqa.util.check_opt_in(name, collection_name): + # email results to mailing list and to pkg owner if they optin + repo = repoinfo.getrepo_by_tag(kojitag) + if repo is not None and autoqa.util.check_opt_in(name, repo['collection_name']): #FIXME - hardcoded partial email address here - obviously sub-par self.mail_to.append('%s-owner(a)fedoraproject.org' % name) diff --git a/tests/rpmlint/rpmlint.py b/tests/rpmlint/rpmlint.py index adcfb13..1189298 100644 --- a/tests/rpmlint/rpmlint.py +++ b/tests/rpmlint/rpmlint.py @@ -69,8 +69,8 @@ class rpmlint(AutoQATest): self.summary = '%s for package %s' % (summary, envr) # email results to mailing list and to pkg owner if they optin - collection_name = repoinfo.getrepo_by_tag(kojitag)['collection_name'] - if collection_name and autoqa.util.check_opt_in(name, collection_name): + repo = repoinfo.getrepo_by_tag(kojitag) + if repo is not None and autoqa.util.check_opt_in(name, repo['collection_name']): #FIXME - hardcoded partial email address here - obviously sub-par self.mail_to.append('%s-owner(a)fedoraproject.org' % name) -- 1.7.2.1

2 1

Re: [PATCH] add upgradepath test
by Kamil Paral 20 Aug '10

20 Aug '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > On Thu, 2010-08-19 at 14:58 -0400, James Laska wrote: > > I was interested in testing out just the upgrade-path portion of the > > test (not the autotest/autoqa integration). How would you feel > about > > moving the test to a stand-alone script, and then changing > > upgradepath.py to call the stand-alone script and process output? No, don't say this, Vojta will try to kill me! That is exactly how he started to implement that and I talked him out of it :) > > > > I think this would make it a lot easier for folks without a full > autoqa > > and autotest-server setup to run the test at home. > > I agree with James here - it's best if we can run the tests outside of > AutoQA, both for debugging purposes and so we can extend/reuse test > code > elsewhere. > There are a few reasons why I recommended Vojta to write it as a single AutoQA script: 1. This task is very short and simple, we can easily join the the test logic and AutoQA test object stuff. 2. This task requires access to almost whole repoinfo.conf. Using it as a standalone script would require keeping a copy of repoinfo.conf next to it or passing all those information with command line options (all those repo URLs and stuff, that wouldn't be pretty). 3. This task requires a lot of functionality that we already implemented in our autoqa python library, like querying koji for latest package with some tag, comparing EVR or easily working with repoinfo. Using a standalone script would still require autoqa library installed (and therefore whole autoqa effectively) or Vojta would have to re-implement all that stuff from scratch. Of course it is possible to have this as a standalone script, but it seemed extremely cumbersome to me and I discouraged Vojta from doing that. The code would be twice as long, handling many options passing and re-implementing some autoqa code. But maybe I'm just looking at it all wrong. Tell me what you think.

2 2

[PATCH] add upgradepath test
by Vojtěch Aschenbrenner 20 Aug '10

20 Aug '10

Hello, I created new test for AutoQA named 'upgradepath'. It will check if package which you want to push into repository has correct version. Also there will be warning if you're pushing into stable repo or pushing older or the same version of package. Requests for comments :-). --- diff --git a/tests/upgradepath/control b/tests/upgradepath/control new file mode 100644 index 0000000..997c81e --- /dev/null +++ b/tests/upgradepath/control @@ -0,0 +1,14 @@ +# vim: set syntax=python +# Notice: Most recent documentation is available at doc/control.template. +TIME="SHORT" +AUTHOR = "Vojtech Aschenbrenner <vaschenb(a)redhat.com>" +DOC = """ +This test runs upgradepath to catch problems with package versions +in repositories. +""" +NAME = 'upgradepath' +TEST_TYPE = 'CLIENT' # SERVER can be used for tests that need multiple machines +TEST_CLASS = 'General' +TEST_CATEGORY = 'Functional' + +job.run_test('upgradepath', config=autoqa_conf, **autoqa_args) diff --git a/tests/upgradepath/control.autoqa b/tests/upgradepath/control.autoqa new file mode 100644 index 0000000..6b2ea92 --- /dev/null +++ b/tests/upgradepath/control.autoqa @@ -0,0 +1,9 @@ +# vim: set syntax=python +# Notice: Most recent documentation is available at doc/control.autoqa.template. + +# upgradepath can be run just once and on any architecture +archs = ['noarch'] + +# do not run automaticaly +execute = False + diff --git a/tests/upgradepath/upgradepath.py b/tests/upgradepath/upgradepath.py new file mode 100755 index 0000000..8c4547e --- /dev/null +++ b/tests/upgradepath/upgradepath.py @@ -0,0 +1,147 @@ +#!/usr/bin/python +# +# Copyright 2010, Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +# +# Author: Vojtech Aschenbrenner <vaschenb(a)redhat.com> + +import os +import sys +import operator +import autoqa.koji_utils +import autoqa.util +import rpmUtils.miscutils + +from autoqa.test import AutoQATest +from autoqa.decorators import ExceptionCatcher +from autotest_lib.client.bin.test_config import config_loader +from autotest_lib.client.common_lib import error + + +class upgradepath(AutoQATest): + version = 1 # increment if setup() changes + test_result = 'PASS' + log = [] + envr_list = set() + + def initialize(self, config): + self.config = config_loader(config, self.tmpdir) + #URL of logs/results stored on autotest-server + self.autotest_url = autoqa.util.make_autotest_url(self.config) + + def setup(self): + pass + + def compare(self, matching_build, tags, op): + koji = autoqa.koji_utils.SimpleKojiClientSession() + for tag in tags: + latest_build = koji.latest_by_tag(tag, matching_build['name']) + if latest_build is None: + msg = tag + '... OK' + self.log.append(msg) + print msg + + msg = "-> Package %s in %s doesn't exist" % (matching_build['name'], tag) + self.log.append(msg) + print msg + else: + result = autoqa.koji_utils.compareEVR(matching_build, latest_build) + + if op(result, 0): + msg = tag + '... OK' + print msg + self.log.append(msg) + else: + self.test_result = 'FAIL' + msg = tag + '... FAIL' + print msg + self.log.append(msg) + self.envr_list.add(matching_build['envr']) + msg = "-> Pushing package: %s\n-> Latest package: %s" % (matching_build['envr'], latest_build['nvr']) + self.log.append(msg) + print msg + + @ExceptionCatcher("self.run_once_failed") + def run_once(self, envrs, name, kojitag, **kwargs): + reponames = [reponame for reponame in autoqa.koji_utils.repoinfo.repos() if not reponame.endswith('-testing')] + repotags = [autoqa.koji_utils.repoinfo.getrepo(reponame)['tag'] for reponame in reponames] + repotags.sort() + try: + current_tag = repotags.index(kojitag) + except ValueError: + self.test_result = 'FAIL' + msg = "ERROR: Entered bad kojitag" + print msg + self.log.append(msg) + self.envr_list.add(matching_build['envr']) + raise error.TestFail + + hi_tags = repotags[(current_tag + 1):] # tags higher than current + low_eq_tags = repotags[:(current_tag + 1)] # tags lower or equal + + for envr in envrs: + msg = 40*'=' + '\n' + envr + '\n' + 40*'=' + self.log.append(msg) + print 40*'=' + print envr + print 40*'=' + + # our testing package + (name, version, release, epoch, arch) = rpmUtils.miscutils.splitFilename(envr) + matching_build = { + 'name': name, + 'version' : version, + 'release' : release + '.' + arch, + 'epoch' : epoch, + 'envr' : envr, + } + + if kojitag.find('updates') == -1 and repotags[current_tag] != repotags[-1]: + msg = "Warning: Pushing into stable repository" + self.log.append(msg) + print msg + + koji = autoqa.koji_utils.SimpleKojiClientSession() + latest_build = koji.latest_by_tag(kojitag, matching_build['name']) + if latest_build is not None: + result = autoqa.koji_utils.compareEVR(matching_build, latest_build) + if result != 1: + msg = "Warning: Pushing older or current version of package" + self.log.append(msg) + print msg + + # compare with lower or equal tags, so version has to be greater or equal + self.compare(matching_build, low_eq_tags, operator.ge) + # compare with higher tags, so version has to be lower or equal + self.compare(matching_build, hi_tags, operator.le) + + # mailing stuff + self.result = self.test_result + + self.outputs = "" + for i in self.log: + self.outputs += i + '\n' + self.outputs += '\n \n' + + packages_fail = "" + for i in self.envr_list: + packages_fail += i + ', ' + packages_fail = packages_fail[:-2] + self.summary = packages_fail + + if self.test_result == 'FAIL': + raise error.TestFail +

4 3

[AutoQA] #199: Standardize (some) AutoQA test args
by fedora-badges 19 Aug '10

19 Aug '10

#199: Standardize (some) AutoQA test args -----------------------+---------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Multi-hook tests Component: docs/wiki | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- The various hooks that AutoQA provides each have different test args. Some of these arguments are similar or identical across hooks - for example, 'config' applies to all hooks. Each hook also has an argument that represents the name of the thing to be tested, but this varies from hook to hook - it's a reponame for post-repo-update, treename for post-tree- compose, etc. We should document and standardize these argument names so that it's easier to write tests that will run for multiple hooks. For example, maybe all hooks should use 'name' to represent the name of the thing being tested. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/199> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

depcheck - next steps!
by Will Woods 18 Aug '10

18 Aug '10

Time for a Depcheck Roundup! Whee! So now that I've messed with the unittests a bit and proven to myself that yes, depcheck does what it's supposed to (at least for *trivial* cases) it's time to look at the next steps: == write control/control.autoqa/test wrapper object == I took a stab at this in the depcheck branch but I'm pretty sure I did it wrong / left some stuff out. Kamil/Joza, if you guys can give me some help/pointers here I'd appreciate it... but don't worry, I'll send the patch to the list for a final review before I merge anything. == integrate mash == Basically, we need to be using the 'mash' tool rather than 'createrepo' to create the repos we use when testing updates, otherwise we'll miss anything involving multilib (like the infamous nss-softokn bug). I'm not at all sure how mash works - does anyone want to volunteer to figure this out? == more test cases == Once we're using mash instead of createrepo, we'll want to a) write a unittest or two to be sure mash is doing what it's supposed to, and b) write some depcheck unittests that prove depcheck handles multilib problems (and normal multilib packages) properly. == Get it running in autoqa! == Technically, we could start running depcheck *before* we finish the mash work - we just would ignore all multilib issues. I feel like we should wait until depcheck is working *correctly* before wiring it up, but maybe it would be helpful to have it partially working? What do you all think? == And beyond.. == Once we're satisfied that depcheck is working and doing the right thing.. then we actually talk to rel-eng and infrastructure about wiring up the proper bits so we can *reject* packages that fail depcheck. Exciting stuff! Expect flamewars! Just to state the obvious: we're probably not going to have depcheck working by F14Alpha. Oh well. But I am undaunted - we're getting reeally close. Anything else depcheck-related that I'm forgetting about here? I need to review the trac tickets, I think.. -w

1 0

some unittests in depcheck branch
by Will Woods 18 Aug '10

18 Aug '10

Hi, all - Last week we talked about writing unittests for depcheck - Kamil and I tried to list a few required test cases, and I did a little experimenting with the unittest module in the depcheck branch. Here's an example test: def test_depcheck_good_update(self): '''Make sure depcheck accepts a good update as expected''' (n, v, r) = ('mint', '2.0', '1') # twice as minty as mint 1.0! p = rpmfluff.SimpleRpmBuild(n,v,r) p.make() (rv, problems, ok_packages) = depcheck_main(self.mainrepo.id, [p.get_built_rpm(self.arch)]) self.assertEqual(len(problems), 1) self.assertTrue(problems[0].startswith(u'Success')) okp = ok_packages[0] self.assertEqual((n,v,r),(okp.name, okp.version, okp.release)) self.assertEqual(rv, 2) p.clean() (there's a setUp method that creates a very simple repo which contains three packages - 'julep', which depends on 'mint' and 'bourbon = 1.0') Kamil, is this the sort of thing you were thinking of? I think this makes sense but I'd love some input from the rest of you. If you have a chance, check out the 'depcheck' branch and have a look at the unit tests, in tests/depcheck/depcheck.py. I'd love suggestions / more unittests / whatever. Thanks! -w

1 0

Re: [PATCH 5/5] allow hooks to throw parser error on incorrectly provided arguments
by Kamil Paral 12 Aug '10

12 Aug '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > On Tue, 2010-08-10 at 15:32 +0200, Kamil Páral wrote: > > The hooks can now use their process_testdata method to check the > > correctness of provided options and arguments and abort the program > if > > something is wrong. It also allows hooks to have some mandatory > options > > defined. > > This patch looks fine, but one question: > > > -def process_testdata(opts, args, arch, **extra): > > +def process_testdata(parser, opts, args, **extra): > > Where'd 'arch' go? Was it just not needed? That was part of my control.autoqa patch: http://git.fedorahosted.org/git/?p=autoqa.git;a=commitdiff;h=55ddd92dfffe45… If you see the current workflow, hook.process_testdata() is now called just once for the whole job, not several times for every arch. This is needed, because hook.process_testdata() must be executed before control.autoqa is evaluated, and control.autoqa needs also to be run just once for the whole job, not once for every arch (otherwise control.autoqa wouldn't be able to modify the archlist). The result is the hooks are now not able to modify test input values according to a specific architecture. They have to pass some general value or the autoqa harness must be run several times for different architectures if needed (in the aforementioned patch I have modified watch-composes.py in exactly that way - it runs autoqa several times, because different archs need different arguments).

2 1

[PATCH #199] Standardize AutoQA test arguments
by Kamil Paral 11 Aug '10

11 Aug '10

This is an attempt to solve ticket #199: https://fedorahosted.org/autoqa/ticket/199 I standardized the test argument names amongst the hooks as much as possible. I also wanted to rename 'kojitag' to just 'tag', but 'tag' is a reserved word in Autotest harness (similarly to 'url'), so I left it intact. As usual, I also added a few more goodies to the patchset, so we now use native python objects for passing lists of arguments, and the hooks are now able to specify mandatory arguments and fail on bad input. I tested the changes a little, no regressions found. Comments welcome. Thanks, Kamil

2 11

Re: [PATCH 1/5] Use native python list when passing list of ENVRs for bodhi tests
by Kamil Paral 11 Aug '10

11 Aug '10

----- "Will Woods" <wwoods(a)redhat.com> wrote: > On Tue, 2010-08-10 at 15:32 +0200, Kamil Páral wrote: > > --- > > hooks/post-bodhi-update/README | 2 +- > > hooks/post-bodhi-update/hook.py | 3 +-- > > 2 files changed, 2 insertions(+), 3 deletions(-) > > One thing to note - you've changed the definition of testdata. > Previously, we (informally) defined it to be a dictionary of key=val > pairs, where both key and val were objects of type 'str'. > > Passing a value of type 'list' will still work, because autoqa is > using > %r to format testdata when it writes it into the control file, and the > result of repr(listval) can be evaluated to reproduce the original > list. > > So this is fine - in fact we can pass list, str, dict, set, or any > other > type where the result of repr(val) can be evaluated back into the > original object. > > But NOTE: while I'm fairly sure that all builtin python types should > work, this does NOT mean we can throw ANY python object into testdata > and have it come out the other side intact. For example, if we tried > to > pass (let's say) a bugzilla.Bug() into testdata, we'd end up with a > str > value like: > '<Bug #1337 on https://bugzilla.redhat.com/xmlrpc.cgi at 0x999f06c>' > > But that's being a little pedantic - since we're populating testdata > from the results of OptionParser, we're unlikely to be passing > anything > other than str and lists of strs. So this patch is fine, so long as we > remember not to get too fancy with our values in the future. I have quite forgotten we write out this data into the control file. But still, passing native python list seemed to me a better choice than using comma/space separated strings (different approach for different hooks) and easier for test writers. Sure, we can't pass on arbitrary object type.

1 0

[PATCH] treat ghost files appropriately, i.e. match rpm's behaviour
by François Cami 10 Aug '10

10 Aug '10

From: Francois Cami <fcami(a)fedoraproject.org> --- tests/conflicts/potential_conflict.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/tests/conflicts/potential_conflict.py b/tests/conflicts/potential_conflict.py index 62f62f3..c18dba0 100755 --- a/tests/conflicts/potential_conflict.py +++ b/tests/conflicts/potential_conflict.py @@ -197,7 +197,7 @@ for repo in my.repos.listEnabled(): print repo.urls[0] for pkg in my.pkgSack.returnNewestByNameArch(): - for fn in pkg.filelist + pkg.ghostlist: + for fn in pkg.filelist: if not fulldict.has_key(fn): fulldict[fn] = [] fulldict[fn].append(pkg) -- 1.7.2

2 1

[AutoQA] #216: pst: signature check?
by fedora-badges 10 Aug '10

10 Aug '10

#216: pst: signature check? --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan - package sanity tests Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- It's not explicitly stated in the Package Sanity Test Plan but I think verifying the package signature might be part of Package Sanity? We need to verify package signatures for packages that are supposed to be pushed as updates - there have been at least two recent incidents of unsigned packages landing in the live repos. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/216> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

Correct handling of %ghost
by James Laska 10 Aug '10

10 Aug '10

Greetings, Testing against F-14-Alpha highlighted several potential file conflicts between MTA's. For details see https://bugzilla.redhat.com/show_bug.cgi?id=621864. Long story short, turns out potential_conflicts.py is incorrectly including %ghost files when comparing files between packages. François Cami and Seth Vidal made the following patch and tested to confirm that the %ghost files are no longer included when comparing file conflicts. Thanks, James

1 1

New test for autoqa - upgradepath
by Vojtěch Aschenbrenner 10 Aug '10

10 Aug '10

Hi, I created new test for autoqa. Upgradepath will check if version of pushing package has good version. See ticket #123 at https://fedorahosted.org/autoqa/ticket/123 . This is first version, which will NOT consider update-testing repository (just dist-fXX and dist-fXX-update). You can get it from upgradepath branch in git. Try for example: $ autoqa post-bodhi-update --updateid FEDORA-2010-11132 \ --title mc-4.7.1-2.fc11 --targettag dist-f11 1:mc-4.7.1-2.fc11 \ --arch x86_64 -t upgradepath --local This will check if 1:mc-4.7.1-2.fc11 could be pushed to dist-f11. So every lower repos have to have lower or equal version and higher repos higher or equal version. Next version will consider update-testing repos. Request for comments :-). Vojta

3 4

[PATCH] schedule post-bodhi-update watcher with cron
by Kamil Paral 09 Aug '10

09 Aug '10

I just wonder, shouldn't we start scheduling post-bodhi-update watcher with cron? Patch attached.

2 7

AutoQATest work (jskladan branch) update
by Will Woods 09 Aug '10

09 Aug '10

So! I've been testing Josef's work on the AutoQATest class and I think I've got the rough edges filed down so it's basically working as expected! You can check out my updates to his work on the jskladan branch - I'm going to keep testing things on our local setup here until I'm more sure it's ready, and then I should hopefully be merging that branch on Monday or Tuesday. -w

2 2

Re: [AutoQA] #42: Automate test case - Basic Functionality / X basic input handling
by fedora-badges 06 Aug '10

06 Aug '10

#42: Automate test case - Basic Functionality / X basic input handling ---------------------+------------------------------------------------------ Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Future test cases Component: tests | Version: 1.0 Resolution: | Keywords: ---------------------+------------------------------------------------------ Changes (by kparal): * milestone: => Future test cases -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/42#comment:2> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

Re: [AutoQA] #41: Automate test case - Basic Functionality / X startup/basic display configuration
by fedora-badges 06 Aug '10

06 Aug '10

#41: Automate test case - Basic Functionality / X startup/basic display configuration ---------------------+------------------------------------------------------ Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Future test cases Component: tests | Version: 1.0 Resolution: | Keywords: ---------------------+------------------------------------------------------ Changes (by kparal): * milestone: => Future test cases -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/41#comment:2> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

[AutoQA] #190: Test idea - ABI compatability check
by fedora-badges 06 Aug '10

06 Aug '10

#190: Test idea - ABI compatability check -------------------------+-------------------------------------------------- Reporter: jlaska | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: tests | Version: 1.0 Keywords: | -------------------------+-------------------------------------------------- Remi Collet posted a blog article on how to use the [http://ispras.linux- foundation.org/index.php/ABI_compliance_checker ABI compliance checker] to ensure that ABI is maintained during updated releases of a package (this could be used to compare versions changes as well). See the full post at http://blog.famillecollet.com/post/2010/06/20/ABI -:-stability-check -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/190> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

Re: [AutoQA] #31: Test for unowned directories
by fedora-badges 06 Aug '10

06 Aug '10

#31: Test for unowned directories ----------------------+----------------------------------------------------- Reporter: dgregor | Owner: Type: task | Status: new Priority: minor | Milestone: Future test cases Component: tests | Version: 1.0 Resolution: | Keywords: ----------------------+----------------------------------------------------- Changes (by kparal): * milestone: => Future test cases -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/31#comment:2> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

Re: [AutoQA] #59: Add debugrepo check
by fedora-badges 06 Aug '10

06 Aug '10

#59: Add debugrepo check --------------------+------------------------------------------------------- Reporter: scop | Owner: Type: task | Status: new Priority: minor | Milestone: Future test cases Component: tests | Version: Resolution: | Keywords: --------------------+------------------------------------------------------- Changes (by kparal): * milestone: => Future test cases -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/59#comment:1> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

Re: [AutoQA] #92: Adapt font tests for autoqa
by fedora-badges 06 Aug '10

06 Aug '10

#92: Adapt font tests for autoqa ---------------------+------------------------------------------------------ Reporter: wwoods | Owner: Type: task | Status: new Priority: minor | Milestone: Future test cases Component: tests | Version: 1.0 Resolution: | Keywords: ---------------------+------------------------------------------------------ Changes (by kparal): * milestone: => Future test cases -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/92#comment:2> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 0

[AutoQA] #118: New test proposal: Python debugability
by fedora-badges 06 Aug '10

06 Aug '10

#118: New test proposal: Python debugability --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: minor | Milestone: Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- From Adam Williamson: I've cut some of the context, but basically David wants to write a test case for checking whether Python debugging is possible as intended, and I asked exactly how he wanted it to be used. {{{ On Tue, 2010-01-26 at 15:30 -0500, David Malcolm wrote: > > > Can I request a test case to cover debuggability of the Python > runtime > > > please (both in Fedora and in RHEL). > > > > > > This is in relation to: > > > https://bugzilla.redhat.com/show_bug.cgi?id=556975 > > > https://bugzilla.redhat.com/show_bug.cgi?id=558977 > > > https://bugzilla.redhat.com/show_bug.cgi?id=557772 > > > > > > as there seem to be gcc and gdb issues, which are conspiring to > make > > > python impossible to debug. > > > > > > > > > The requirement is: within "gdb python", I must be able to select > a > > > PyEval_EvalFrameEx frame, and have the following work: > > > (gdb) print co > > > (gdb) print f > > > > > > rather that have <variable optimized out> > > > > > > so that I can do this: > > > (gdb) print (char*)(((PyStringObject*)co->co_name)->ob_sval) > > > to get the function name > > > > > > (gdb) print (char*)(((PyStringObject*)co->co_filename)->ob_sval) > > > to get the source filename > > > > > > and > > > > > > (gdb) print f->f_lineno > > > to get the approximate source line number. > > > > > > If the above isn't working, it becomes extremely hard to > meaningfully > > > debug any issues that arise inside Python. > This is probably scriptable, and a good candidate for AutoQA and foe > RHTS. > }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/118> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[PATCH 1/2] add repoinfo.getreleases() and get_bodhi_releases()
by Will Woods 05 Aug '10

05 Aug '10

--- lib/python/repoinfo.py | 6 ++++++ repoinfo.conf | 4 ++++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/lib/python/repoinfo.py b/lib/python/repoinfo.py index 420becc..3a8c1f8 100644 --- a/lib/python/repoinfo.py +++ b/lib/python/repoinfo.py @@ -39,6 +39,12 @@ class RepoinfoConfig(object): return [t.strip() for t in itemlist.split(',')] def getarches(self, name): return self._getlist(name, 'arches') + def getreleases(self): + '''Return the list of known, non-EOL releases.''' + return [r for r in repoinfo.repos() if self.config.get(r,'isactiverelease')] + def get_bodhi_releases(self): + '''Return the list of releases known to Bodhi.''' + return [r.upper() for r in self.getreleases()] def getparents(self, name): '''Given a repo name, give the names of its parent repos e.g.: getparents('f13-updates-testing') -> ['f13','f13-updates]''' diff --git a/repoinfo.conf b/repoinfo.conf index 3241044..d36dd93 100644 --- a/repoinfo.conf +++ b/repoinfo.conf @@ -8,6 +8,7 @@ baseurl = http://download.fedoraproject.org/pub/fedora/linux goldurl = %(baseurl)s/releases/%(path)s/Everything/%(arch)s/os updatesurl = %(baseurl)s/updates/%(path)s/%(arch)s rawhideurl = %(baseurl)s/%(path)s/%(arch)s/os +isactiverelease = # Rawhide [rawhide] @@ -19,6 +20,7 @@ collection_name = devel # Fedora 14 [f14] +isactiverelease = yes path = development/14 url = %(rawhideurl)s collection_name = F-14 @@ -37,6 +39,7 @@ collection_name = F-14 # Fedora 13 [f13] +isactiverelease = yes path = 13 url = %(goldurl)s collection_name = F-13 @@ -55,6 +58,7 @@ collection_name = F-13 # Fedora 12 [f12] +isactiverelease = yes path = 12 url = %(goldurl)s collection_name = F-12 -- 1.7.2

2 3

[PATCH] post-bodhi-update watcher: a few fixes
by Kamil Paral 05 Aug '10

05 Aug '10

These two patches fix some issues with post-bodhi-update watcher, as discussed on IRC. I haven't really studied working with repoinfo.conf-style config files, so please review if it is correct. Thanks. It is tested and it works.

2 5

[PUSHED] post-bodhi-update: fix hook to conform to control.autoqa patchset
by Kamil Paral 04 Aug '10

04 Aug '10

This was an obvious omission of the control.autoqa patchset, I forgot to fix post-bodhi-update hook. I pushed it straightaway. http://git.fedorahosted.org/git/?p=autoqa.git;a=commit;h=4dff3d9761eaafca3e…

1 0

[PATCH] deprecate email opt-in for koji watcher
by Kamil Paral 04 Aug '10

04 Aug '10

I'm not sure about all consequences of this patch. But it seems to me that we don't need collection_name and email opt-in anymore. What do you think, Will?

2 4

[AutoQA] #184: Add tags for destructive testing
by fedora-badges 02 Aug '10

02 Aug '10

#184: Add tags for destructive testing ----------------------------+----------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- We need to add tags into autotest and autoqa tests that will determine whether test is (may be) destructive. This tag must be propagated and appropriate test client must be selected. After finishing this test the test machine must be restored to a previous safe state. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/184> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #181: Quick and dirty support for Virtualization
by fedora-badges 02 Aug '10

02 Aug '10

#181: Quick and dirty support for Virtualization ----------------------------+----------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- The task is to implement quick and dirty support for virtualization for AutoQA tests. We can already install autotest-client into virt machine, but there are some tests which must not run inside virt environment (tests using virt themselves for example). Therefore we must ensure all our tests are marked properly ("may use virt", "may not use virt"), all our tests clients are marked properly ("is virt", "is bare metal"), and this information are propagated from AutoQA to autotest-server and used properly when selecting appropriate host and scheduling jobs. This could be quite easy, there is some support for tags in autotest already. As an enhancement, we may think about preferences. Do we want to mark some test that it is preferred to run it in virt, but may also be run on bare metal? How do we achieve correct behavior for autotest-server for such task? For example: We may want to prefer executing sanity tests in virt env, but it's not really mandatory, just desired. Food for thoughts. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/181> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 10

[AutoQA] #207: post-koji-build watcher does not watch for new updates-candidate builds
by fedora-badges 02 Aug '10

02 Aug '10

#207: post-koji-build watcher does not watch for new updates-candidate builds ----------------------+----------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: watchers | Version: 1.0 Keywords: | ----------------------+----------------------------------------------------- I have found out that our post-koji-build watches only these tags: dist-f10 dist-f10-updates dist-f10-updates-testing dist-f11 dist-f11-updates dist-f11-updates-testing dist-f12 dist-f12-updates dist-f12-updates-testing dist-f13 dist-f13-updates dist-f13-updates- testing dist-f14 Which is incorrect, we should be watching dist-f*-updates-candidate (and maybe dist-f14, I don't know). We don't need to watch for the rest of it. The change happened in commit 580734d4f2aa768aefa1c0498904df2296747918. See the diff and inspect current contents of repoinfo.conf to grasp the change. The result is that we don't currently test almost any packages except f14 ones. Do we have it already deployed on our production machine? What we can do about it? Well, I hope someone with a deep understanding of repoinfo stuff knows what's the best approach :) It seems to me that we could define the *-updates-testing repositories in repoinfo.conf (and make sure we didn't break something else) and retrieve and select just those in watch-koji-builds. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/207> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

2 5

[AutoQA] #200: Multiple control files / multiple test wrappers
by fedora-badges 02 Aug '10

02 Aug '10

#200: Multiple control files / multiple test wrappers --------------------+------------------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Multi-hook tests Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- For a test to run under multiple hooks, it may need multiple control files, since we may want to do slightly different things to prepare for each test. We could have multiple test wrappers and conditionalize the job.run_test() argument based on the calling hook, for example: if hook == 'post-koji-build': job.run_test('mytest_koji', name=name, envrs=envrs, ...) elif hook == 'post-bodhi-update': job.run_test('mytest_bodhi', name=name, envrs=envrs, ...) Alternately, we could have multiple control.HOOKNAME files - one for each hook the test can be run under. This would also allow us to eliminate the use of the 'testlist' file to determine the tests that are eligible for use in each hook - each test would be able to declare how to run itself under each hook. This ticket requires deciding on a method for handling multi-hook tests, documenting that method, and (perhaps) converting existing tests. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/200> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #211: Update repoinfo.conf - Add branch f14
by fedora-badges 02 Aug '10

02 Aug '10

#211: Update repoinfo.conf - Add branch f14 ----------------------------+----------------------------------------------- Reporter: jkeating | Owner: Type: task | Status: new Priority: critical | Milestone: Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- We branched, SOp says make a ticket. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/211> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #195: Write HelloWorld test
by fedora-badges 02 Aug '10

02 Aug '10

#195: Write HelloWorld test --------------------+------------------------------------------------------- Reporter: kparal | Owner: kparal Type: task | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Write a HelloWorld test that will just serve for testing purposes. More information here: https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000727.html If anyone is eager to write this, feel free to reassign. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/195> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

Update repoinfo.conf for F-14 branch
by James Laska 02 Aug '10

02 Aug '10

Greetings, Per ticket#211, I have updated repoinfo.conf for the new F-14 branch. Instructions for updating repoinfo.conf are on the wiki at https://fedoraproject.org/wiki/How_to_update_AutoQA_repoinfo.conf. I have tested this change and confirmed that watch-koji-builds.py goes from monitoring these repos: Looking for builds with tags dist-f10-updates-candidate dist-f11-updates-candidate dist-f12-updates-candidate dist-f13-updates-candidate dist-f14 To monitoring for koji builds in these repos: Looking for builds with tags dist-f10-updates-candidate dist-f11-updates-candidate dist-f12-updates-candidate dist-f13-updates-candidate dist-f14-updates-candidate dist-f15 Thanks, James

2 2

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel August 2010