March 2012 - autoqa-devel - Fedora mailing-lists

[AutoQA] #251: Move autotest applications into /usr/*bin
by fedora-badges 26 Sep '12

26 Sep '12

#251: Move autotest applications into /usr/*bin -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: minor | Milestone: Packaging, Review, & Deployment Component: packaging | Keywords: -----------------------+---------------------------------------------------- The autotest package should supply binaries in /usr/bin/ ... so we can rely on PATH and not have to call autotest with the full-path (/usr/share/autotest/client/bin/autotest) when scheduling jobs in '/usr/bin/autoqa'. Alternatively, the autotest package could drop a script that updates PATH appropriately into /etc/profile.d/autotest. But yuck. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/251> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #191: Submit autotest for Fedora package approval
by fedora-badges 26 Sep '12

26 Sep '12

#191: Submit autotest for Fedora package approval -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Packaging and Review Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- * https://bugzilla.redhat.com/show_bug.cgi?id=548522 Currently stalled, pending packaging of the gwt package. This includes packaging all bundled .jar files included with gwt. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/191> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #414: rats_install: run twice, once for updates, once for updates+updates-testing
by fedora-badges 14 Aug '12

14 Aug '12

#414: rats_install: run twice, once for updates, once for updates+updates-testing --------------------------+------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Nice to have soon Component: tests | Keywords: Blocked By: | Blocking: --------------------------+------------------------------- We decided it would be beneficial to run rats_install twice. Once for stable updates only, once for main+updates, once for main+updates+updates- testing. Currently we're running just the second option. The discussion is here: https://fedoraproject.org/wiki/QA/Meetings/20120319#t15:55:42 Change rats_install to execute twice and report both variants. The easiest way is probably to modify 'control' file to execute the job twice and modify some input argument to distinguish the two cases. Also we need to change the summary accordingly so that we can easily distinguish that in our ResultsDB view. Marking "Nice to have soon" if we can do it before F17 Final, otherwise it'll probably go into "Future tasks". -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/414> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 10

[AutoQA] #157: conflicts: requested arch not respected
by fedora-badges 06 Aug '12

06 Aug '12

#157: conflicts: requested arch not respected --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This is a bug in the potential_conflict.py script in tests/conflicts/. Usage suggests using --arch options for specifying requested architecture to be tested. But if you look at the attached log, the first and second output match (that's correct), but the third doesn't match, even though it should. Everything was tested on x86_64 machine. Also from the usage line it is not clear whether --arch is related only to particular yum repository selection (replacing $basearch keyword), or whether it also relates to packages built for one architecture but being in different repo (e.g. i686 packages are very often in x86_64 repo). I have looked into the source code of the script and it seems that the --arch option is not used at all, just printed out in the usage help?? Seth, you're listed as script author. Any comments on that? Thanks. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/157> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #270: Package and submit for review - compat-Django-1.0.4
by fedora-badges 02 Aug '12

02 Aug '12

#270: Package and submit for review - compat-Django-1.0.4 -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: packaging | Keywords: -----------------------+---------------------------------------------------- Currently packaged so that it can coexist with the current Django package (see http://repos.fedorapeople.org/repos/fedora- qa/autoqa/fedora-14-testing/). I need to port existing security patches linked from http://www.djangoproject.com/weblog/2010/dec/22/security/ Once patched, submit for review ... and enjoy. Eventually autotest will be updated to use Django-1.2 ... but packaging Django-1.0.4 removes one more packaging obstacle to having autotest as an official Fedora package. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/270> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #329: implement RPM checksum verification
by fedora-badges 30 Jul '12

30 Jul '12

#329: implement RPM checksum verification -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: Finger Food Component: core | Keywords: -------------------------+-------------------------------------------------- Currently this method is in autoqa.util module: {{{ 261 def valid_rpm(rpm_file): 262 '''Check that RPM file is valid. 263 Note: This currently only extracts RPM header and presumes the file is 264 OK if that succeeds. Full RPM hashsum check would be nice to have 265 implemented too (FIXME). 266 @param rpm_file RPM file path 267 @return True if file seems OK or False if there is some problem (file 268 can't be read, RPM header is corrupt) 269 ''' ... }}} We would like to have full RPM checksum verification, not just reading the header and saying that it's OK. Find out how to do it and change the method accordingly. It would be nice to have a method argument to enable full checksum verification/fall back to original header verification, because it can be performance intensive operation and it may not be suitable for all our use cases). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/329> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #416: Make the code pylint friendly
by fedora-badges 30 Jul '12

30 Jul '12

#416: Make the code pylint friendly --------------------------+------------------------- Reporter: mkrizek | Owner: Type: enhancement | Status: new Priority: trivial | Milestone: Finger Food Component: core | Keywords: Blocked By: | Blocking: --------------------------+------------------------- The task here is to go through the code and fix whatever [http://pypi.python.org/pypi/pylint pylint] complaints about. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/416> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #396: bodhi_util: Could not connect to bodhi!
by fedora-badges 02 Jul '12

02 Jul '12

#396: bodhi_util: Could not connect to bodhi! ---------------------+------------------------ Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: ---------------------+------------------------ There seems to be some problem when trying to submit certain results to Bodhi: {{{ 12/22 07:10:13 INFO | test:0478| Submitting results (log, email, bodhi) for: depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16 12/22 07:10:13 INFO | test:0361| Log created: /usr/share/autotest/results/default/depcheck/results/kde- l10n-4.7.4-1.fc1.html 12/22 07:10:13 INFO | test:0307| Email not sent: "depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16" 12/22 07:10:17 ERROR|bodhi_util:0278| An error occured: ServerError(http://aqd/bodhi/list, 500, Internal Server Error) 12/22 07:10:17 ERROR|bodhi_util:0279| Could not connect to bodhi! 12/22 07:10:17 ERROR|bodhi_util:0279| Could not post a comment to bodhi }}} This is running on my development machine when using mock_fedorainfra tool. I haven't spotted it yet on either production or staging machine. That indicates that this could be a problem inside the mock_fedorainfra tool. I have seen this bug happening several times, always only for updates consisting of several builds. Investigate whether this is really a problem in mock_fedorainfra and then act on accordingly (fix the problem/report in upstream). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/396> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

Playing around with Sikuli
by Tim Flink 05 Jun '12

05 Jun '12

I've been wanting to try out Sikuli (http://www.sikuli.org) for a while now and finally got around to making it build and work on Fedora. The basic concept is to create GUI test cases based on images instead of absolute (x,y) coordinates (like selenium), scripted key sequences (openqa) or using an accessability layer (like dogtail). I've put together a tarball with source and build scripts if anyone else wants to try it out: http://tflink.fedorapeople.org/tools/sikuli/README_build-sikuli.txt http://tflink.fedorapeople.org/tools/sikuli/fedora-sikuli-20120220.tar.gz The distribution method is a really dirty hack but I've been able to build on both F15 and F16 after installing a huge number of -devel dependencies and building all of Sikuli's direct dependencies from source. Xpresser (https://wiki.ubuntu.com/Xpresser) might be another option - it's a re-implementation of the concepts from Sikuli. It seems to be missing some of Sikuli's features but would likely be quite a bit easier to build and integrate. One of the things I have in mind is to attempt automating at least some of the installation test cases because a lot of them pass most of the time and they can be a bit mind-numbing to do over and over again. I'm of the opinion that our human resources could be put to better use than monkey-button-pushing their way through simple test cases. A non-trivial amount of work would be needed before we could fully utilize this (fixing the code so that it is package-able, better integration with VNC etc.) but I'm wondering if it could be a good candidate for automating some of the installation test cases. I've written two test cases for Sikuli thus far: a simple DVD graphical install and graphical firstboot. I had to workaround some timing issues due to occasional lag in the VM's VNC interface but they seem to be working well with F17 Alpha RC2 for now. F17 DVD Basic Graphical Install: http://tflink.fedorapeople.org/tools/sikuli/f17-basicGraphicalDVD_sikuli.ta… F17 Graphical Firstboot: http://tflink.fedorapeople.org/tools/sikuli/f17-graphicalFirstboot_sikuli.t… - NOTE: This only works some of the time due to rhbz#750527 which was closed with no bug comment by the developers. I'm going to bug them about this but will likely update the test case to work with unpredictable order of the firstboot screens. To run those test cases, extract them somewhere and point the Sikuli IDE at the extracted directories. I'm going to be running my basic graphical install testcase for different F17 releases to get an initial feel for how fragile the test cases are and how much effort would be required to maintain these kinds of test cases over a release. Anyhow, long email over. Any thoughts on whether this might be worth investigating farther? Tim

3 4

[AutoQA] #415: results not being reported to bodhi (Expectation Error)
by fedora-badges 11 May '12

11 May '12

#415: results not being reported to bodhi (Expectation Error) -----------------------+------------------------ Reporter: tflink | Owner: Type: defect | Status: new Priority: critical | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: -----------------------+------------------------ I was just pinged regarding the lack of a depcheck run for a bodhi update - https://admin.fedoraproject.org/updates/abattis-cantarell- fonts-0.0.8-1.fc17 After looking into the logs: - http://autoqa.fedoraproject.org/results/315536-autotest/virt01.qa/debug/cli… we're getting errors when attempting to make the bodhi comment to report the results: {{{ 03/29 14:16:37 ERROR|bodhi_util:0249| An error occured: ServerError(https://admin.fedoraproject.org/updates/list, 417, Expectation Failed) 03/29 14:16:37 ERROR|bodhi_util:0250| Could not connect to bodhi! 03/29 14:16:37 ERROR|bodhi_util:0250| Could not post a comment to bodhi }}} I'm not sure what's going on here but will investigate and update here as I learn more. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/415> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

AutoQA and Fedora Messaging
by Tim Flink 29 Mar '12

29 Mar '12

I think that we had plans to use Fedora Messaging back when it was first proposed but I don't think that we've talked about it recently. The current timeline predicts that fedmsg will be deployed into production by the end of the calendar year, potentially sooner. From what I can gather, there is an expectation that AutoQA will eventually use fedmsg to broadcast results and schedule tests, replacing the current method of direct Bodhi comments and cron jobs to poll Bodhi/Koji. I don't think that it's anything we have to worry about right now but it's probably something that we want to start thinking about in the near future. Hopefully there will be some test support code coming with fedmsg (mocks/stubs, mostly) that would help us in development but AFAIK, none of that has been written yet. I'm planning to keep attending the Messaging meetings and will pass on updates as I learn about them. Tim

2 1

Fw: Messaging SIG - IRC Meetings on Tuesdays
by Tim Flink 26 Mar '12

26 Mar '12

Just a heads up in case this gets lost in the noise on devel@ I'm planning to go since this will affect us pretty heavily. Tim Begin forwarded message: Date: Mon, 26 Mar 2012 15:25:39 -0400 From: RJ Bean <rbean(a)redhat.com> To: devel <devel(a)lists.fedoraproject.org> Subject: Messaging SIG - IRC Meetings on Tuesdays Just an announcement - the Messaging SIG is starting regular IRC meetings on Tuesdays at 16:00 UTC. Wiki page for the SIG: http://fedoraproject.org/wiki/Messaging_SIG My work on a proposal and a python library: https://github.com/ralphbean/fedmsg/blob/develop/doc/proposal.rst -- devel mailing list devel(a)lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel

1 0

[AutoQA] #206: Update bodhi to enforce Package Update Acceptance policy
by fedora-badges 26 Mar '12

26 Mar '12

#206: Update bodhi to enforce Package Update Acceptance policy ----------------------------+----------------------------------------------- Reporter: wwoods | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- In order to enforce the Package Update Acceptance Test Plan, Bodhi will need be modified to reject the push of any package that fails acceptance testing. The acceptance tests may need to send status/data to bodhi in order for it to make decisions about policy (see e.g. ticket #205). Later bodhi might just get data from resultdb (see that milestone for details). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/206> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #252: Provide all autotest client combinations that we need
by fedora-badges 16 Mar '12

16 Mar '12

#252: Provide all autotest client combinations that we need ------------------------+--------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: 0.5.0 Component: production | Keywords: ------------------------+--------------------------------------------------- Currently our production server at http://autoqa.fedoraproject.org/ has 6 autotest clients assigned, all bare metals with Fedora 13 installed, with mixed architectures. But we need more. According to https://fedoraproject.org/wiki/Managing_autotest_labels we have two architecture labels, two fedora release labels (the two currently supported Fedora releases, I would omit Rawhide for now) and two virt labels (virtual machine or bare metal). We need machines for every combination of those labels (that is 8 IICC - if I count correctly). And we would like to see a bit more 64bit machines, because we use them for noarch tests. Currently the only test that really requires all those combinations is initscripts test. But there will be more in the future. It is quite some farm, but it seems we'll have to build it. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/252> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

AutoQA 0.8 released
by Kamil Paral 16 Mar '12

16 Mar '12

Hey, it's celebration time! AutoQA 0.8 has been released and deployed. Together with that our first attempt at hosting a public ResultsDB instance has been made. I've summarized some changes in the blogpost: http://kparal.wordpress.com/2012/03/16/autoqa-0-8-released/ See the linked NEWS file for more detailed and developer-oriented changes. Also you might find interesting that our production and staging server now redirects to ResultDB by default if you request the root document in your browser. That is http://autoqa.fedoraproject.org/ (or autoqa-stg). If you want to access Autotest interface, you have to access http://autoqa.fedoraproject.org/afe (or autoqa-stg). As a nice bonus, today is the first day when our rats_install test (automated anaconda installation) passed for F17. A new era begins! http://autoqa.fedoraproject.org/resultsdb/frontend/search?type=Testcase&ter… Thanks for all your work, Kamil

1 0

[AutoQA] #391: re-enable autoqa-results ML
by fedora-badges 15 Mar '12

15 Mar '12

#391: re-enable autoqa-results ML ------------------------+--------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: 0.8.0 Component: production | Keywords: ------------------------+--------------------------------------------------- Read these threads: * https://fedorahosted.org/pipermail/autoqa- devel/2011-September/002894.html * https://fedorahosted.org/pipermail/autoqa- devel/2011-October/002898.html Find and implement fix so that we can easily inspect the test results again. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/391> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

[AutoQA] #413: conflicts: yum.Errors.NoMoreMirrorsRepoError: failure: Packages/cernlib-g77-devel-2006-33.fc12.i686.rpm from parent-2: [Errno 256] No more mirrors to try.
by fedora-badges 15 Mar '12

15 Mar '12

#413: conflicts: yum.Errors.NoMoreMirrorsRepoError: failure: Packages/cernlib-g77-devel-2006-33.fc12.i686.rpm from parent-2: [Errno 256] No more mirrors to try. ---------------------+--------------------- Reporter: kparal | Owner: mkrizek Type: defect | Status: new Priority: major | Milestone: 0.8.0 Component: tests | Keywords: Blocked By: | Blocking: ---------------------+--------------------- This is really weird, I already saw it several times. I don't see 'conflicts' test as very important, but still, Martin, can you have a look at it? How does it happen that there's a package that is not available from any repos? Please note the package name is different for different repos. Are we having just connection issues? {{{ Traceback (most recent call last): File "./potential_conflict.py", line 243, in <module> elif file_conflict(fn, pkglist): File "./potential_conflict.py", line 140, in file_conflict pkg.repo.getHeader(pkg) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 883, in getHeader cache=cache, size=size, File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 842, in _getFile raise Errors.NoMoreMirrorsRepoError, errstr yum.Errors.NoMoreMirrorsRepoError: failure: Packages/cernlib-g77-devel-2006-33.fc12.i686.rpm from parent-2: [Errno 256] No more mirrors to try. }}} http://autoqa- stg.fedoraproject.org/results/32302-autotest/qa07.qa.fedoraproject.org/conf… -updates-testing-.html http://autoqa- stg.fedoraproject.org/results/32311-autotest/virt23.qa/conflicts/results/f16 -updates-testing-.html http://autoqa- stg.fedoraproject.org/results/32308-autotest/virt24.qa/conflicts/results/f16 -updates-x86_64.html http://autoqa- stg.fedoraproject.org/results/32296-autotest/virt27.qa/conflicts/results/f15 -updates-x86_64.html http://autoqa- stg.fedoraproject.org/results/32001-autotest/virt24.qa/conflicts/results/f1… http://autoqa- stg.fedoraproject.org/results/32488-autotest/virt24.qa/conflicts/results/f1… http://autoqa- stg.fedoraproject.org/results/32617-autotest/virt26.qa/conflicts/results/f17 -updates-testing-.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/413> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 8

[AutoQA] #373: ResultsDB: Add 'store to ResultsDB' functionality to test.py
by fedora-badges 14 Mar '12

14 Mar '12

#373: ResultsDB: Add 'store to ResultsDB' functionality to test.py ----------------------+----------------------------------------------------- Reporter: jskladan | Owner: jskladan Type: task | Status: new Priority: major | Milestone: 0.7.0 Component: resultdb | Keywords: ----------------------+----------------------------------------------------- In order to store data into resultsdb, we need to create a method which will access resultsdb's XMLRPC interface and store data from inside the post_results() method. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/373> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

Fedora Message Bus
by Tim Flink 14 Mar '12

14 Mar '12

I had a chance to talk with lmacken and 3bean while at PyCon and learned that the Fedora Message Bus project [1] is being worked on again. It sounds like there is still a lot of work that needs to happen before we see it implemented but it might be a good way out of our current method of test scheduling. I'm planning to attend the meetings for now and we'll see where this goes. I expect that it will eventually change the way that we schedule tests in AutoQA. Tim [1] https://fedoraproject.org/wiki/Messaging_SIG

1 0

[AutoQA] #412: depcheck crash: KeyError: 'at-spi2-atk-2.3.91-1.fc17'
by fedora-badges 14 Mar '12

14 Mar '12

#412: depcheck crash: KeyError: 'at-spi2-atk-2.3.91-1.fc17' ---------------------+------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: 0.8.0 Component: tests | Keywords: Blocked By: | Blocking: ---------------------+------------------- {{{ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/autoqa/decorators.py", line 72, in newf f_result = f(*args, **kwargs) #call the decorated function File "/usr/share/autotest/tests/depcheck/depcheck.py", line 344, in run_once td.output.append(digest.digest[build['nvr']]['all']) KeyError: 'at-spi2-atk-2.3.91-1.fc17' }}} http://autoqa- stg.fedoraproject.org/results/31804-autotest/qa06.qa.fedoraproject.org/depc… -updates-testing-.html http://autoqa- stg.fedoraproject.org/results/31803-autotest/qa07.qa.fedoraproject.org/depc… -updates-testing-.html I would like to fix this in AutoQA 0.8 milestone. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/412> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[PATCH] #412 - Depcheck crash: KeyError
by Josef Skladanka 14 Mar '12

14 Mar '12

I'v added try-except block around the critical area. Kamil and I found out, that this error happens, when packagers push new builds into Bodhi update, in the time of depcheck-run on the respective update (to be more specific, after we query Koji for -pending builds, and before we sum-up the build results to update results). If this happens, we now add the build do problematic builds list (with appropriate error message), which subsequently FAILs the update. ======================================================================= diff --git a/tests/depcheck/depcheck.py b/tests/depcheck/depcheck.py index 819748f..ea101f0 100644 --- a/tests/depcheck/depcheck.py +++ b/tests/depcheck/depcheck.py @@ -340,8 +340,21 @@ class depcheck(AutoQATest): desc = 'Update %(result)s: %(title)s\n' . for build in update['builds']: + # Sometimes, update is changed during depcheck testrun + # and new build (which was not in -pending at the start) + # is added. When this happens, we want to skip the build + # and mark it as problematic (to subsequently fail the. + # update) + try: + td.output.append(digest.digest[build['nvr']]['all']) + except KeyError: + msg = "Build %s is part of update '%s', but it was " \ + "added during the depcheck run" % (build['nvr'], update['title'])) + self.log(msg, stderr = True, highlight = True, details = td) + td.update_result("ABORTED") + self.problematic[build['nvr']] = msg + continue . - td.output.append(digest.digest[build['nvr']]['all']) td.output.append("\n\n") for h in digest.digest[build['nvr']]['highlights']: td.highlights.append((h, None)) @@ -349,7 +362,7 @@ class depcheck(AutoQATest): if build['nvr'] not in pending_ok: msg = "Build %s is part of update '%s', but it wasn't " \ "scheduled for testing" % (build['nvr'], update['title']) - self.log(msg, strerr = True, highlight = True, details = td) + self.log(msg, stderr = True, highlight = True, details = td) td.update_result('ABORTED') self.problematic[build['nvr']] = msg continue

2 1

[AutoQA] #411: depcheck: Ignored builds now don't cause the update to fail depcheck
by fedora-badges 07 Mar '12

07 Mar '12

#411: depcheck: Ignored builds now don't cause the update to fail depcheck ---------------------+-------------------------- Reporter: kparal | Owner: jskladan Type: task | Status: new Priority: major | Milestone: 0.8.0 Component: tests | Keywords: missing_docs Blocked By: | Blocking: ---------------------+-------------------------- This is a documentation request for: {{{ commit 5dd9214 Author: Josef Skladanka <jskladan(a)redhat.com> Date: Thu Feb 9 15:35:26 2012 +0100 Ignored builds now don't cause the update to fail depcheck }}} This page may need to be updated: https://fedoraproject.org/wiki/AutoQA_tests/Depcheck -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/411> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #255: Use Config class to simplify access to configuration options
by fedora-badges 07 Mar '12

07 Mar '12

#255: Use Config class to simplify access to configuration options --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: minor | Milestone: Component: core | Keywords: --------------------+------------------------------------------------------- Currently all our libraries and tests use ConfigParser and parse our configuration files (autoqa.conf, fas.conf, etc) directly. That is not ideal, it involves many lines of code, converting to correct variable types (int, bool), handling default values, and so on. We should create a Python class Config, which is initialized once in the beginning, and then can be access globally and easily from any other module. It will handle default values and type conversion centrally in one place. Also access will be much easier. PS: We already have autoqa.config module. Maybe we just don't use it? -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/255> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

[AutoQA] #350: Add support for using file proxy
by fedora-badges 07 Mar '12

07 Mar '12

#350: Add support for using file proxy -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Future tasks Component: core | Keywords: -------------------------+-------------------------------------------------- Currently we allow to cache downloaded RPMs with allow_pkg_cache in autoqa.conf. That uses our own code in util.py:download(). Tim was talking about using file proxy and it now seems to me as a far better approach. It can cache more files (like yum metadata, ISO files, etc). The proxy can run on the same computer or on some local-network computer. Several computers can share the same proxy. We don't have to care about storing the files and cleaning up the directories. It means less code. It's transparent for us. The task now is to play with some proxy, try whether it works, implement some support for AutoQA (if required), and document the process of setting it up (probably in [https://fedoraproject.org/wiki/AutoQA_Development here]). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/350> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

[AutoQA] #394: All tests must have their maintainer defined
by fedora-badges 07 Mar '12

07 Mar '12

#394: All tests must have their maintainer defined ---------------------+------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: minor | Milestone: Finger Food Component: tests | Keywords: Blocked By: | Blocking: ---------------------+------------------------- Occasionally we are running into issues with tests for which we are not sure who to ask help with. Every test should have its "maintainer" defined. The maintainer is a person who to ask about implementation details or who to harass about bugs. Currently the autotest's control file contains AUTHOR="..." line. Maybe we should use that for maintainer definition (even though it might be a little misleading). Or maybe we should define and use a MAINTAINER="..." variable. The task is: 1. Find the best way where to define each test's maintainer (AUTHOR/MAINTAINER inside control file, etc). 2. Decide whether we need to check its existence. 3. Find maintainers for every test present inside AutoQA and fill it in inside the source code. 4. Amend the documentation. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/394> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 4

[AutoQA] #390: update rats_install
by fedora-badges 07 Mar '12

07 Mar '12

#390: update rats_install ----------------------+----------------------------------------------------- Reporter: hongqing | Owner: hongqing Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: core | Keywords: ----------------------+----------------------------------------------------- rats_install does not work after F14. after investigating in it, The following things should be updated:[[BR]] 1. install.img is compressed into initrd.img, remove the exist check 2. initrd.img is compressed as XZ compressed file instead of gzip compressed file, gzip in python has to be replaced -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/390> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 25

[AutoQA] #338: AutoQATest.version - is it needed?
by fedora-badges 06 Mar '12

06 Mar '12

#338: AutoQATest.version - is it needed? -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: trivial | Milestone: Finger Food Component: tests | Keywords: -------------------------+-------------------------------------------------- In all our tests we have code like this: {{{ class rpmlint(AutoQATest): version = 1 # increment if setup() changes }}} The question is - is the ''version'' attribute needed and being used at all? I have never seen any difference in behavior, the setup() phase is run every time. We should have look at autotest documentation or source code whether this is being used and when. If we don't need it, let's remove it from all our tests and templates (the less code the better). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/338> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

documentation for 0.8
by Kamil Paral 05 Mar '12

05 Mar '12

You didn't think you can get away from this, did you? I have created and assigned tickets for every piece of documentation we can be possibly missing: https://fedorahosted.org/autoqa/report/3 Onwards, my faithful coders, fulfill your duty! The Slave Master

1 0

[AutoQA] #186: Automate media kit sanity tests
by fedora-badges 05 Mar '12

05 Mar '12

#186: Automate media kit sanity tests --------------------+------------------------------------------------------- Reporter: jlaska | Owner: lili Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This ticket is intended to track automating the mediakit sanity tests (http://fedoraproject.org/wiki/Category:Installer_Image_Sanity_Test_Cases) Discussion already underway, along with code in Liam's private branch. For details see https://fedorahosted.org/pipermail/autoqa- devel/2010-June/000595.html -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/186> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 9

[AutoQA] #392: Check whether compose_tree needs updating for F17 compose changes (treebuilder)
by fedora-badges 05 Mar '12

05 Mar '12

#392: Check whether compose_tree needs updating for F17 compose changes (treebuilder) ----------------------+----------------------------------------------------- Reporter: adamwill | Owner: tflink Type: task | Status: new Priority: major | Milestone: 0.8.0 Component: tests | Keywords: ----------------------+----------------------------------------------------- I don't really grok the details, but I know there have been some changes to the image compose process post-F16, involving a new branch of lorax called treebuilder: https://fedoraproject.org/wiki/Anaconda/Features/Lorax-TreeBuilder wwoods has a guide up at: https://fedoraproject.org/wiki/User:Wwoods/Building_images_with_treebuilder We should check the impact of this change on AutoQA. immediately, does the compose_tree script need updating? -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/392> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #408: depcheck: Argument list too long
by fedora-badges 05 Mar '12

05 Mar '12

#408: depcheck: Argument list too long ---------------------+------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: minor | Milestone: Nice to have soon Component: tests | Keywords: Blocked By: | Blocking: ---------------------+------------------------------- {{{ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/autoqa/decorators.py", line 72, in newf f_result = f(*args, **kwargs) #call the decorated function File "/usr/share/autotest/tests/depcheck/depcheck.py", line 174, in run_once depcheck_output = utils.system_output(cmd, retain_output=True) File "/usr/share/autotest/common_lib/base_utils.py", line 931, in system_output args=args).stdout File "/usr/share/autotest/common_lib/base_utils.py", line 654, in run stderr_level=get_stderr_level(stderr_is_expected)),), File "/usr/share/autotest/common_lib/base_utils.py", line 79, in __init__ stdin=stdin) File "/usr/lib64/python2.7/subprocess.py", line 672, in __init__ errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1202, in _execute_child raise child_exception OSError: [Errno 7] Argument list too long }}} http://autoqa- stg.fedoraproject.org/results/27644-autotest/virt24.qa/depcheck/results/ful… Sometimes we hit system limits. Proposed solution: Write pending builds to one file, accepted builds to a different file, then provide these files using different command line options to depcheck. Place both files in the results directory. I don't think this issue is really hot, because it happens seldom. Putting to "nice to have soon". -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/408> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

ResultsDB integration in master *fireworks* (also ticket 408)
by Josef Skladanka 05 Mar '12

05 Mar '12

Hi gang, after a long long time, ResultsDB integration has landed into master. Kamil will push it to stagign shortly, and we'll see what went wrong, before going live to production. Also, the patch for ticket #408 (depcheck arguments too long), is there so we should have a bit-less CRASHED results from now on. j.

1 0

[AutoQA] #194: Incorporate automated anaconda storage suite
by fedora-badges 05 Mar '12

05 Mar '12

#194: Incorporate automated anaconda storage suite --------------------+------------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Automate installation test plan Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- Chris Lumens has developed a virt-based test suite to automate different disk/storage install scenarios. I've discussed this briefly with Chris, but I'd be interested in seeing his test suite incorporated within autoqa and run on a regular interval (perhaps along-side Liam's install tests). The tests are currently available at http://clumens.fedorapeople.org /anaconda-storage-test/ -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/194> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 1

[AutoQA] #361: improve pretty log output for anaconda tests
by fedora-badges 05 Mar '12

05 Mar '12

#361: improve pretty log output for anaconda tests --------------------+------------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: minor | Milestone: Finger Food Component: tests | Keywords: --------------------+------------------------------------------------------- With the advent of the super awesome html pretty logs, some of the less frequented tests are in need of some care and feeding. While reviewing installer test results with the anaconda team, I noticed that the installer tests don't all look good with pretty logs. Sample output for existing anaconda test results ... * [http://autoqa.fedoraproject.org/results/143510-autotest/qa02.qa.fedoraproje… anaconda_storage] - Doesn't look bad ... I don't think I'd recommend changes here. * [http://autoqa.fedoraproject.org/results/143512-autotest/qa07.qa.fedoraproje… compose_tree] - Missing some info * [http://autoqa.fedoraproject.org/results/143577-apache/10.5.124.249/anaconda… anaconda_checkbot] - Yuck, no info at all :( This ticket is intended to address improving the html result output for the above installer tests. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/361> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 11

Review Request: conflicts and repoclosure tests minor refactoring
by Martin Krizek 05 Mar '12

05 Mar '12

While studying conflicts and repoclosure tests I've done some minor code refactoring that might be worth pushing to master, please review at origin/mkrizek: $ git log origin/master..origin/mkrizek Thanks, Martin

3 4

require maintainer defined for each test & clean up current tests
by Kamil Paral 01 Mar '12

01 Mar '12

Before releasing new version of AutoQA, I want to solve one more issue, and that is the current mess in tests we manage. Honestly stated, we have some tests that we have no idea whether they work, what exactly they do, and how to fix them if needed. Some of them are probably not being used at all, and we still need to care about them when re-factoring our framework. Worst of all, for many of them we have no idea who should be responsible for keeping them in shape. I have spent some time and executed all of them. I also tried to provide my best guess about their current maintainer. This is the result: Test Maintainer Works Currently useful ==== ========== ===== ================ anaconda_checkbot clumens ? no no anaconda_storage clumens ? no no compose_tree ? no no conflicts ? yes unlikely depcheck jskladan ? yes yes helloworld kparal yes yes rats_install hongqing ? yes yes rats_sanity hongqing ? yes probably repoclosure ? yes unlikely rpmguard kparal yes somewhat rpmlint kparal yes somewhat upgradepath kparal yes yes My proposal is: 1. Every test will have a maintainer defined. In its 'control' file we will change AUTHOR line to MAINTAINER line (patch is ready). This will ensure that we always know who to talk to when the test seems broken. It doesn't mean you can't work on a test you don't maintain, no. But at least the contact point will always be defined. I tried to place my best guess in the matrix. Please speak up who wants to maintain some test. 2. Tests without maintainer will be archived and deleted. They can stay in a separate git branch and wait for their future revival (if any), but they won't be in master. 3. To save resources, we should also archive tests that don't seem to be currently much useful. We can re-enable them once required architecture is in place. More specifically: * rpmlint, rpmguard - the results are sent to opted-in maintainers, some of them said it's useful. I'd keep them enabled. * repoclosure, conflicts - this lists potential dependency problems and file conflicts for the whole repository. Until now no one cared. With resultsdb frontend we can finally have a page that lists all the results day by day. It means someone can go through the results occasionally and file some bugs. The question is: who? It's nice to have some results, but if we just *hope* someone will do something about it, that seems too uncertain for me. They are also somewhat obsoleted by depcheck. I'm sitting on the fence here. * compose_tree - this tries to build boot.iso and pxeboot images. This is basically a releng test that we execute. Previous maintainer was James, but I doubt we should ask him for future maintenance. Currently it is broken. Even if it worked, was there someone reporting the issues? It's easier to look into releng logs online: http://kojipkgs.fedoraproject.org/mash/branched-20120227/logs/ Does the above link render compose_tree test useless? * anaconda_* - anaconda build, unit test, and automated installation using various test cases. Great work, but currently broken. I was talking to clumens some time ago and asked whether he received results. He said he did not. I fixed the opt-in emails and told him. No response since, so I guess he just doesn't care. And I'm not surprised. With the speed of anaconda development, it's much easier for anaconda devs to execute the test on their own machines (at least I suppose they do). They can't send us patches all the time. Since there hasn't been any drive from anaconda devs, I propose to obsolete this tests until we have something better to offer. * the rest of the tests stay enabled For the next autoqa release I'd like to focus mainly on easy deployment of tests, so that we (for our tests) and some other teams (like anaconda) can easily update tests without tedious process of "new autoqa release". After that I expect some of the tests to return. What do you think? And please, put your name next to the test you want to maintain.

5 10

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel March 2012