autoqa-devel March 2012

autoqa-devel@lists.fedorahosted.org

5 participants
36 discussions

[AutoQA] #251: Move autotest applications into /usr/*bin
by fedora-badges 26 Sep '12

26 Sep '12

#251: Move autotest applications into /usr/*bin -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: minor | Milestone: Packaging, Review, & Deployment Component: packaging | Keywords: -----------------------+---------------------------------------------------- The autotest package should supply binaries in /usr/bin/ ... so we can rely on PATH and not have to call autotest with the full-path (/usr/share/autotest/client/bin/autotest) when scheduling jobs in '/usr/bin/autoqa'. Alternatively, the autotest package could drop a script that updates PATH appropriately into /etc/profile.d/autotest. But yuck. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/251> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #191: Submit autotest for Fedora package approval
by fedora-badges 26 Sep '12

26 Sep '12

#191: Submit autotest for Fedora package approval -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Packaging and Review Component: packaging | Version: 1.0 Keywords: | -----------------------+---------------------------------------------------- * https://bugzilla.redhat.com/show_bug.cgi?id=548522 Currently stalled, pending packaging of the gwt package. This includes packaging all bundled .jar files included with gwt. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/191> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #414: rats_install: run twice, once for updates, once for updates+updates-testing
by fedora-badges 14 Aug '12

14 Aug '12

#414: rats_install: run twice, once for updates, once for updates+updates-testing --------------------------+------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Nice to have soon Component: tests | Keywords: Blocked By: | Blocking: --------------------------+------------------------------- We decided it would be beneficial to run rats_install twice. Once for stable updates only, once for main+updates, once for main+updates+updates- testing. Currently we're running just the second option. The discussion is here: https://fedoraproject.org/wiki/QA/Meetings/20120319#t15:55:42 Change rats_install to execute twice and report both variants. The easiest way is probably to modify 'control' file to execute the job twice and modify some input argument to distinguish the two cases. Also we need to change the summary accordingly so that we can easily distinguish that in our ResultsDB view. Marking "Nice to have soon" if we can do it before F17 Final, otherwise it'll probably go into "Future tasks". -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/414> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 10

[AutoQA] #157: conflicts: requested arch not respected
by fedora-badges 06 Aug '12

06 Aug '12

#157: conflicts: requested arch not respected --------------------+------------------------------------------------------- Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Component: tests | Version: 1.0 Keywords: | --------------------+------------------------------------------------------- This is a bug in the potential_conflict.py script in tests/conflicts/. Usage suggests using --arch options for specifying requested architecture to be tested. But if you look at the attached log, the first and second output match (that's correct), but the third doesn't match, even though it should. Everything was tested on x86_64 machine. Also from the usage line it is not clear whether --arch is related only to particular yum repository selection (replacing $basearch keyword), or whether it also relates to packages built for one architecture but being in different repo (e.g. i686 packages are very often in x86_64 repo). I have looked into the source code of the script and it seems that the --arch option is not used at all, just printed out in the usage help?? Seth, you're listed as script author. Any comments on that? Thanks. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/157> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 7

[AutoQA] #270: Package and submit for review - compat-Django-1.0.4
by fedora-badges 02 Aug '12

02 Aug '12

#270: Package and submit for review - compat-Django-1.0.4 -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: packaging | Keywords: -----------------------+---------------------------------------------------- Currently packaged so that it can coexist with the current Django package (see http://repos.fedorapeople.org/repos/fedora- qa/autoqa/fedora-14-testing/). I need to port existing security patches linked from http://www.djangoproject.com/weblog/2010/dec/22/security/ Once patched, submit for review ... and enjoy. Eventually autotest will be updated to use Django-1.2 ... but packaging Django-1.0.4 removes one more packaging obstacle to having autotest as an official Fedora package. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/270> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

[AutoQA] #329: implement RPM checksum verification
by fedora-badges 30 Jul '12

30 Jul '12

#329: implement RPM checksum verification -------------------------+-------------------------------------------------- Reporter: kparal | Owner: Type: enhancement | Status: new Priority: major | Milestone: Finger Food Component: core | Keywords: -------------------------+-------------------------------------------------- Currently this method is in autoqa.util module: {{{ 261 def valid_rpm(rpm_file): 262 '''Check that RPM file is valid. 263 Note: This currently only extracts RPM header and presumes the file is 264 OK if that succeeds. Full RPM hashsum check would be nice to have 265 implemented too (FIXME). 266 @param rpm_file RPM file path 267 @return True if file seems OK or False if there is some problem (file 268 can't be read, RPM header is corrupt) 269 ''' ... }}} We would like to have full RPM checksum verification, not just reading the header and saying that it's OK. Find out how to do it and change the method accordingly. It would be nice to have a method argument to enable full checksum verification/fall back to original header verification, because it can be performance intensive operation and it may not be suitable for all our use cases). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/329> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #416: Make the code pylint friendly
by fedora-badges 30 Jul '12

30 Jul '12

#416: Make the code pylint friendly --------------------------+------------------------- Reporter: mkrizek | Owner: Type: enhancement | Status: new Priority: trivial | Milestone: Finger Food Component: core | Keywords: Blocked By: | Blocking: --------------------------+------------------------- The task here is to go through the code and fix whatever [http://pypi.python.org/pypi/pylint pylint] complaints about. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/416> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 3

[AutoQA] #396: bodhi_util: Could not connect to bodhi!
by fedora-badges 02 Jul '12

02 Jul '12

#396: bodhi_util: Could not connect to bodhi! ---------------------+------------------------ Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: ---------------------+------------------------ There seems to be some problem when trying to submit certain results to Bodhi: {{{ 12/22 07:10:13 INFO | test:0478| Submitting results (log, email, bodhi) for: depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16 12/22 07:10:13 INFO | test:0361| Log created: /usr/share/autotest/results/default/depcheck/results/kde- l10n-4.7.4-1.fc1.html 12/22 07:10:13 INFO | test:0307| Email not sent: "depcheck: PASSED; 66 PASSED for kde- l10n-4.7.4-1.fc16,PyKDE4-4.7.4-1.fc16,blinken-4.7.4-1.fc16,cantor-4.7.4-1.fc16,gwenview-4.7.4-1.fc16,kalgebra-4.7.4-1.fc16,kalzium-4.7.4-1.fc16,kamera-4.7.4-1.fc16,kanagram-4.7.4-1.fc16,kate-4.7.4-1.fc16,kbruch-4.7.4-1.fc16,kcolorchooser-4.7.4-1.fc16 ,kde- wallpapers-4.7.4-1.fc16,kdeaccessibility-4.7.4-1.fc16,kdeadmin-4.7.4-1.fc16,kdeartwork-4.7.4-1.fc16,kdebase-4.7.4-2.fc16 ,kdebase-runtime-4.7.4-2.fc16,kdebase- workspace-4.7.4-5.fc16,kdeedu-4.7.4-1.fc16,kdegames-4.7.4-1.fc16,kdegraphics-4.7.4-1.fc16 ,kdegraphics-strigi-analyzer-4.7.4-1.fc16,kdegraphics- thumbnailers-4.7.4-1.fc16,kdelibs-4.7.4-1.fc16,kdemultimedia-4.7.4-2.fc16,kdenetwork-4.7.4-1.fc16,kdepim-4.7.4-1.fc16 ,kdepim-runtime-4.7.4-1.fc16,kdepimlibs-4.7.4-2.fc16,kdeplasma- addons-4.7.4-1.fc16,kdesdk-4.7.4-1.fc16,kdetoys-4.7.4-1.fc16,kdeutils-4.7.4-1.fc16,kgamma-4.7.4-1.fc16,kgeography-4.7.4-1.fc16,khangman-4.7.4-1.fc16,kig-4.7.4-1.fc16,kiten-4.7.4-1.fc16,klettres-4.7.4-1.fc16,kmplot-4.7.4-1.fc16,kolourpaint-4.7.4-1.fc16,konsole-4.7.4-1.fc16 ,kross- interpreters-4.7.4-1.fc16,kruler-4.7.4-1.fc16,ksaneplugin-4.7.4-1.fc16,ksnapshot-4.7.4-1.fc16,kstars-4.7.4-1.fc16,ktouch-4.7.4-1.fc16,kturtle-4.7.4-1.fc16,kwordquiz-4.7.4-1.fc16,libkdcraw-4.7.4-1.fc16,libkdeedu-4.7.4-1.fc16,libkexiv2-4.7.4-1.fc16,libkipi-4.7.4-1.fc16,libksane-4.7.4-1.fc16,marble-4.7.4-1.fc16,okular-4.7.4-1.fc16 ,oxygen-icon- theme-4.7.4-1.fc16,parley-4.7.4-1.fc16,rocs-4.7.4-1.fc16,smokegen-4.7.4-1.fc16,smokekde-4.7.4-1.fc16,smokeqt-4.7.4-1.fc16,step-4.7.4-1.fc16,svgpart-4.7.4-1.fc16" 12/22 07:10:17 ERROR|bodhi_util:0278| An error occured: ServerError(http://aqd/bodhi/list, 500, Internal Server Error) 12/22 07:10:17 ERROR|bodhi_util:0279| Could not connect to bodhi! 12/22 07:10:17 ERROR|bodhi_util:0279| Could not post a comment to bodhi }}} This is running on my development machine when using mock_fedorainfra tool. I haven't spotted it yet on either production or staging machine. That indicates that this could be a problem inside the mock_fedorainfra tool. I have seen this bug happening several times, always only for updates consisting of several builds. Investigate whether this is really a problem in mock_fedorainfra and then act on accordingly (fix the problem/report in upstream). -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/396> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 5

Playing around with Sikuli
by Tim Flink 05 Jun '12

05 Jun '12

I've been wanting to try out Sikuli (http://www.sikuli.org) for a while now and finally got around to making it build and work on Fedora. The basic concept is to create GUI test cases based on images instead of absolute (x,y) coordinates (like selenium), scripted key sequences (openqa) or using an accessability layer (like dogtail). I've put together a tarball with source and build scripts if anyone else wants to try it out: http://tflink.fedorapeople.org/tools/sikuli/README_build-sikuli.txt http://tflink.fedorapeople.org/tools/sikuli/fedora-sikuli-20120220.tar.gz The distribution method is a really dirty hack but I've been able to build on both F15 and F16 after installing a huge number of -devel dependencies and building all of Sikuli's direct dependencies from source. Xpresser (https://wiki.ubuntu.com/Xpresser) might be another option - it's a re-implementation of the concepts from Sikuli. It seems to be missing some of Sikuli's features but would likely be quite a bit easier to build and integrate. One of the things I have in mind is to attempt automating at least some of the installation test cases because a lot of them pass most of the time and they can be a bit mind-numbing to do over and over again. I'm of the opinion that our human resources could be put to better use than monkey-button-pushing their way through simple test cases. A non-trivial amount of work would be needed before we could fully utilize this (fixing the code so that it is package-able, better integration with VNC etc.) but I'm wondering if it could be a good candidate for automating some of the installation test cases. I've written two test cases for Sikuli thus far: a simple DVD graphical install and graphical firstboot. I had to workaround some timing issues due to occasional lag in the VM's VNC interface but they seem to be working well with F17 Alpha RC2 for now. F17 DVD Basic Graphical Install: http://tflink.fedorapeople.org/tools/sikuli/f17-basicGraphicalDVD_sikuli.ta… F17 Graphical Firstboot: http://tflink.fedorapeople.org/tools/sikuli/f17-graphicalFirstboot_sikuli.t… - NOTE: This only works some of the time due to rhbz#750527 which was closed with no bug comment by the developers. I'm going to bug them about this but will likely update the test case to work with unpredictable order of the firstboot screens. To run those test cases, extract them somewhere and point the Sikuli IDE at the extracted directories. I'm going to be running my basic graphical install testcase for different F17 releases to get an initial feel for how fragile the test cases are and how much effort would be required to maintain these kinds of test cases over a release. Anyhow, long email over. Any thoughts on whether this might be worth investigating farther? Tim

3 4

[AutoQA] #415: results not being reported to bodhi (Expectation Error)
by fedora-badges 11 May '12

11 May '12

#415: results not being reported to bodhi (Expectation Error) -----------------------+------------------------ Reporter: tflink | Owner: Type: defect | Status: new Priority: critical | Milestone: Hot issues Component: core | Keywords: Blocked By: | Blocking: -----------------------+------------------------ I was just pinged regarding the lack of a depcheck run for a bodhi update - https://admin.fedoraproject.org/updates/abattis-cantarell- fonts-0.0.8-1.fc17 After looking into the logs: - http://autoqa.fedoraproject.org/results/315536-autotest/virt01.qa/debug/cli… we're getting errors when attempting to make the bodhi comment to report the results: {{{ 03/29 14:16:37 ERROR|bodhi_util:0249| An error occured: ServerError(https://admin.fedoraproject.org/updates/list, 417, Expectation Failed) 03/29 14:16:37 ERROR|bodhi_util:0250| Could not connect to bodhi! 03/29 14:16:37 ERROR|bodhi_util:0250| Could not post a comment to bodhi }}} I'm not sure what's going on here but will investigate and update here as I learn more. -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/415> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 6

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel March 2012