autoqa-devel June 2011

autoqa-devel@lists.fedorahosted.org

12 participants
97 discussions

Improving the testability of AutoQA
by Tim Flink 03 Jun '11

03 Jun '11

As I've been working on the email reduction feature, I've been rather frustrated with how difficult it is to test AutoQA as a framework. I've written scripts to make testing depcheck (minus commenting and build gathering) easier but I have yet to find a reasonable way to exercise the whole AutoQA stack other than running jobs on my dev system with comments disabled and hope that they finish before production does. Before I go farther, I realize that what I'm discussing would be a non-trivial amount of work and I'm kind of thinking that it would be a little too much for 0.5.0. Then again, more testing could help improve the release. Thoughts on that part would be welcome. I can think of two methods that we could use to make testing easier and start down the road of retiring our mascot: - Refactor bodhi_utils and koji_utils so that they can be stubbed out in testing - Find an alternative to the production instances of koji and bodhi Either way, setting up the conditions for self-tests (making mock rpms, setting up the metadata, creating repos) is going to be a decent amount of work and will likely end up with fragile, high maintenance tests if we go beyond basic smoke tests. There were concerns over the refactoring I proposed for koji_utils a while back so I assume that isn't the way we want to proceed for now. I looked into setting up test instances of bodhi and koji but I'm rather intimidated by the amount of maintenance and hacking that would be required to induce the conditions that we would need for more complete testing on a regular basis. With this in mind, I started hacking at a mockup for replacing bodhi and koji for testing purposes that does nothing more than implement the parts of those interfaces that we're using in AutoQA. It took a little while to reverse engineer the interfaces to bodhi and koji but I do have some code that is able to fool the koji client with hard-coded results (haven't tried it with AutoQA yet nor have I gotten very far on mocking up bodhi). I can send out code if anyone wants to see it (kind of ugly ATM, though). While finishing the mockup would also be non-trivial, I think that it will be more manageable and flexible in the long run in addition to being better suited to our needs since we can add in testability features as needed without having to bug other teams about features that other people may never use. Anyhow, thoughts on the concepts? On the timing (try to have some for 0.5.0 or not)? Anything that I didn't consider? Tim

2 2

A puzzle about the ways of adding repos when installing
by Tao Wu 03 Jun '11

03 Jun '11

Hi folks, I'm trying to build a hard drive auto installation script but have a puzzle on the several methods of locating repo. As we know, there are at least 3 ways to locate the iso image: 1. add the boot argument 'askmethod', then choose the location when anaconda prompts, like: https://fedoraproject.org/wiki/QA/TestCases/InstallSourceHardDrive 2. add the boot argument 'repo=hd:<device>:/<path>', like: http://fedoraproject.org/wiki/Anaconda/Options#repo 3. write the repo location into kickstart file, like: https://fedoraproject.org/wiki/Anaconda/Kickstart#install 4. add the repo graphically (this method does not support using iso file), like: https://fedoraproject.org/wiki/QA:Testcase_Ftp_Repository Can someone tell me that are these methods the same to anaconda, and if they are different, which one can be treat as the third one(kickstart method)? Thanks! -- Best Regards, Tao Wu

1 0

A puzzle about the ways of adding repos when installing
by Tao Wu 03 Jun '11

03 Jun '11

1 0

dealing with i686 builds
by Tim Flink 03 Jun '11

03 Jun '11

One of the things that I haven't dealt with yet for email reduction is i686 packages. At the moment, I assume that there are 2 or 3 tests that will be run depending on the koji tag: - depcheck i386 - depcheck x86_64 - upgradepath noarch (not run for updates-testing) These 2 or 3 tests are used to determine the state of an update's tests and that state is used as part of the decision process to send an email or not. I'm trying to figure out the best way to deal with i686 packages and the things that I have come up with so far are: - Ignore them and accept that emails will not be sent for those updates - Interface with bodhi/koji to determine whether an update has i686 builds and adjust the state determination as needed I remember Kamil found a list of i686 packages (but can't seem to find the email at the moment) but I'll find the list of updates that would be affected by not handling i686 builds. Any other thoughts on how to go about doing this or preferences on one over the other? Tim

2 2

Release Readyness
by Tim Flink 02 Jun '11

02 Jun '11

Well, it's been 2 weeks since the last time this question was asked and it's time to do it again: Are we ready for release? My vote is going to be for no again but I'm also wondering if we should re-evaluate next week instead of 2 weeks from now. I think that the email reduction feature is getting close to being done and it sounds like the logging feature is also getting close. Thoughts? Tim

2 1

[AutoQA] #271: Create autotestd systemd.service file for F15 (and beyond)
by fedora-badges 01 Jun '11

01 Jun '11

#271: Create autotestd systemd.service file for F15 (and beyond) -----------------------+---------------------------------------------------- Reporter: jlaska | Owner: Type: task | Status: new Priority: major | Milestone: Package Update Acceptance Test Plan Component: packaging | Keywords: -----------------------+---------------------------------------------------- 07:41:40 Viking-Ice: jlaska: nr.1 upstream man pages nr.2 comparing converting https://fedoraproject.org/wiki/User:Johannbg/QA/Systemd/compatability ( The Fedora autotest-server package includes a sysVinit script. A systemd.service file will be needed for f15 and beyond. Guidance from Viking-Ice included below ... {{{ 07:41:40 Viking-Ice: jlaska: nr.1 upstream man pages nr.2 comparing converting https://fedoraproject.org/wiki/User:Johannbg/QA/Systemd/compatability ( you can side by side view them ) nr.3 scim through Lennart's blog 07:42:36 jlaska: Viking-Ice: perfect, just the bread-crumb trail I needed. Thank you 07:43:04 * jlaska goes to create a ticket with these instructions (need to eventually create an autotestd.service) 07:44:02 Viking-Ice: I deliberately designed the wiki page so those wanting to convert could side by side compare what already had been converted for you know ah.. they solve this problem that way moment.. 07:45:14 <-- juhp [~petersen(a)66.187.239.10] has quit (Ping timeout: 240 seconds) 07:46:51 Viking-Ice: jlaska: throw this link into your pool http://0pointer.de/blog/projects/systemd-for-admins-3.html 07:48:21 Viking-Ice: jlaska: what he mentioned there is the skeleton to be built upon as inn the first example of the service file. Always start like that then gradually build on top of that if needed }}} -- Ticket URL: <https://fedorahosted.org/autoqa/ticket/271> AutoQA <http://autoqa.fedorahosted.org> Automated QA project

1 2

Simple Fix Pushed to Master
by Tim Flink 01 Jun '11

01 Jun '11

I pushed a quick fix to master (same code isn't in stable yet) that was throwing exceptions if an arch wasn't specified for koji (which is the default). https://fedorahosted.org/autoqa/changeset/7a230188ed2926498ceac4a4061e267eb… I can't think of any problems this would cause but figured that I would send out a quick email. Tim

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

autoqa-devel June 2011