Upgrade Path
by Josef Skladanka
Hey, gang!
Yesterday, Kamil, Vojta and me were discussing some Upgrade Path issues,
since we're not really sure, what are the problematic situations we need
to catch. But from the discussion below the #123 we have come to this
problem we're not really sure about:
There is a package Foo in Fedora-Updates repository. Lets say:
[updates]
F13: foo-1.1
F12: foo-1.0.3
F11: foo-1.0
And the maintainer/developer/whoever decides to deliver new version
foo-2.0 in all of these, so, he adds foo-1.2 into the all respective
Updates-Testing repos.
[updates-testing]
F13: foo-1.2
F12: foo-1.2
F11: foo-1.2
// Note, that updates-testing is not enabled by default.
Now, let's imagine a user on F11, who has Updates-testing repository
enabled, and installs foo-1.2. Then he disables the testing repo, and
decides to upgrade to F12 (which has only access to foo-1.0.3).
Is this still breaking the upgrade path, or not? From our point of view,
it should not.
The thing is, that foo-1.2 needs to stay in bodhi for certain amount of
time (lets say a week), before it gets pushed from updates-testing to
updates. If the before mentioned scheme broke the upgrade path, then the
workflow would need to look like this:
(day 0): add foo-1.2 to F13 updates-testing
(day 1-6): wait in koji
(day 7): push from F13 updates-testing to F13 updates
(day 7): add foo-1.2 to F12 updates-testing
...
You can see the terrible delay... And yes, i know that karma can make it
push faster, but still, you need to work sequentialy for all the
releases, which does not seem to be OK.
Joza
13 years, 9 months
[AutoQA] #115: Update post-tree-compose to recognize new stage2 URL
by fedora-badges
#115: Update post-tree-compose to recognize new stage2 URL
-------------------------+--------------------------------------------------
Reporter: jlaska | Owner:
Type: enhancement | Status: new
Priority: major | Milestone: Automate installation test plan
Component: watchers | Version: 1.0
Keywords: |
-------------------------+--------------------------------------------------
During Fedora 13, release engineering will provide schedule drops of
rawhide for automated testing against rats_install. The provided data
will include a compose tree (without packages). For proper automated
testing of these images, post-tree-compose will need to monitor a new URL
for testing (likely candidate is
http://alt.fedoraproject.org/pub/alt/stage/rawhide-testing).
In addition, rats_install will need to support installing with stage2 and
the package repo in different locations. That will be tracked in another
ticket.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/115>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 9 months
Re: [PATCH 4/4] Modified templates for all hooks for new autoqa script
by Will Woods
----- "Vojtěch Aschenbrenner" <vaschenb(a)redhat.com> wrote:
> -# post-bodhi-update tests can expect the following variables from
> autoqa:
> -# title: title of the update request
> -# updateid: Unique ID of the update request (may be empty)
> -# targettag: target koji tag for this update
> -# envrs: comma-separated list of package envrs in this update
> request
Removing the argument list from the template is probably not a good
idea, since the templates are a primary source of documentation for
the test hook.
It'd be better to reword the comments to something like:
"post-bodhi-update tests can expect the following keys to be present
in the autoqa_args dict."
-w
13 years, 9 months
[AutoQA] #198: complete DVD auto install test driver
by fedora-badges
#198: complete DVD auto install test driver
-------------------+--------------------------------------------------------
Reporter: liam | Owner: liam
Type: task | Status: new
Priority: major | Milestone: Automate installation test plan
Component: tests | Version: 1.0
Keywords: |
-------------------+--------------------------------------------------------
We are developing dvd auto install test driver:
http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/anaconda/dvd...
to meet our auto install requirement:
https://fedoraproject.org/wiki/Is_anaconda_broken_roadmap
The tests can run like:
PYTHONPATH=../../lib/python/ python dvd_install.py -i
/path/to/Fedora-13-x86_64-DVD.iso -k /path/to/ks.cfg
Issues we are facing:
1. stage2 to located install.img may can not be achieved in this version,
more details see:
https://bugzilla.redhat.com/show_bug.cgi?id=609379
2. the minimon can not get logs when provide a local ks file and the
network is not enabled in ks file. In this case, the network would not
active during install. the minimon should configure the IP address for
guest when start to run in guest and did not find IP?
3. Optimize the lib code in anaconda.py, to inherit the class VirtGuest
and extend some methods like Will Woods said last time. Some methods may
need to rewrite when write cd_insall.py
4. In order to support some install like: updates, graphic/test/vnc,
ksdevice, ip/dns/netmask/gateway, we may need to add some options to
support them.But this also may confuse testers. So I only provide a "-x"
option to support them. If tester wants to support them, can run like:
PYTHONPATH=../../lib/python/ python dvd_install.py -i
/path/to/Fedora-13-x86_64-DVD.iso -k /path/to/ks.cfg -x "serial
console=tty0 console=ttyS ksdevice=eth0 ip=192.168.1.2
netmask=255.255.255.0 updates=http://url/to/updates".
Make sense?
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/198>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 9 months
[AutoQA] #129: create auto install test script which maps 1x1 to the install method.
by fedora-badges
#129: create auto install test script which maps 1x1 to the install method.
-------------------+--------------------------------------------------------
Reporter: liam | Owner:
Type: task | Status: new
Priority: major | Milestone: Automate installation test plan
Component: tests | Version: 1.0
Keywords: |
-------------------+--------------------------------------------------------
Each install method should have an install test script, For example:
* Single ISO install - dvd_install.py
* Multi ISO install - cd_install.py
* Hard Drive ISO install - hdiso_install.py
* NFS install - nfs_install.py
* NFS ISO install - nfsiso_install.py
* HTTP/FTP install - url_install.py <-- similar to
rats_install.py now
we need to create these scripts, but each of the script should share code
as much as possible.
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/129>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
13 years, 9 months
Re: [PATCH] add helloworld test
by Kamil Paral
----- "Vojtěch Aschenbrenner" <vaschenb(a)redhat.com> wrote:
> On Wed, Jul 14, 2010 at 05:41:29PM -0400, Will Woods wrote:
> > job.run_test('helloworld', config=autoqa_conf, **locals())
> >
> > (doing **locals() passes the contents of locals as if they had all
> been
> > given to the function as "var_name=var_value" pairs)
>
> This is problem, because locals() will pass many parameters (not only
> ours), output included at the end of mail (sorry for LONGLONG mail).
>
> > Probably we want *args as well as **kwargs here, just in case.
>
> OK, *args included now, but here is problem with this mess in
> arguments
> from locals().
I didn't occur to me that there will be some many local variables.
I think we one several choices:
== Temporary solutions ==
1. Leave helloworld available just for post-koji-build
2. Test the presence of all variables (defined for any hook) in
helloworld control file and pass to the job only those that are
currently available. This means more work when maintaining this test.
== Proper solutions ==
3. Pass all autoqa variables into the control file as a dictionary,
instead of a pile of variables. We have talked about this some time
ago. It will need a modification of all the test cases and some
tweaks in autoqa harness. But overall it shouldn't be such a hard
change. Vojtěch could do it easily I believe.
What do you think?
13 years, 9 months
aqc-create.py: install autoqa clients easily
by Kamil Paral
Hello,
I have created a simple python script to simplify installation of autoqa
clients into virtual machines. As our autoqa efforts progress (setting
up staging servers etc) it is quite common task to do, so this script will
save you a lot of manual labor.
Script is called "aqc-create.py" ("_Auto_QA _client - create") and is available
in 'aqc' branch in utils/ directory in our git:
http://git.fedorahosted.org/git/?p=autoqa.git;a=tree;f=utils;hb=aqc
Usage is simple:
$ aqc-create.py -n Example
will create Fedora 13 virtual machine with <your-arch> arch, 512MB RAM and
storage /dev/vg_autoqa/Example.
$ aqc-create.py -n Example2 -f 12 -a i386 -r 1024 --disk /tmp/Example2.img -v -d
will create Fedora 12 machine with i386 arch, 1024MB RAM and storage
/tmp/Example2.img. It will be verbose and it will not attach you to the serial
console of the new machine, so the installation will run on background.
More options are available, see --help.
The installation is fully automatized and it uses kickstart generated by simple
wsgi application available in utils/aqc-ks/ directory. It is currently deployed
at our staging server:
* help usage:
http://test1185.test.redhat.com/wsgi/aqc-ks/aqc-ks
* example usage:
http://test1185.test.redhat.com/wsgi/aqc-ks/aqc-ks?release=13&arch=x86_64
(This is just an implementation detail, the script uses it automatically and you
don't have to care about it.)
aqc-create.py will do these things for you:
* call virt-install with correct options (kickstart path, serial console
redirection, boot kernel location, etc)
* generate a kickstart
* add jlaska's autoqa repo
* install autotest and autoqa packages and disable their cron tasks
* add autotest-server's ssh certificate to /root/.ssh/authorized_keys
(currently the staging server's certificate is added by default)
The client is fully prepared automatically and all you have to do at the
autotest-server is call these commands:
$ ssh root@<hostname> 'true' # just to add the fingerprint to known_hosts
$ atest host create -t x86_64 -b fc13,virt <hostname> # change labels if needed
(If you have ideas how to get rid of the first line, comments are welcome.)
So, please comment and report problems, but I have used it many times in the last
few days and it seems to be working really well. Maybe we can even use it in our
tutorial wiki pages, it may help beginners to start with AutoQA. I'll try to
document it later.
If you think this script is valuable, I would like to integrate it into our master
branch. Do you think the name (aqc-create/aqc-ks) and the location (utils/) is fine,
or do you have other suggestions? Or maybe we want to have it stored somewhere else
than in autoqa source code? Let me know.
Kamil
13 years, 9 months
Re: [PATCH] add autotest labels support
by Kamil Paral
----- "Will Woods" <wwoods(a)redhat.com> wrote:
> So. I think the sensible thing to do might be to have a new file
> where
> we can store/override test metadata - things like labels and hooks.
> Yes,
> this means another file for each test (boo! more files!) but if we're
> smart about the file format/syntax we won't need to add any others. I
> hope.
Ok, you bought me in. I have meditated over all the possible stuff we
could use the new file for and this is came out of my fantasies:
control.autoqa:
=============================================
# This file can expect some variables pre-defined:
# hook: name of the hook to run this test
# archs: list of host archs to run this test on
# execute: whether to execute this test at all, by default True
# all test arguments (envr, name, kojitag, etc)
# 1. Metadata definitions
# a) Autotest labels needed for this test to run
labels = ['virt']
labels.append(envr.split('.')[-1]) # distro part of envr (like 'fc13')
# b) Host architectures override
# Some tests may need to override the default set of pre-selected
# host architectures. That is logical, because the watcher can't know
# all the specifics of a certain test, so the architecture selection
# may not be optimal. For example some tests need to be executed only
# once on an arbitrary architecture, even though the packages are built
# for several architectures. Rpmlint as an example of such a test.
# These tests can override the default selection here.
archs = ['noarch'] # this is rpmlint, we will test all archs at once,
# no need to run it on several hosts
# 2. Decision whether to run this test at all
# This is a place where a test may decide whether it should be run
# under specified conditions, or just skipped. There might be many
# reasons why to skip this test:
# - it doesn't make sense to execute this test under current hook
# - the test doesn't support current combination of specified
# arguments
# - combination of the above (it doesn't make sense to run this test
# under this hook with these arguments)
# If you don't want to run this test, just set 'execute' to False
# a) Hook decision
supported_hooks = ['post-koji-build', 'post-bodhi-build']
if hook not in supported_hooks:
execute = False
# b) Arguments decision
# this is an initscripts test, we support only a few packages where
# initscripts should be tested. it doesn't make sense to run this
# test for all other packages. therefore if we don't have a testfile
# for this package in our tests/ subdirectory, just skip this test
if name not in os.listdir('tests'):
# not supported
execute = False
============================================= (EOF)
<implementation>
In the autoqa harness we would just evaluate such control.autoqa file
for every test available and then look at some output variables
(execute, archs, labels) and schedule jobs accordingly. The watchers
would no longer pass testlist option to autoqa, that would be resolved
inside autoqa script.
</implementation>
It's good to mention:
* That was just an example file. Real files may be much simpler or
much more complicated.
* I have tried to come up with all the important stuff we might need,
but it's certain we will discover further metadata/decision items
in the future.
And some questions:
1. Are we able to do all this with just a template/config file?
Personally the security risk of scripting language seems negligible
to me and defining this stuff with config files seems overcomplicated
to me (and some parts would have to be left out completely).
2. This new file seems to solve several of our issues at once, I think
it's quite neat (self-applause! booo!) :) But is also seems like a
major architecture overhaul. Do we want to do it right now?
I'll gladly volunteer, but first we must decide - do we want to
concentrate purely on depcheck/other PUATP stuff right now and just
add a quick dirty labels support (with plans of doing it properly
in the future), or is this worth doing properly right now?
>
> Ah, true. But if you're iterating over the keys of a dict, you can
> do:
> for key in vars.keys():
> rather than copying the entire dict.
Err, I thought it returns an iterator, but it returns a standard
list. One is still learning, isn't he? :) Patched, thanks.
13 years, 9 months
Re: [PATCH] add autotest labels support
by Kamil Paral
----- "Will Woods" <wwoods(a)redhat.com> wrote:
> On Wed, 2010-07-14 at 05:23 -0400, Kamil Paral wrote:
>
> > Personally the security risk of scripting language seems negligible
> > to me and defining this stuff with config files seems
> overcomplicated
> > to me (and some parts would have to be left out completely).
>
> I'm inclined to agree. Let's go with actual Python code, but try to
> keep
> it simple - so test authors don't have to depend on Super-Clever QA
> Boffins like us just to get a test working (heh).
>
> > 2. This new file seems to solve several of our issues at once, I
> think
> > it's quite neat (self-applause! booo!) :) But is also seems like a
> > major architecture overhaul. Do we want to do it right now?
> >
> > I'll gladly volunteer, but first we must decide - do we want to
> > concentrate purely on depcheck/other PUATP stuff right now and just
>
> > add a quick dirty labels support (with plans of doing it properly
> > in the future), or is this worth doing properly right now?
>
> I like it, and I think it's worth doing this properly right now - I
> think we had identified this as one of the prerequisites for the
> depcheck/PUATP work anyway.
>
> (Also, I think we have some time in the schedule to do foundation
> work,
> since Alpha isn't until September 21 or so.)
>
> The thing I wonder about is this: if we're going to exec the
> 'control.autoqa' file as-is for each test, don't we risk having each
> file stomp on the variables set by the previous one? It seems like
> we'll
> end up needing to maintain different environment/scope/namespace for
> each file. So maybe we want to be able to import them as normal
> python
> modules, rather than manipulating the global/local namespace
> ourselves?
Oh, that shouldn't be a problem (at least if I understand your concerns
properly), exec statement supports execution in a custom
(dictionary handled) namespace:
http://docs.python.org/reference/simple_stmts.html#the-exec-statement
I already use it in the provided patch (autoqa, line 106). So we can
have a new dictionary for every test we want to evaluate.
>
> We already import the 'hook.py' for each hook, so there's some
> precedent
> there, but I'd be happy with whichever way you think is cleanest.
>
> And thanks for volunteering - I'm looking forward to seeing how it
> works!
I will try to provide some sample patch and we can fine-tune the details
then.
>
> -w
>
> _______________________________________________
> autoqa-devel mailing list
> autoqa-devel(a)lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/autoqa-devel
13 years, 9 months
