#181: Quick and dirty support for Virtualization ----------------------------+----------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Keywords: | ----------------------------+----------------------------------------------- The task is to implement quick and dirty support for virtualization for AutoQA tests. We can already install autotest-client into virt machine, but there are some tests which must not run inside virt environment (tests using virt themselves for example). Therefore we must ensure all our tests are marked properly ("may use virt", "may not use virt"), all our tests clients are marked properly ("is virt", "is bare metal"), and this information are propagated from AutoQA to autotest-server and used properly when selecting appropriate host and scheduling jobs.
This could be quite easy, there is some support for tags in autotest already.
As an enhancement, we may think about preferences. Do we want to mark some test that it is preferred to run it in virt, but may also be run on bare metal? How do we achieve correct behavior for autotest-server for such task? For example: We may want to prefer executing sanity tests in virt env, but it's not really mandatory, just desired. Food for thoughts.
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Comment (by kparal):
Ah, and a good remark - when creating all the tags, we should clearly distinguish the tag "needs hardware with virt support" from tag "needs to be run inside virt system". The same goes for "this machine supports virt systems" and "this machine is a virt system". Those are really different things, so the tags should not be ambiguous. And we may need both.
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Comment (by jlaska):
Replying to [comment:1 kparal]:
Ah, and a good remark - when creating all the tags, we should clearly
distinguish the tag "needs hardware with virt support" from tag "needs to be run inside virt system". The same goes for "this machine supports virt systems" and "this machine is a virt system". Those are really different things, so the tags should not be ambiguous. And we may need both.
Good point. I read this as we have machine tags that indicate what the system is capable of, and then we'll have tests that require all (or a subset) of tags when selecting an applicable test client to use.
Right now we have the following tags (autotest calls them labels): * platform tags (architecture) * i386 * x86_64 * ppc * release tags (unused) * f11 * f12 * f13 * hardware feature tags (unused) * kvm (aka vmx) - indicates VT support for virt guests * hypervisor - indicates if it's a virt system
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Comment (by jlaska):
I couldn't help but continue playing with this setup. Using the script provided in comment#1, along with 2 patches to dracut (thanks to haraldh for guidance), I have a working setup that uses virtualization and LVM snapshots to provide disposable test systems. The patches are out for review with dracut and can be tracked in bugzilla (see [https://bugzilla.redhat.com/show_bug.cgi?id=602649 bug#602649] and [https://bugzilla.redhat.com/show_bug.cgi?id=602784 bug#602784]).
With this setup, you just reboot the test system when finished testing (this is the default behavior with autotest). During boot, the system creates and uses new LVM snapshot for the '/' volume on boot. Any changes previously made to the disk are gone and the system is restored to the initial state. If this works, and is accepted, I believe this obsoletes the need for ticket#183 and ticket#185.
Comments/questions/flame encouraged!
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Comment (by jskladan):
This looks awesome James! My only concern is this: are there any tests which require reboot during the testrun? (Are these tests even possible to write? this is maybe more of a "i'm curious about it" type of question, than a concern :) ) Because even thought this is a quick'n'dirty temporary solution, my experience tells me, that temporary solutions tend to last :-D
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Comment (by kparal):
Jskladan is right, we need to have the possibility to reboot the machine and continue executing the test sometimes. For example for package sanity test we may need to do "yum update" and reboot first if the machine is not fully up-to-date from regular maintenance (some updates were pushed in the meantime) and only then continue executing the test.
We might still use it for other tests, but I think some kind of signaling to the host machine will be necessary in the end.
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Comment (by jlaska):
Good thinking gents, keep the ideas coming. I'd like to poke holes in this solution to see how well it stands up and if it's good enough.
With regards to tests that require reboot... in the current dracut+LVM implementation, for any tests that need to reboot, we would just need to remove the boot argument ''rd_LVM_SNAPSHOT''. That will allow dracut to continue using the existing LVM snapshot. When the test is complete, just add the ''rd_LVM_SNAPSHOT'' boot argument back to grub.conf.
As an aside, I'm sure it's just a matter of time for tests that require rebooting. However, I'm not sure how that works yet with autotest.
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: kparal Type: task | Status: new Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Changes (by kparal):
* owner: => kparal
Comment:
(This comment concerns the original ticket description.)
I'm in time press, so just shortly: I've played with autotest-server and it seems that "atest job create -b labels" behaves exactly like we need it - it looks for hosts that satisfy all the labels at once.
{{{ -b LABELS, --labels=LABELS Comma separated list of labels to get machine list from. }}}
There are also some dependencies, I haven't still managed to explore what they are good for:
{{{ -d DEPENDENCIES, --dependencies=DEPENDENCIES Comma separated list of labels this job is dependent on. }}}
So, when I'm back from PTO I will patch autoqa to transfer required labels to autotest-server. We will also need some standards how individual labels are defined for particular tests. Also we should make a list of labels we use and document them on the wiki. I believe I can do it in a few days and then we will have support for unsafe tests to be executed inside VMs, isn't that cool? :) Looking forward for autotest-server testing instance, where we can try out all this stuff.
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: kparal Type: task | Status: assigned Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Changes (by kparal):
* status: new => assigned
Comment:
Initial (and hopefully working) code was pushed as 'labels' branch, read more at[[BR]] https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000742.html
I have documented the new labels, read more at[[BR]] https://fedorahosted.org/pipermail/autoqa-devel/2010-June/000740.html
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: kparal Type: task | Status: assigned Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: | Keywords: -----------------------------+---------------------------------------------- Comment (by kparal):
Patch sent to list: https://fedorahosted.org/pipermail/autoqa-devel/2010-July/000820.html
#181: Quick and dirty support for Virtualization -----------------------------+---------------------------------------------- Reporter: kparal | Owner: kparal Type: task | Status: closed Priority: major | Milestone: Virtualization Component: infrastructure | Version: 1.0 Resolution: fixed | Keywords: -----------------------------+---------------------------------------------- Changes (by kparal):
* status: assigned => closed * resolution: => fixed
Comment:
This has been implemented when my control.autoqa patchset was merged to master: http://git.fedorahosted.org/git/?p=autoqa.git;a=commit;h=d7f91c3e89f173b1b92...
We can now specify that test needs virt environment and correct machine is chosen.
autoqa-devel@lists.fedorahosted.org