Beaker-devel October 2013

beaker-devel@lists.fedorahosted.org

7 participants
17 discussions

Adding resource management to beaker
by Bill Peck 21 Oct '14

21 Oct '14

Hello Beaker, One of the jobs we run uses shared storage. For example: HostA with Driver1 HostB with Driver2 HostC with Driver3 All connected to StorageX. We can't run the test on more than one host at a time since it would conflict. Being able to specify the following in the job xml is the goal: <recipeSet> <recipe/>  <resource/>  </recipeSet> <recipeSet> <recipe/>  <resource/>  </recipeSet> <recipeSet> <recipe/>  <resource/>  </recipeSet> I know there is a resource type for recipe but this doesn't seem like a good solution. I did try an experiment where I created StorageX host in beaker with no power control and scheduled three recipesets like above. But this hack won't work because a watchdog with a NULL expire time is set and that means it will sit there forever. Even if it did work, maybe if we created a dummy power type, we would still end up with an abort. I don't think it would take much work to support this type of multi-host/resource requirement. Right now we have the following model Object Recipe, which has the following attributes: distro_tree, resource, rendered_kickstart, watchdog, systems, dyn_systems, tasks, dyn_tasks, tags, repos, rpms, logs, custom_packages, ks_appends Object MachineRecipe which inherits from Recipe, and adds a guests attribute. Object GuestRecipe which inherits from Recipe. A ResourceRecipe object wouldn't need most of these attributes, probably just the following: resource, systems, and dyn_systems. (no watchdog, just return when recipeSet completes or aborts) And resource currently maps to a RecipeResource which can be one of the following: SystemResource VirtResource GuestResource I think adding another Object here would make sense, but the term Resource is overused and its a little confusing right now. If done correctly we could re-use the filtering code in needproperty, but I think the only things we could search on for resources would be the following: Name, lab_controller, and key_values? (storing how these shared resources are connected would be another option I suppose, but maybe too complicated for a first implementation?) I'm sure there are other things to consider, we could support setting up and tearing down shared resources (almost like a power on/off command). I want to be mindful of things like that but I don't want to be paralysed with what if's and never get anything done. I know you guys have been working hard on the groups features and want to re-do the scheduler as well. Maybe this would be something for after the scheduler re-write? I don't see any mention of this on the roadmap [1] (although the roadmap is quite full already!). In any event, I'm interested in hearing your ideas and feedback. Thanks! 1 - http://beaker-project.org/dev/tech-roadmap.html

2 2

Docs/overview for using autotest in Beaker?
by Nick Coghlan 05 Dec '13

05 Dec '13

Hi Don, Bill, LMR, I saw that the pull request for Beaker harness API support in autotest has been merged. I'd like to link to that feature from beaker-project.org, so I was wondering if there was anything published anywhere (even a blog or mailing list post) that covered the details of: 1. What to add to a job/recipe definition to specify autotest as the harness 2. What, if anything, to add to the task list in the recipe to configure autotest appropriately 3. What, if anything, to include in an autotest job to make it suitable for running inside Beaker 4. How the result reporting works when using autotest as a harness inside Beaker (Do the results just go to Beaker? Or does normal autotest reporting still happen as well?) If there isn't, we'd be happy to accept a patch on Gerrit to add an autotest section to http://beaker-project.org/docs/alternative-harnesses/ :) Somewhat related, is there a tentative date yet for the next autotest release? Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)

5 20

New sub-command to list access policy for a system
by Amit Saha 01 Dec '13

01 Dec '13

I noticed that there is no sub-command to list the existing policy for a system, so tried writing one [1]. My primary objective is to of course having it as a sub-command, unless there was another reason other than time that it wasn't added earlier. The second objective is to use it as the second example in our developer guide to show how you expose controller methods and write a client to access it. Example usage: # bkr policy-list --system beaker-test-vm1 --hub http://localhost/bkr --username admin --password foobar { "rules": [ { "everybody": true, "permission": "control_system", "group": null, "id": 1, "user": null }, { "everybody": true, "permission": "reserve", "group": null, "id": 4, "user": null } ], "possible_permissions": [ { "value": "edit_policy", . .. . .. The authentication bit is not required I suspect. [1] http://paste.fedoraproject.org/41757/13800067/ Thoughts and suggestions welcome. Best, Amit. -- Amit Saha <http://echorand.me> Infrastructure Engineering and Development Red Hat, Inc.

3 16

Beaker tasks in a separate repository?
by Amit Saha 04 Nov '13

04 Nov '13

I am beginning to think why don't we maintain the Beaker tasks in a separate repository? Each task has and will have their own version/release info, so there won't be a "release" of the tasks per se. We have made task upgrades in the past at various points during the release cycle, so there doesn't seem to be any pressing reason not to do this. Thoughts? Best, -Amit. -- Amit Saha <http://echorand.me> Infrastructure Engineering and Development Red Hat, Inc.

3 4

Current state of Beah's development and release process
by Amit Saha 30 Oct '13

30 Oct '13

Currently, Beaker's default test harness is Beah [1]. This post will attempt to summarize how Beah is currently developed, released and upgraded with the rationale explained at the end. To learn briefly about Beah in the context of Beaker, please refer to the architecture guide [2]. *Current development process* Bugs against Beah are currently filed using the "test harness" component under Beaker's bugzilla. All development happens on the "master" branch. Here is a brief overview of what is the current approach of contributing a feature, a bug fix or another change to Beah: - Clone beah: git clone git://git.beaker-project.org/beah - Create a new branch for the changes, say mybranch - Once you are happy with the changes, push your branch to gerrit: git push git+ssh://gerrit.beaker-project.org:29418/beah mybranch:refs/for/master - Once the change has been approved, the patch is merged to master *Release and upgrade process* Currently, the release process involves tagging a new release using tito and pushing the tag to remote beah repository. Beah releases happen at the same time as new Beaker releases. Beah packages are served from a yum repository at [3]. If there is a new Beah release, the harness packages have to be updated on the Beaker server using the beaker-repo-update utility. *Existing Beah documentation* These are the current "mentions" of Beah in the Beaker documentation. - http://beaker-project.org/docs/architecture-guide/capabilities.html#test-ha… - http://beaker-project.org/docs/admin-guide/upgrading.html#new-harness-packa… - http://beaker-project.org/docs/user-guide/task-metadata.html *Future plan* As is fairly apparent from the above summary, the release cycle and documentation of Beah is tightly coupled with Beaker's. In fact, currently there is no independent documentation about what Beah is, where to file a bug against it or how code changes can be contributed. The rationale behind this post is to consolidate the current state and use it as a base for decoupling Beah's documentation from Beaker's and gradually adopt a separate release cycle for Beah. Once we have done that for Beah, we hope to do the same for some of the other projects listed at http://git.beaker-project.org/cgit/. Thoughts, suggestions and comments welcome. [1] http://git.beaker-project.org/cgit/beah/ [2] http://beaker-project.org/docs/architecture-guide/capabilities.html [3] http://beaker-project.org/yum/harness/ Best, Amit. -- Amit Saha <http://echorand.me> Infrastructure Engineering and Development Red Hat, Inc.

3 9

Beah develop branch
by Amit Saha 29 Oct '13

29 Oct '13

I have created a 'develop' branch for Beah and hence new patches should now be submitted against it. Quick ref: git push git+ssh://gerrit.beaker-project.org:29418/beah yourbranch:refs/for/develop Best, Amit. -- Amit Saha <http://echorand.me> Infrastructure Engineering and Development Red Hat, Inc.

1 0

Beaker 0.15.1 released!
by Raymond Mancy 23 Oct '13

23 Oct '13

On behalf of the Beaker development team, I'm happy to announce the release of Beaker 0.15.1! It can be found on beaker-project.org[1]. This release fixes a number of regressions found in the 0.15.0 release, as well as some other minor bug fixes. The full changelog can be see here[2]. Regards Raymond. 1 http://beaker-project.org/releases/ 2 http://beaker-project.org/releases/beaker-0.15.1-ChangeLog

1 0

Some thoughts on UI change review and regression testing
by Nick Coghlan 21 Oct '13

21 Oct '13

I read an interesting article on the weekend about the collaborative nature of corporate software development (https://medium.com/better-humans/2e2ed4daeb99). While the whole article was quite informative, what specifically caught my attention were some of the comments on UI testing, especially in conjunction with some comments Ray made on IRC about the challenages of effectively reviewing UI changes on Gerrit. Specifically: 1. Gerrit 2.7 [1] includes the ability to display image files inline in the code review (showing the old and new versions adjacent to each other rather than a diff) 2. Facebook's Huxley [2] is based on automatically taking screenshots based on a webdriver test and committing them to Git It seems to me we may be able to kill two birds with one stone here: by creating Huxley tests for various parts of the UI, we'll get screenshot changes to review as part of the commits, *and* potentially be able to automate regression testing for cosmetic problems that are painful to test with explicit WebDriver API calls. The same article also referenced this UI acceptance testing tool which looks potentially useful for acceptance testing purposes: https://github.com/bslatkin/dpxdt Unlike Huxley, which is designed to integrate with the normal code review and automated regression testing processes for a project, dpxdt (pronounced "depicted") is designed to run as a separate service, capturing screenshots and generating perceptual diffs which are then approved (or not) based on whether they represent deliberate or neutral changes or UI regressions. Regards, Nick. [1] http://gerrit-documentation.googlecode.com/svn/ReleaseNotes/ReleaseNotes-2.… [2] https://github.com/facebook/huxley -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)

3 5

Results of test suite run on Fedora 20
by Amit Saha 20 Oct '13

20 Oct '13

Overall results: 964 tests, 5 failures, 0 errors, 3 skips in 3032.7s Key packages and versions: python-sqlalchemy-0.8.2-1.fc20.x86_64, python-requests-1.1.0-4.fc20.noarch, python-formencode-1.2.6-2.fc20.noarch I deliberately used python-requests-1.1.0-4.fc20.noarch, instead of python-requests-1.2.3-5.fc20 (latest on Fedora 20). Failures: These all result in assertion failures with the new version for python-formencode: bkr.inttest.client.test_group_create:GroupCreateTest.test_group_create bkr.inttest.client.test_group_modify:GroupModifyTest.test_group_modify_display_name bkr.inttest.client.test_group_modify:GroupModifyTest.test_group_modify_group_name bkr.inttest.server.selenium.test_group_edit:TestGroupsWD.test_check_group_name_display_name_length The problem is the error message has been "fixed" in the new version, but it remains the same on the older versions. Here is an example: - Enter a value less than 255 characters long? + Enter a value not more than 255 characters long? So shall we instead have an "or" condition here in the assertion so that it tests against both possible error messages? "Known" Fedora failure: bkr.inttest.server.test_jobs:TestJobsController.test_job_xml_can_be_roundtripped With python-requests-1.2.3-5.fc20, this patch is needed: diff --git a/Client/src/bkr/client/__init__.py b/Client/src/bkr/client/__init__.py index eb4a4fd..2414bf7 100644 --- a/Client/src/bkr/client/__init__.py +++ b/Client/src/bkr/client/__init__.py @@ -63,7 +63,7 @@ def request(self, method, url, **kwargs): # turn 'json' parameter into a suitably formatted request if 'json' in kwargs: kwargs['data'] = json.dumps(kwargs.pop('json')) - kwargs.setdefault('headers', []).append(('Content-Type', 'application/json')) + kwargs.setdefault('headers', {}).update({'Content-Type': 'application/json'}) return super(BeakerClientRequestsSession, self).request(method, url, **kwargs) return BeakerClientRequestsSession() Fortunately, this is compatible with 1.1, so we should get this patch merged. Thoughts on the above suggested changes? Best, Amit. -- Amit Saha <http://echorand.me> Infrastructure Engineering and Development Red Hat, Inc.

1 5

Changes to building local RPMs
by Nick Coghlan 15 Oct '13

15 Oct '13

For Beaker 0.15, we migrated away from using tito to generate RPMs (we wanted to start building release candidate RPMs and it was easier to replace tito with some suitable scripts than it was to figure out how to get tito to do what we wanted). The new commands are in the Misc directory: rpmbuild.sh and tag-release.sh Misc/rpmbuild.sh is just a thin wrapper around rpmbuild (use rmpbuild.sh -bs and mock to generate RPMs for other versions) $ ./rpmbuild.sh Usage: -bs|-bb <rpmbuild-options...> Hint: -bs builds SRPM, -bb builds RPM, refer to rpmbuild(8) Misc/tag-release.sh handles the edits needed to bump the version number and then commits and tags the result. $ ./tag-release.sh Usage: ./tag-release.sh <version> [<prerelease>] Example: ./tag-release.sh 0.15.0 rc1 The latter change means that the "*.py.in" input files that were previously used to inject the version are now gone, and Beaker should run from a fresh clone without needing to run "make" first. Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Beaker-devel October 2013