F21 schedule: what would you do with more time?

[Tim Flink]

On Thu, 22 Aug 2013 11:03:52 -0400
Matthew Miller <mattdm at fedoraproject.org> wrote:

> Based on discussion at Flock, on the devel mailing list, and in the
> FESCo meeting, we are looking for feedback on the idea of a longer
> release cycle for Fedora 21 -- not (right now at least) the bigger
> question of the 6-month cycle overall, but just, right now, slowing
> down for a release to get some things in order.
> 
> Specifically, both Release Engineering and QA have clear needs (and
> even plans for) greater automatiion, but are also incredibly busy
> simply doing the things they need to do _now_ to get the release out
> the door.
> 
> So, FESCo would like to see some specifics, like "If we had one week
> with nothing else to worry about, we could have automated generation
> and upload of cloud images" (to pick an example I personally care
> about). Or "with six months of overall delay, we could have
> continuous integration testing of a key subset of rawhide". Or "we
> could spend a couple of weeks and automate the new package and review
> workflow".

For QA tools/automation, a week isn't going to give us much - we already
have a couple of weeks between releases and we're more blocked by
projects that will take longer than a week.

I've been meaning to sit down and come up with a more detailed
list/plan for qa development (automation, other tools) but that hasn't
happened yet. At the end of this email, I made a list of the things that
I've been talking about with various folks in the order of both how
important I think they are and how much the project would benefit from
a more extended break between releases. Exactly how much of this we
could do with more time depends on both how much time we're talking
about and who all would be involved.

Tim


Taskbot [1]:
  This will become the foundation for future automation work and at
  the moment is at least somewhat blocking our other automated testing
  initiatives from moving forward. This would (eventually, not all of
  this would be part of the first deliverable) give us:
   - easier for new people to get up to speed and help
     creating/maintaining checks/tasks
   - more flexibility in the types of checks/tasks that could be
     automated
   - better triggering (run X check for builds of Y package, run Z at a
     certain time etc.)
   - better reporting
   - automated analysis of logs for oddities or to answer questions
     like "how long have we been seeing this error in syslogs"

[1] http://tirfa.com/tag/taskbot.html


Automated Install Testing:
  Many of our current validation test cases [2] are very straight
  forward and could be automated to free up human testers to do other
  testing that isn't (easily) automate-able.

  We have a start on this with infinity [3] but there is still some
  development work, a lot of integration work to do and test cases to
  write before any of this is usable.

[2] https://fedoraproject.org/wiki/QA:Fedora_20_Install_Results_Template
[3] https://github.com/garretraziel/infinity


Smoke image build automation:
  This has been started [4] as part of GSoC 2012 but is still a little
  shy of being usable in production. The idea would be to build images
  as soon as new packages (anaconda, maybe others) so that a set of
  automated install smoke tests could be kicked off. This could involve
  working with releng on something to do the composes - I'm less
  interested in who does the work than I am in being able to get images
  to test on demand.

[4] https://fedorahosted.org/fedora-build-service/


Test Case and Results Management:
  We want to replace our current wiki-based system of test cases and
  results matrices. I'm not aware of any existing system that would
  fit our needs and I think we're going to end up rolling our own
  unless something new shows up.


Update/Build Gating:
  There's been talk about gating updates based on automated test
  results for a while but nothing's finished yet. A lot of this is
  integration with bodhi/koji but there are still some bits that
  haven't been implemented (test result manual override is the first
  thing that comes to mind)


Better Automated Checks:
  Rewriting depcheck to be more useable, abi breakage checks, running
  gnome's new test suite or anything else that people can come up with.
  This can happen just as easily in parallel with releases once we have
  the infrastructure in place to run them, though and doesn't really
  require an extended break between releases.







> What Infrastructure projects would be helped by this? Web and design
> team, would slowing down the release focus allow time to work on, oh,
> say, getting the Wiki beautiful (or does it not matter)? What else?
> 
> As we look at Fedora.next ideas and possibly decide to start
> implementation in the F21 timeframe, we will likely find _new_ things
> that take specific work. Let's not worry about that right now. What
> things we do _now_ could be improved with the investment of some
> effort?
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20130822/824badb7/attachment.sig>