QA Process

[John J. McDonough]

On Wed, 2010-11-24 at 08:23 -0500, Eric "Sparks" Christensen wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 11/24/2010 02:52 AM, Zach Oglesby wrote:

> > The first possibility is to require all changes to be entered as
> > patches in bugzilla, each patch would then have to be verified by a QA
> > person prior to being included in the guide. I don't particularly like
> > this idea as a content writer, I don't really want to have to submit
> > tickets to bugzilla every time I make changes to a guide that I am the
> > POC on.

> While this may be a cumbersome solution, it may also be the easiest to
> manage.  If I make a change in the third paragraph on the second page of
> X guide then I have to convey that in some way to the QA person and we
> need to track it to make sure that it gets done.

No kidding cumbersome.  Probably triples the work, and why, just to get
it into BZ?  There is a commit message automatically generated with
every change, we can easily roll back the change in git, and it doesn't
affect anything but the repo until we publish, so adding a huge amount
of manual work just to make the commit message show up in BZ sounds
incredibly dumb beyond all belief.

That being said, I do like the idea of tracking everything in BZ, but
this is just horrible.

> > The second idea is to write to the proposed docs-qa list to inform the
> > QA team that a doc has been changed and need to be looked over. We may
> > be able to semi-automate this process by emailing the list when
> > commits are pushed a repo.

It happens now.  If you subscribe to docs-commits you get it.  Nothing
to be done.

> You'd still need to know what changed and be able to approve or
> disapprove that change.  Once it is in the repo it is often too late to
> pull back out and make more changes or know if something has already
> been through QA.  I'd prefer to not have anything go into the repo until
> it has been approved.

The whole point of an RCS of any flavor is that you CAN roll back a
change.  If we want to know something has been QAed we could use git
tags.  Further, if we don't push a change until something has been QAed
in BZ, then we frustrate the advantages of a collaborative RCS.  People
would need to wait for QA before doing any change, and EVERYONE would
need to know what was waiting in the wings to be pushed.  So, we would
need another list or database or something with the QA status of
everything, preferably automatically updated with "wait a minute"
whenever someone did a clone with the intention of editing.

Another thought....

1) Encourage people to commit and push frequently.  I think sometimes
people are reluctant to push and that leads to potential conflicts.
Push ALWAYS, you can always back it out.

2) When you get to a reasonable stopping point, tag your commit with a
"needs QA" tag, we would need to work out what that looks like because
there could be several.

3) When something is QAed, it gets tagged as QAed.

4) Publish from a QAed tag.

Very little extra work, far less potential for problems.  Only downside
is that it doesn't show up in BZ.

Certainly an email filter on QA tags could be written to identify
docs-commits messages about tags.  Some bright bash guy might even
figure out how to create a BZ entry on a "Needs QA" tag.

> It would also be good if everyone on the QA team was fluent in DocBook
> XML so tags could be checked or could be added where needed.

Well, we need that in the POs a lot more than in the XML.  And Publican
finds that for you.  The new Publican is fast enough that it isn't very
painful to use it for a syntax check, especially now that we don't need
to do the separate merge.  Admittedly, finding the line in the PO that
caused the error is sometime painful, but I would hope that anybody
making edits at least does a build in the original language to find any
tagging errors in the XML.

--McD