QA Process

[Eric "Sparks" Christensen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/24/2010 09:35 AM, John J. McDonough wrote:
> On Wed, 2010-11-24 at 08:23 -0500, Eric "Sparks" Christensen wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 11/24/2010 02:52 AM, Zach Oglesby wrote:
> 
>>> The first possibility is to require all changes to be entered as
>>> patches in bugzilla, each patch would then have to be verified by a QA
>>> person prior to being included in the guide. I don't particularly like
>>> this idea as a content writer, I don't really want to have to submit
>>> tickets to bugzilla every time I make changes to a guide that I am the
>>> POC on.
> 
>> While this may be a cumbersome solution, it may also be the easiest to
>> manage.  If I make a change in the third paragraph on the second page of
>> X guide then I have to convey that in some way to the QA person and we
>> need to track it to make sure that it gets done.
> 
> No kidding cumbersome.  Probably triples the work, and why, just to get
> it into BZ?  There is a commit message automatically generated with
> every change, we can easily roll back the change in git, and it doesn't
> affect anything but the repo until we publish, so adding a huge amount
> of manual work just to make the commit message show up in BZ sounds
> incredibly dumb beyond all belief.

It isn't that cumbersome.  It would take me less than a minute to post a
patch on BZ.

> 
> That being said, I do like the idea of tracking everything in BZ, but
> this is just horrible.

Horrible?

> 
>>> The second idea is to write to the proposed docs-qa list to inform the
>>> QA team that a doc has been changed and need to be looked over. We may
>>> be able to semi-automate this process by emailing the list when
>>> commits are pushed a repo.
> 
> It happens now.  If you subscribe to docs-commits you get it.  Nothing
> to be done.

But once the change is in the repo it is too late.  You shouldn't commit
a change before it is QA'd.  If the change is in the repo then there is
a high chance that the change will make it to the final guide without
being checked.

That said we could go the other way and just commit all the changes and
have a QA person go over the entire document before release.
> 
>> You'd still need to know what changed and be able to approve or
>> disapprove that change.  Once it is in the repo it is often too late to
>> pull back out and make more changes or know if something has already
>> been through QA.  I'd prefer to not have anything go into the repo until
>> it has been approved.
> 
> The whole point of an RCS of any flavor is that you CAN roll back a
> change.  If we want to know something has been QAed we could use git
> tags.  Further, if we don't push a change until something has been QAed
> in BZ, then we frustrate the advantages of a collaborative RCS.  People
> would need to wait for QA before doing any change, and EVERYONE would
> need to know what was waiting in the wings to be pushed.  So, we would
> need another list or database or something with the QA status of
> everything, preferably automatically updated with "wait a minute"
> whenever someone did a clone with the intention of editing.

The pending list already exists...  it's called BZ.

> 
> Another thought....
> 
> 1) Encourage people to commit and push frequently.  I think sometimes
> people are reluctant to push and that leads to potential conflicts.
> Push ALWAYS, you can always back it out.

When was the last time you tried to roll back an edit that was made
several pushes ago?

> 
> 2) When you get to a reasonable stopping point, tag your commit with a
> "needs QA" tag, we would need to work out what that looks like because
> there could be several.
> 
> 3) When something is QAed, it gets tagged as QAed.
> 
> 4) Publish from a QAed tag.

I'm good with 2-4.  I hadn't really thought about doing something like
this.  I don't know that this is possible in SVN, though.

> 
> Very little extra work, far less potential for problems.  Only downside
> is that it doesn't show up in BZ.
> 
> Certainly an email filter on QA tags could be written to identify
> docs-commits messages about tags.  Some bright bash guy might even
> figure out how to create a BZ entry on a "Needs QA" tag.

nb?

> 
>> It would also be good if everyone on the QA team was fluent in DocBook
>> XML so tags could be checked or could be added where needed.
> 
> Well, we need that in the POs a lot more than in the XML.  And Publican
> finds that for you.  The new Publican is fast enough that it isn't very
> painful to use it for a syntax check, especially now that we don't need
> to do the separate merge.  Admittedly, finding the line in the PO that
> caused the error is sometime painful, but I would hope that anybody
> making edits at least does a build in the original language to find any
> tagging errors in the XML.

Well, I was talking more about a tag that isn't there but should/could
be and not a broken tag.

> --McD

- -- Eric
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJM7S2TAAoJEDbiLlqcYamxb9QP/1YRgeMQoLixBUg6RBgBz46R
r3+lW4ehg0P7ChhdWgXZQM9zymiE5onAyG6d0z6uCsBHJ+hy0uWmKEMFo1/4ZCmO
wbcVJ3wWo3/GcoKssGxyPsH/Ah68NjM/pEVfc+gXC/4nbb/JY3JAiwxm51XyuF8i
AyMxeKywUta6bTOgRsWUFwTdZmjZdzSOZbX5jvIgbZIQDzpiyKl3VGMrmDYIUp2s
VFbuNOO44cKQMoSe+Ujk7V+c2t0P4qB/Iew96jQ560V50ZWMqXLR+Dwtc/K8OW9o
wzHwjbcDLNm+w30bIj5QVG5VNMXuHz0KzN1jdN7RmrX7fQ5aZ9G/McVwJVBzTcyo
HOR4zTFfwbKzIRyQ7JnF65AjkR3vT4WZJkmDxomdei7x4x24ceQwsyKnJ6SAIjqU
1gmsTm3VpZmZXx/y+sfUh8u2MXp91TZ1O8/Gex7dh99PxSS3C1cGi4X+rtJKAWRB
ameFpiYuG4qzTJfffEgbb+H4Hy8IU/P13Nod4EGdNRUM/X9IvTZFXvtKqE7IApcs
me8KOefW2PjQM27iRGIwM9ACRWo4jhmE2j5G58EwJ+HkL5rGyvFBug2quWn4sgPe
454qK3WZSjrpL0I+9bNqFREQC52NjWiTXJP3yqzgCWf1GMP9X8fXi94swn1zvoDl
/EsBK7ygjlszd34koVu7
=KKqO
-----END PGP SIGNATURE-----