I would like to propose adding more test days for spins to the Fedora QA process.
Fedora Spins ( http://spins.fedoraproject.org/ ) are customized Live CDs which can easily be given to volunteers to test. They are convenient, pre-grouped sets of related items arranged in a logical fashion.
Some of them are a bit obscure; but touch functionality which is not looked at often.
If we can regularly test spins and add test cases for them, then we can get better manual test coverage without forcing testers to install packages they do not regularly use.
It may be a bit crazy to have a test day for every spin every release cycle; but hopefully the spinners are looking at them every cycle already.
--- SJG
On 06/12/2012 04:54 PM, Samuel Greenfeld wrote:
I would like to propose adding more test days for spins to the Fedora QA process.
Fedora Spins ( http://spins.fedoraproject.org/ ) are customized Live CDs which can easily be given to volunteers to test. They are convenient, pre-grouped sets of related items arranged in a logical fashion.
Some of them are a bit obscure; but touch functionality which is not looked at often.
If we can regularly test spins and add test cases for them, then we can get better manual test coverage without forcing testers to install packages they do not regularly use.
It may be a bit crazy to have a test day for every spin every release cycle; but hopefully the spinners are looking at them every cycle already.
SJG
A new Fedora Audio Spin will be prepared for this release and is part of a Summer of Code project - I imagine we will be wanting to schedule a test day for this, but I'll defer that to the student.
One has to be mindful of the fact that errors uncovered in the Spins testing may have no relevance on the default desktop at all. Spin kickstarts add and remove packages as the requirement dictates and this may introduce errors. However, a group primarily focussed on audio may uncover errors in that domain (pulseaudio for example) where normal testing has not.
regards,
Brendan
I would like to propose adding more test days for spins to the Fedora QA process.
Hi,
spin maintainers are free to propose a Test Day for their Spin if they wish. Test Days are open for anyone who has something to be tested and there's a possible interest in the community to test it. The spin maintainer just has to propose the test day [1] and then attend the event and help testers debug their issues, that's all.
On Wed, Jun 13, 2012 at 6:23 AM, Kamil Paral kparal@redhat.com wrote:
spin maintainers are free to propose a Test Day for their Spin if they wish. Test Days are open for anyone who has something to be tested and there's a possible interest in the community to test it. The spin maintainer just has to propose the test day [1] and then attend the event and help testers debug their issues, that's all.
I think you are presuming that Spin maintainers know how to request QA help, and are willing to run test days.
What spins are looking for from QA likely is a topic for the Spins SIG list.
On Tue, Jun 12, 2012 at 11:22 AM, Brendan Jones brendan.jones.it@gmail.com wrote:
One has to be mindful of the fact that errors uncovered in the Spins testing may have no relevance on the default desktop at all. Spin kickstarts add and remove packages as the requirement dictates and this may introduce errors. However, a group primarily focussed on audio may uncover errors in that domain (pulseaudio for example) where normal testing has not.
The point of this testing would be to test non-default installations. To the best of my knowledge Fedora does not seem to have much coverage in this area. The current test procedures seem to be written around doing the "default" installation in as many different ways as possible (via PXE, with RAID, etc.).
Instead of creating an audio, graphics, etc., test day from scratch, Spins come prepackaged with several items in their specific areas. There is a formal process for Spin approval so interest in the health of Spins and their contents can be presumed.
If there are concerns about packages removed from spins to meet size constraints then it should be possible to test most spins using group installs.
The alternative would seem to be getting users to provide feedback for packages in Bodhi besides "It works for me" (or none at all).
-
Brendan Jones -
Kamil Paral -
Samuel Greenfeld