Anaconda exception handling testing

[Adam Williamson]

On Wed, 2012-09-19 at 13:31 -0400, Kamil Paral wrote:
> > Today, I have pushed (to master branch) a patch that adds an easy way
> > for testing exception handling (and bug reporting) in Anaconda. It
> > utilizes unix signals and can be fired by sending SIGUSR1 signal to
> > the
> > main anaconda process (there are usually two processes, main is the
> > one
> > with the lower PID). Probably the easiest way to do that is running
> > this
> > command from tty2 (or window #2 in tmux):
> > 
> > $ kill -USR1 `cat /var/run/anaconda.pid`
> 
> I have updated our test case with the new instructions:
> https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzilla
> 
> 
> We have more test cases regarding bug reporting from Anaconda:
> 
> https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_disk
> According to developers in #anaconda, every time there is an exception
> it is automatically written to anaconda-tb-* file. Jiri Moskovcak says
> there are no plans to add 'save to disk' option to libreport.
> 
> https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_remote_system
> Jiri Moskovcak says there are no plans to add 'save to remote system'
> option to libreport.

If this isn't supposed to come back into libreport, we should just drop
the test case, or replace it with one to ensure the tb file is
accessible in /tmp.

> These test cases were created because in Fedora 17 Anaconda/libreport
> supported these options. Our release criteria also say:
> " The installer must be able to report failures to Bugzilla and local
> disk, with appropriate information included "
> Of course we have to change them, otherwise Alpha couldn't have been
> released (unless we consider creating anaconda-tb-* file as "report to
> local disk", which we very well might do).
> 
> 
> My opinion is that we should merge "save to bugzilla" and "save to
> disk" test cases into one - we will check the presence of
> anaconda-tb-* file and try to report to bugzilla. We should remove
> "save to remote system" test case, because it is no longer supported
> and we have no criteria related to it. Our existing criterion about
> "reporting to local disk" doesn't need adjustments, if we agree that
> the automatic creation of anaconda-tb-* file satisfies it.

I'd say that's a fine plan except we should change the 'report to local
disk' wording in the criterion. That's certainly not how we'd write it
if we were starting with a libreport that didn't have a 'save crash
report to local disk' option, so we should change it.

> Last but not least, we should consider removing
> https://fedoraproject.org/wiki/QA:Testcase_Anaconda_traceback_debug_mode
> because I don't see how that helps our users. That is mainly useful
> for developers, but no user will ever use that, they will use
> updates.img at best. There is no reason for us to test it.

I usually want to at least recall why we had something in the criteria
in the *first* place before we take it out, but aside from that I have
no concrete objections. Anyone remember why we put that in there?
anaconda folks, is it something you consider critical?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net