On Thu, Mar 12, 2020 at 2:58 PM Kamil Paral <kparal(a)redhat.com> wrote:
The adjusted criterion seems to affect only this test case:
https://fedoraproject.org/wiki/QA:Testcase_desktop_menus
My current plan is to redo it somewhat. I want to:
1. Drop all instructions to run applications from Testcase_desktop_menus
and make it Optional. If will once again reflect its title and only care
about menus and icons in menus.
2. Create the following new test cases:
Testcase_desktop_app_web_browser
Testcase_desktop_app_file_manager
Testcase_desktop_app_package_manager
Testcase_desktop_app_image_viewer
Testcase_desktop_app_document_viewer
Testcase_desktop_app_text_editor
Testcase_desktop_app_archive_manager
Testcase_desktop_app_terminal_emulator
Testcase_desktop_app_problem_reporter
Testcase_desktop_app_help_viewer
Each of these will instruct people to start and test basic functionality
of an application of that type, provided by the desktop. It will also
explain what to do when there are multiple such applications.
These test cases will be mandatory for all desktops.
3. Create the following new test case:
Testcase_desktop_app_others
This will include the same instructions as in 2), but it will instruct
people to test all applications not covered by specific test cases in the
same matrix. This test case will be mandatory for Workstation x86_64 and
Optional for others.
This will have the benefit of multiple people being able to easily
collaborate on desktop testing. It will definitely be less off-putting than
a single "do it all" test case. It allows people to participate in desktop
testing even if they have limited time. And it allows us to write OpenQA
tests for certain applications and fill out the results on the wiki.
Note: In point 2), we don't actually need 10 different test cases
(templated or not). We might just create Testcase_desktop_app_single
instead of those 10 and differentiate them on the matrix level the same way
as we do with e.g. "Testcase_Anaconda_save_traceback_to_bugzilla" [1]. So
like this:
Testcase_desktop_app_single (web browser)
Testcase_desktop_app_single (file manager)
...
I don't know if it complicates OpenQA results submission, though.
The changes are now live.
The Desktop matrix changed a lot:
https://fedoraproject.org/w/index.php?title=Template%3ADesktop_test_matri...
I've created these two new test cases:
https://fedoraproject.org/wiki/QA:Testcase_desktop_app_basic
https://fedoraproject.org/wiki/QA:Testcase_desktop_app_basic_others
I found out that we already had a specific test case for terminal and web
browser. The terminal test case didn't contain anything extra, so I
replaced it with the generic desktop_app_basic test case, and removed the
associated criterion:
https://fedoraproject.org/w/index.php?title=QA%3ATestcase_desktop_termina...
The browser test case contains specific browser requirements, so I left it
intact.
The desktop_menus testcase now only tests menus (again):
https://fedoraproject.org/w/index.php?title=QA%3ATestcase_desktop_menus&a...
And I updated criteria associations on release criteria pages:
https://fedoraproject.org/w/index.php?title=Fedora_32_Final_Release_Crite...
https://fedoraproject.org/w/index.php?title=Basic_Release_Criteria&ty...
I hope I didn't make any errors during the process, that was a lot of lines
to change.
OpenQA admins, please note that I adjusted how
Testcase_Printing_New_Printer is displayed in the matrix. I wanted to make
it look the same as the other rows I added. If that breaks results
submission, I'm sorry, can you adjust it please? Thanks!