Fedora Upgrade - release criteria update proposal

[Frantisek Zatloukal]

We have encountered a bug[0] which seemingly “broke” offline updates after
systems were upgraded from an older Fedora to Fedora 29 and had some
multilib packages installed. After the discussion at last week's Release
Retrospective meeting, I am proposing some changes to our blocking
criterions in order to address similar issues in the future:

What we test now

We are currently blocking on upgrading from Fedora n-1 and n-2 releases
only with default package set installed.

What we can test

We can alter our upgrade test cases to also verify that updates after an
upgrade work. The test case might look like this:

   -

   Install Fedora n-1
   -

   Upgrade to Fedora n (updates-testing disabled)
   -

   Enable updates-testing
   -

   Update to latest packages
   -

   Verify that upgrade and update went fine and that the multilib packages
   installed before are still present


Apart from that, we can add (Final Blocking) test case
upgrade_dnf_current_workstation_multilib /
upgrade_dnf_current_minimal_multilib which will test upgrades (and then
updates as described above) with at least one i686 package installed on
x86_64 system. The other (less-generic, but closer to what users probably
do have on their systems) approach would be to test upgrade with steam (or
wine) installed.


What are our opinions about this?

[0] https://bugzilla.redhat.com/show_bug.cgi?id=1642796