F20 -> F21 test matrix and scope

[Chris Murphy]

On Sep 17, 2014, at 1:14 AM, Adam Williamson <adamwill at fedoraproject.org> wrote:

> On Mon, 2014-09-15 at 15:50 -0600, Chris Murphy wrote:
>> For reference:
>> https://lists.fedoraproject.org/pipermail/desktop/2014-September/010657.html
>> 
>> I'm uncertain what this means for F20->F21 testing scope. For sure,
>> fedup upgrade from Fedora 20 to Fedora 21, just like any other
>> release. But do we need to test the 'yum swap' and 'yum install
>> <product>-environment' conversion after doing the upgrade?
>> 
>> Presumably it'd be for final release, but there's some float between
>> final go/no-go and actual release day.
> 
> Man, why did I not know 'yum swap'? I've been using 'yum shell' to do
> that since forever.
> 
> It's an interesting question, I'd say we should certainly test it, and
> if it's going to be promoted/mentioned in the Fedora 21 publicity it
> should probably block final. Do you want to write up a test case? Thanks
> for spotting this!

Time frame for the write up?

What's the metric for fail vs success? Ideally there'd be some way to compare clean F21 <product> *rpm, to F20->F21"standard"->F21"productized" *rpm, and they'd be identical. Maybe some fc20 rpms are expected to remain in the latter case, I'm not sure whether those are cleaned up.

What do you suppose the delta download is between standard and each of the three products?

I definitely think my best usage here is ways to cheat and be lazy by using snapshotting to avoid having to install F20 three times to do these four tests. My bet is that fedup would be the part most sensitive to problems related to esoteric partitioning/layouts. So if that step works, I don't expect 'yumswap' + 'yuminstall' "productizing" steps to give a hoot about layout, or even baremetal vs VM. Therefore it should be fine to predicate the test being done on a VM, and using qcow2 (or vbox) snapshots.


Chris Murphy