submitters +1ing their own packages

[Josh Boyer]

On Thu, Sep 8, 2011 at 12:54 PM, Nils Philippsen <nils at redhat.com> wrote:
> On Wed, 2011-09-07 at 18:02 -0700, Adam Williamson wrote:
>> On Wed, 2011-09-07 at 18:47 -0600, Kevin Fenzi wrote:
>> > * As a maintainer it's easy to have a env that gets out of sync with
>> >   what a QA or end user would use. Ie, you make 20 iterations of a
>> >   package to test something, tweak configs to check something, and get
>> >   it all working, but perhaps your machine is no longer setup the way a
>> >   fresh install or upgrade of your package would be. Or you tested a
>> >   version and then changed just 'one little thing' and pushed that and
>> >   it turned out to break it.
>>
>> Both hughsie and myself, and I think everyone else in favour of
>> maintainer +1s, suggested maintainers should test *in a vanilla
>> environment*, with the actual package they were submitting, before
>> +1ing. +1ing on the basis of a test build or in a dirty environment
>> would be a no-no and could lead to the removal of +1 privileges if
>> repeated.
>
> I think we should define what a "vanilla environment" is then. One could
> argue that either of the following could be described as "vanilla":
>
> - A fresh system without any modifications or unstable updates other
> than that being tested. Pro: makes testing comparable. Contra: You
> essentially need a special VM for testing which needs to be installed
> freshly for each tested update. Makes tests comparable ;-), i.e. reduces
> the amount of different environments in which an update is tested. I do
> actually want testing to be done on systems that aren't just "minimal
> install plus updates plus a user beside root".
>
> - A system in good condition (packages verify well, no dupes) that's
> used normally, i.e. what you would see being used by normal persons
> without any fancy hacks in configuration, or worse, non-config files
> owned by packages. Pro: Easy to test as you don't need to do anything
> fancy, just yum --enablerepo=updates-testing update <pkg>; <use pkg>

Neither one of those definitions addresses the large variety of
configurations that are possible with vanilla Fedora packages.  E.g.
your update might work wonderfully running a default Gnome desktop
install, but crash portions of the KDE or XFCE stack (for cases of
underlying desktop infrastructure).

I don't think a maintainer can realistically replace wide-spread user
based testing in a variety of environments.  In light of that, we can
either accept a maintainer +1 as "I tested this as I would use it and
it worked" (which should be implied by them submitting the update
already anyway), or we can disallow it as the policy says.

I don't think adding more definitions or steps to the existing policy
is really going to improve anything.

josh