Res: Open Letter: Why I, Kevin Kofler, am not rerunning for FESCo

[Kevin Kofler]

Matthew Garrett wrote:
> If updates cause regressions in functionality then that indicates that
> our update testing process failed. The answer to that is to fix the
> update testing process, not bypass it.

Your assumption there is that it is possible for a testing process to catch 
ALL regressions. I couldn't disagree more, and so far evidence has proven me 
right. For example, the critical path process, upon which the new update 
process is modeled, failed to catch the regressions in non-GNOME spins you 
caused by splitting out hal-storage-addon to a subpackage. NO amount of 
testing will EVER catch ALL regressions before they hit users. There will 
ALWAYS be a need for a way to fasttrack regression fixes! (And in fact a 
direct stable push is how we fixed the KDE spin.)

> There is no change too trivial to not require testing. The software
> industry is full of examples of obviously correct fixes causing hideous
> breakage. Most developers get to learn that the hard way at some point,
> but it's still preferable to put processes in place to protect users
> from accidents.

While you do have a point in principle, in practice, our maintainers are 
quite good at judging the risk of their changes, and often the risk is so 
extremely low that it is far outweighed by the benefits of getting the 
change out ASAP. This is always a tradeoff. And 100% bugfreeness doesn't 
exist anyway, testing is NOT going to catch all issues either.

There is a point at which the risk is so low that other real-world risks 
such as hardware failure are several orders of magnitude larger. So why 
bother worrying about such a low risk?

> Regardless of your definition, there were several users who felt that
> the KDE 4.4 update broke things. That's a problem. It makes us look bad.
> We'd like to avoid those users being unhappy.

It is true that KDE 4.4 wasn't as smooth as we had hoped for some users. 
This is strange, because for most of us, things just worked! I'll note that 
KDE 4.4 got A LOT of testing, and all testing indicated that we are good to 
go. We would NOT have pushed this out if we hadn't been convinced that our 
update is regression-free. This is just yet another proof that testing will 
NEVER catch ALL issues. What happened was that:
* the Akonadi migration seems to be fragile in some ways, and choke on 
various corner cases, often of unknown nature. This did not show up during 
testing. For most of the issues, KDE UserBase offers workarounds.
* Akonadi needs manual configuration to work with NFS home directories. We 
were not aware of this when we pushed 4.4 out and none of our testers was 
using this configuration. This issue can be worked around by the user by 
following the instructions on KDE UserBase, which detail the required manual 
configuration.
* we underestimated the annoyance factor of a warning Akonadi pops up if 
Nepomuk is not running. (This warning is mostly harmless at this time. It 
just means that some Akonadi functionality is disabled.) This has been 
remedied by a kde-settings update which enables Nepomuk by default.

It shall however be noted that the Akonadi migration is a one-time event 
(well, there will be a second part in KDE 4.5, but once that's done too, the 
migration problem is solved), so I don't expect this kind of issues in 
future KDE upgrades. (In fact, we had NO similar complaints for our previous 
4.1, 4.2 and 4.3 upgrades.)

And IMHO most of the blame for the roughness of the Akonadi migration goes 
to upstream KDE. If we had been aware of the issues this is causing, we 
would have evaluated alternatives (e.g. staying on kdepim 4.3, or shipping 
the pre-Akonadi kdepim-enterprise4 branch). But all the feedback we got at 
the time of the decision pointed to the migration being trouble-free, and in 
fact I still believe that for the vast majority of our users, it was. You 
only hear from the handful people who ran into trouble. And we also need to 
weigh those against the people for whom KDE 4.4 fixed their issues. It has 
fixed thousands of bugs!

        Kevin Kofler