The problems with free CVS committing

[Christian Rose]

ons 2004-07-07 klockan 02.10 skrev Bernd Groh:
> >>What is the major drama about having the changes in the cvs? If the 
> >>translation is no good, you can always roll back. I prefer to have the 
> >>changes in cvs, at least then everyone can have a look at the most recent 
> >>changes without having to talk to the person who has the po-file in 
> >>her/his inbox. A commit is not a final thing, it can easily be undone.
> >
> >That depends on when the commit is done.  If the commit is done shortly
> >before a freeze of some sort, there's a good chance that it _can't_ be
> >easily undone.  It first requires getting the maintainer of the module
> >to respin when they thought they were already done.  A lot of the QA
> >which has been done at that point then also needs to be at least checked
> >to see if it needs to be reverified or not.  And if it's not noticed for
> >a day or two (for whatever reason, that's not unreasonable) and you then
> >hit the point at which the tree is frozen, then getting it changed is
> >that much more difficult.
> 
> Well taken. And I don't really have anything against QAing before a file 
> is commited, iff there is a maintainer who is happy to QA it. If the 
> maintainer just doesn't have the time, given it's all voluntary after 
> all, and good translations are simply sitting in an inbox, rather than 
> getting into the packages, than that's a situation I'd like to avoid. Or 
> is it really the case that most committed translations are bad?

No it's not.

And it's not the case that most of the time it rains in the desert
either. Yet, you'll still see that most houses in the desert have
waterproof roofs. Why is that? If we use your logic, this thing with
having roofs would be totally unreasonable, or imply that it rained
often in the desert.

It's that way because we know for sure sometime it's bound to happen,
and the drawbacks of not having the protection are typically considered
more important and have more impact, than the benefits of not having the
protection.

It's the same with CVS access. No, most commits are perfectly fine. Yet,
when someone commits a bad or broken translation, it often wastes other
contributor's time and to a significant extent ruins their effort in the
affected release.


> >> Or do you always 
> >>make sure that your most recent software changes work before you commit 
> >>them? How do you do that with a larger addition? Do you wait till it's 
> >>finished and not keep track of changes for days, but commit a whole 
> >>bunch of changes at once? 
> >
> >The goal has to be to keep CVS working at all times.  Otherwise, you
> >kill testing.  So yes, I tend to try to make sure that either a) my new
> >stuff works or b) at least doesn't break anything new (perhaps the new
> >functionality doesn't work, but old things should continue doing so).
> 
> For the case that my additions can break stuff that's tested, of course 
> I do that too. For my own projects, I usually make sure that all my 
> release branches are working at all time, but I am not that strict with 
> HEAD, since I don't deploy HEAD.

FWIW, in Fedora for translators there is no distinction between HEAD and
stable releases. It's all the same. Everything committed will be
packaged in a product at some point, sometimes later, sometimes *very*
soon.
And usually it's impossible to know exactly when. Even if we know the
rough schedule, maintainers can decide to package sooner or later, and
there can be a large, and usually in advance unknown, difference in time
between the first and the last one.


> >This is even more the case with translations since the code maintainers
> >of a module aren't going to know/keep up with the status of the
> >translations and so need them to be as "correct" as possible at all
> >times since there's no controlling when they're going to do a build.
> 
> Do not disagree in the least. But why should commited translations by 
> default be incorrect? And in particular, why should they now be more 
> incorrect than before?

Why should we have roofs on or houses even in areas where it hardly
rains? Why does my car have crash zones? It surely must imply I'm a bad
driver?
Cars didn't use to have crash zones. Does it automatically imply people
are worse drivers nowadays?


Ok, enough of making fun of the funny logic; let's get to some of the
real, hard reasons free CVS committing without prior review is for the
most part a very bad strategy:

1) If the po file has syntactic errors, the software build will break.
This will make software maintainers, many other contributors, and
testers *very* unhappy.

2) If the po file has other problems, such as bad translations, it will
go undetected for a potentially long time.
Bad translations can be an as bad show-stopper to the end user
perception of product quality as many other bugs, including crashes.
Thus, just because the product doesn't crash, it doesn't mean that's all
there is to QA. Far from it.

3) Prior reviewing is often a lot more likely to get done than post
reviewing. Most people don't consider reviewing to be fun in itself, but
if you know that there's a commit pending the review, you at least have
an incentive to do the review, in order for the commit to get done.
In practice, it's thus very difficult to get volunteers to do post
reviewing, especially if they would have to figure out themselves that
the commit happened in the first place, and how to get the file to
review themselves afterwards.

4) Many teams have public mailing lists. This is a natural place to have
reviews on, since anyone can do a review, and the review doesn't depend
on a single person having to do it. Still, many problems are likely to
be catched.
Thus there's not necessarily the "getting stuck in somone's inbox"
problem you're talking of.

5) If someone commits an erroneous file, someone has to notice it. And
no, tracking CVS takes a nontrivial amount of time and effort. Not
everyone is employed by a company to deal with this kind of things on
their job time. Not many people consider having to constantly monitor
other people's commits and check them afterwards for errors a fun thing
to spend their time on.

6) If someone commits an erroneous file, someone has to fix it. And no,
reverting things in CVS takes a nontrivial amount of time and effort.
Not everyone is employed by a company to deal with this kind of things
on their job time. Not many people consider having to revert other
people's stuff a fun thing to spend their time on.

Out of these, the points 1, 2, 5, and 6 should be obvious to anyone with
any level of QA experience in free software.


Christian