Open Letter: Why I, Kevin Kofler, am not rerunning for FESCo

[Stephen John Smoogen]

On Mon, May 3, 2010 at 5:58 PM, Kevin Kofler <kevin.kofler at chello.at> wrote:
> Jesse Keating wrote:
>> Please stop banding about the forum poll as if it were some sort of
>> scientific measure with meaningful results one could use as a basis for
>> decision making.
>
> It's the best data we have.

And the statisticians I know say that some data is usually much worse
than no data at all. When we used to go about poll data in class, the
professors used a training story like this:

Get on a motorcycle. Drive 40 miles per hour. Spit in the direction
you are going in. Do it multiple times just to be sure. From that data
give the direction of the wind for the countryside. Unless the wind is
very very strong and your methods of getting the data very stringent,
the best your poll will tell you is that 'the wind' was towards your
face. It doesn't matter if at rest the wind was to your back.

The poll has the following problems:
1) self selected pool of subjects being polled (who voted versus who
did not. were the people polled or did anyone who could poll
themselves)
2) unknown controls on who was polled versus who wasn't. (how many
times did someone vote multiple times, how strong can you confirm
that)
3) how neutral were the questions and how many ways were the questions
asked so that language biases were tested?

Any one of those can invalidate the mathematical tests you say to run
as they require random pools, controls on populations polled, and
non-leading questions. People keep telling you this and you seem to
keep ignoring it.

At best what you can say is the following:

183 people who use the Fedora Forum expressed their opinions on X,Y,Z
questions. 78% of those people voted for X and 22% voted for Y. Due to
methodologies the level of uncertainty is not easily quantifiable
making it unknown how it represents the general population. Further
study and better testing methodologies are required.

I would say the same thing if the votes had been the other way.. and
people were harping that this proved that slow updates was what people
wanted.

-- 
Stephen J Smoogen.
“The core skill of innovators is error recovery, not failure avoidance.”
Randy Nelson, President of Pixar University.
"We have a strategic plan. It's called doing things.""
— Herb Kelleher, founder Southwest Airlines