Keith Lofstrom wrote:
I would put myself in the 45% group, and include many of the folks that are complaining about booting or mirror problems. The Fedora test process seems to be geared exclusively for the 5%. As a result, Fedora will release with less than 10% of the potentially available testing effort, with the remaining effort disproportionally focused.
Jef Spaleta jspaleta@princeton.edu writes:
I would sort of agree with that. I would agree that the aggressive timescale for testing for fedora core means relying on those 5% (I'd put it closer to 1%) to be a strong component of the process. I would also say that the dropping of the private betas that rhl use to have makes the situation somewhat more dramatic looking than it is. As others have said, the fedora testing process, is a more open process than what happened with rhl. And more open translates directly into sharper edges for the informed to impale themselves on while blundering around in the dark.
There is a trade-off here, taking the very long time to do things a piece at a time, means the pieces quickly become out of sync with upstream development.
...
To reiterate, my concern is that the testing process gets hung up on the front end, and the distro as a whole is incompletely checked.
I got some feedback, so I will go on to prescription. A "conservative", one-piece-at-a-time development process is of course much too slow. Instead, I would divide up the work into domains so that people can choose a domain and test in that. Where to draw the line is difficult, but I would probably make the domains "kernel and drivers", "installation and update", "X and desktops", and "applications". Obviously, many problems will cross domains, but to the extent that you can partition things you reduce the scope of most problems.
20 years ago, I designed whole chips. Now I design small pieces of chips with 10 times as many transistors per piece. Some times my pieces interact nastily with other pieces, and that can cost a $1M mask set and 3 months. But 99% of the problems stay confined in my box, or in the mid-sized box my piece is in. This is how the processors just keep getting cheaper and faster and better and sooner, because we can partition the hell out of the problem, and this allows chip designers to generate terabyte-sized designs really, really fast.
Code right now is partitioned into modules and programs and libraries and packages, then distros. For those of us interested in testing the interaction of packages, a single package is too small and a whole distro is too big. Hence domains. If we have domains A, B, and C, then testers can work with Anew + Bnew + Cnew if they are in the 5% elite, but the 70% less-skilled testers can work with Aold + Bold + Cnew if they just want to look at interactions in that domain. For my four suggested domains, that suggests 5 test distros rather than one. It would be wonderful to do all combinations, or even any combination with any incremental change in any package, but dependencies make that impractical. In any case, all 5 test distros at alpha are designed to merge into one distro at beta, probably at test 3.
Schedule would be: Test 1: Domains defined, new packages in each domain defined. Find package problems. Test 2: Packages upgraded/downgraded/replaced to fix package problems. Find between-package problems within domains. Test 3: Integrate changed domains into one beta distro. Find between-domain problems. Release0: This is it! The complete, perfect distro. Release1: Just kidding! We fixed those killer bugs. Release2: We really mean it this time. Release ...
This will be more work to set up; I would guess 20% more, initially. But it captures 10x more labor, making more available for desktop/application testing, which in turn reduces the distractions for the install/kernel testers. The real question is, do you want to accomplish 20% testing or 80% testing? I have real concerns that Fedora Core 2 release, as a coherent distro, will be 20% tested, because intense application/interaction beta testing will never happen. If I wanted untested beta code, I would run windows.
Keith
Keith Lofstrom said:
I got some feedback, so I will go on to prescription. A "conservative", one-piece-at-a-time development process is of course much too slow. Instead, I would divide up the work into domains so that people can choose a domain and test in that. Where to draw the line is difficult, but I would probably make the domains "kernel and drivers", "installation and update", "X and desktops", and "applications". Obviously, many problems will cross domains, but to the extent that you can partition things you reduce the scope of most problems.
What is preventing you from updating the "domain" you choose from Rawhide and testing?
-
Keith Lofstrom -
William Hooper