I have bad news, and some good news. Due to either over use of kickstart, or a subtle difference in my home composes vs the composes done in our Phoenix colo, the freeze done last week turns out to not be a good package set. There is a bug that manifests itself the first time a checkbox is checked in the anaconda gui. I never ran into this with my testing, and so we thought things were good for a refresh. We only discovered the bug while testing composes after the refresh on Tuesday.
We were unable to determine the root cause of the above bug, however we did discover that later rawhide trees (like say the tree from yesterday) did not have this bug, a quick test compose was made and tested, and found to be suitable. We decided to refresh the tag again all the way up to rawhide-20090129.
Unfortunately the testing of i386 and ppc didn't uncover a bug with x86_64 early in anaconda, regarding executing some iscsiadm calls. These only appear to happen on x86_64 in graphical mode. Text mode is fine. I tried VNC, only to find out that vnc caused an early crash of anaconda when booting over the network. This particular vnc bug did not exist in the earlier freeze tag or was otherwise undetected.
So now we're at a point with a critical bug that existed throughout our freeze (gtk checkbox) thus no good point to fall back to, and while this bug is fixed in later rawhide builds (fixed by an unknown component, so no idea what we could cherry pick to the old freeze tag), newer rawhide builds introduce new bugs above.
Then I tested today's rawhide, x86_64, gui, and found that it does not have the iscsi bug. I then proceeded to do a full iso compose from this package set and have continued to do testing (of the actual isos and bits) and continue to find them suitable.
So now I think we have a golden tree, although I can only test on my limited hardware and KVM. Such little testing has bitten us before, but since this time I'm testing the actual bits I feel better about the limited coverage, particularly because it's Alpha.
I'm going to stage the bits on the master mirror, and try to con some other folks into testing the isos I've generated. With luck we'll be able to release it to mirrors tomorrow and have a chance at things being ready for Tuesday. Any further issues or an inability to get a second opinion in time though will incur a slip of the Alpha release.
Thanks for your patience, and help.
On Fri, 2009-01-30 at 22:38 -0800, Jesse Keating wrote:
So now I think we have a golden tree, although I can only test on my limited hardware and KVM. Such little testing has bitten us before, but since this time I'm testing the actual bits I feel better about the limited coverage, particularly because it's Alpha.
I'm going to stage the bits on the master mirror, and try to con some other folks into testing the isos I've generated. With luck we'll be able to release it to mirrors tomorrow and have a chance at things being ready for Tuesday. Any further issues or an inability to get a second opinion in time though will incur a slip of the Alpha release.
I had to jynx myself. Shortly after sending this mail I ran across one major issue. Installation via NFS is not possible due to https://bugzilla.redhat.com/show_bug.cgi?id=483375
Rolling back nfs-utils seems to work, however to generate that into a repo I can compose form is going to take more time than I have energy for tonight.
This likely means a slip, perhaps only a two day slip, of Alpha. More info to come either later this weekend or early next week.
devel-announce@lists.fedoraproject.org