On Monday, December 16, 2019 9:56:01 AM MST Adam Williamson wrote:
> This is not accurate. You're not accounting for the time it takes to
> write the disc, and we also have to check that the media check works,
> which takes quite a while on its own.

I was accounting for that time. Writing to a disk and checking it does not
take a long time, ~20 minutes at most. This is why I said "less than an hour"
in total. Additionally, during that time, no user interaction is required,
once the process is started.

I guess I should provide some better data for those estimated time requirements. I haven't used a stopwatch during the last release cycle, but my estimate is that it takes 1-1.5 hours to check a single install medium. This includes burning the DVD, booting it in BIOS mode including the mandatory and default media check, performing the installation, and then repeating the boot and install in UEFI mode. Occasionally there are some optical reading-related issues, e.g. when a machine gets stuck because it constantly spins up and spins down the disc, having a problem trying to read some area. Sometimes the disc access gets unusably slow, just to work fine after a reboot. All the usual stuff that you come across when using CDs/DVDs. Some of that is definitely caused by our rewritable media being scratched, or DVD drives being old and the laser no longer being well calibrated. We'd have to buy new drives to improve that experience, but I don't really see much sense in that, when optical media is a niche technology nowadays (hence this proposal).

We have 2 release-blocking media, so the total time is somewhere between 2-3 hours (likely closer to 2 hours, because netinst installation is way faster due to downloading packages from the net instead of copying them from the disc). That's not the main problem, though. The main problem is that during that time, one or two of our test machines in our office is fully occupied with spinning the discs, and we can't use it for anything else. That means all other bare-metal testing needs to wait. As Adam already pointed out, sometimes we need to check the final candidate composes in a single day, i.e. in the standard 8 working hours (and yes, we often work overtime in these cases). Blocking half of our bare-metal office test machines for 2 hours out of 8 is not a small deal.

It's simple to say "no user interaction is required", but that's not completely true either. If you want to do the QA job properly, you need to have an eye on the media consistency check, because we've had issues in the past where it timed out and either considered it a pass or fail (both are incorrect). So you can't simply walk away and come back and consider it OK when it reached the installer, you really need to watch the progress in certain critical points. Once the UI is ready, it is much slower than when booting from USB. So you often spend 10, 20 seconds staring at the screen until it decides to do something. The actual installation progress is unattended yet. But you need to check it frequently to see whether it finished, so that you don't waste time of the bare metal machine standing idle. There are many more tests waiting in the queue.

The fact that this whole process is a major annoyance (it really makes you hate optical media, if you deal with this regularly) is of course contributing to the fact that we don't want to do it anymore. We're only humans. But we wouldn't have proposed the criterion change if we hadn't thought the time is right and that it is no longer an important factor for the majority of our users. We've waited very long with this proposal. And I still intend to keep testing optical media functionality from time to time, even when optical-blocking criterion is removed. But I'll do it once or twice per cycle, probably with a Beta GO compose, and not for every release candidate created.