Dear Peter Boy,

I didn't yet test the server variant of the x86_64 architecture of Fedora 35 RC 1.2 manually; however I have tested some of the base images for Fedora workstation 35 RC 1.1 for the MATE spin. All of which passed. Also I tested the "Everything netinst" for Fedora Workstation 35 RC 1.2 that passed as well. Majority of the tests are ran by Coconut, one of the bots. Mostly through openQA. If required I can test the server server variant of the x86_64 architecture of Fedora 35 RC 1.2 manually. I do think that manual testing is better as bots are able to sometimes become faulty. 

On Sun, 31 Oct 2021 at 11:50, Peter Boy <pboy@uni-bremen.de> wrote:
We are officially distributing a Fedora Server Edition disk image for ARM, typically used for SBCs. I just tried to install this image on a Radxa Rock Pi 4 and failed miserably. That SBC device is part of the installation procedure provided by the ARM team. With the F34 image I ended up with a black screen, no display, no chance to get the device working in the normal way (I got it to work in tricky detours). With F35 Beta I got a working display, but no network connection. I haven't found a solution for that yet.

And then I remembered F33, where it was discovered by chance (!) at the last minute, that in the x86_64 version Cockpit did not work properly (and probably also in the aarch64 default install version, we didn't and probably couldn’t even check that afterwards). We have not solved the underlying issue for this until today, although the factual reason has been clear for a long time (issue #32).

Similar problems had emerged after the release with the switch to systemd-resolved, where "surprisingly" problems with some server configurations were encountered, especially with elaborate use of KVM virtualization on a server.



These events are clear violations of our release criteria. In principle, a release must be at least successfully installable and basically working.



For me, this brings up a few questions:

(1)
Did any of us actually have Fedora Beta installed on a ARM device (SBC or other) using the image file and on which model?
Or are we just flying blind with that deliverable?

(2)
What about the two aarch64 install iso files that target SBSA hardware? Does anyone have such a beast at hand and tested the isos? Or is that (also) flying blind?

(3)
And what about x86_64 installation? Most of us will be in possession of this hardware. But what about tests? When I started a discussion about testing a year ago, Adam reassured us, release building and testing is heavily automated these days. No need to worry too much about testing it manually ourselves. But did anyone of us at least worry a bit?


I would like to discuss how we can gain some improvement.

Regarding arch architecture, could some kind of cooperation with arm group be beneficial? And what can we offer? Is it testing? Is it documentation? Or something else?
If we want to take SBC devices seriously, perhaps we could identify a few reference systems where we can actually ensure, i.e. test, whether a successful installation and basic functionality is met. If we don't want that, we should not distribute the image file under the Fedora Server Edition branding.

And in terms of our "main" architecture, we should agree on what QA actually expects in terms of non-automated testing and how we want to tackle that in the next round. Based on Adam's comments, we have relied heavily on automation for the last few releases. So it's not clear to me what, for example, the page
https://fedoraproject.org/wiki/Test_Results:Fedora_35_RC_1.2_Server means in detail in terms of work.





Happy Sunday
Peter






_______________________________________________
server mailing list -- server@lists.fedoraproject.org
To unsubscribe send an email to server-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/server@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure