Hey all,
I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2].
I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game."
For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner."
Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that.
I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma.
Any ideas welcome and appreciated!
[1]: https://pagure.io/fesco/issue/2873 [2]: https://bugzilla.redhat.com/2177287
-- 真実はいつも一つ！/ Always, there's only one truth!
On Thu, 2023-04-20 at 08:20 -0400, Neal Gompa wrote:
Hey all,
I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2].
I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game."
For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner."
Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that.
I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma.
Any ideas welcome and appreciated!
I'm against this. We have never blocked the OS on proprietary third- party applications. I don't think it's a path we want to go down.
I'm in favour of testing common third-party stuff before release and fixing it if we can, but we have always said we will not block Fedora on this, and I don't think we should change that.
(I did actually test Steam, but I used the flatpak version, not RPM...)
I'm fixing the zenity bug, BTW.
On Thu, Apr 20, 2023 at 2:28 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Thu, 2023-04-20 at 08:20 -0400, Neal Gompa wrote:
Hey all,
I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2].
I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game."
For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner."
Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that.
I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma.
Any ideas welcome and appreciated!
I'm against this. We have never blocked the OS on proprietary third- party applications. I don't think it's a path we want to go down.
I'm in favour of testing common third-party stuff before release and fixing it if we can, but we have always said we will not block Fedora on this, and I don't think we should change that.
(I did actually test Steam, but I used the flatpak version, not RPM...)
I'm fixing the zenity bug, BTW.
Is there a way we can add regular testing for it without making it a blocker then?
On Thu, 2023-04-20 at 17:38 -0400, Neal Gompa wrote:
On Thu, Apr 20, 2023 at 2:28 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Thu, 2023-04-20 at 08:20 -0400, Neal Gompa wrote:
Hey all,
I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2].
I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game."
For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner."
Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that.
I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma.
Any ideas welcome and appreciated!
I'm against this. We have never blocked the OS on proprietary third- party applications. I don't think it's a path we want to go down.
I'm in favour of testing common third-party stuff before release and fixing it if we can, but we have always said we will not block Fedora on this, and I don't think we should change that.
(I did actually test Steam, but I used the flatpak version, not RPM...)
I'm fixing the zenity bug, BTW.
Is there a way we can add regular testing for it without making it a blocker then?
Sure, we have lots of these. It's what the "Optional" milestone in the test matrices means: any test whose milestone is "Optional" is...optional.
You can write up a test case (or more than one) for this and propose adding it to the desktop matrix as an Optional row (or, well, probably several rows), I think that would be fine.
On Thu, Apr 20, 2023 at 11:39 PM Neal Gompa ngompa13@gmail.com wrote:
Is there a way we can add regular testing for it without making it a blocker then?
For popular third-party software, I think this testing occurs quite naturally - people just use it. E.g. Steam, Chrome, VSCode, etc. I've been running Steam on F38 since Beta (and it helped me to discover an issue in mutter, which was later accepted as a blocker - but not because of Steam, but because of general issues). But if you want to have an explicit test case, Adam described how to do it. We could also have a test day for popular third-party software, if it makes sense.
Note that we had proprietary software-related blockers in the past. We accepted the EAC glibc issue as a blocker in one of the recent releases. We also accepted the RPM signatures change as a blocker for F38. But in these cases, it was always accepted as an exceptional blocker confirmed by FESCo. I guess it makes sense to use this exceptional process for these cases (blocking on issues with third-party software, which can affect our user base badly). We could talk about some general rule related to popular third-party software, but that would again need to go through FESCo. At the moment, we only block on our own code in our own workflows, with a few very specific exceptions (boot usb creation, dual boot configuration, cloud images deployment).
On Tue, 2023-04-25 at 13:49 +0200, Kamil Paral wrote:
On Thu, Apr 20, 2023 at 11:39 PM Neal Gompa ngompa13@gmail.com wrote:
Is there a way we can add regular testing for it without making it a blocker then?
For popular third-party software, I think this testing occurs quite naturally - people just use it. E.g. Steam, Chrome, VSCode, etc. I've been running Steam on F38 since Beta (and it helped me to discover an issue in mutter, which was later accepted as a blocker - but not because of Steam, but because of general issues). But if you want to have an explicit test case, Adam described how to do it. We could also have a test day for popular third-party software, if it makes sense.
To be fair, we did miss https://bugzilla.redhat.com/show_bug.cgi?id=2177287 (I missed it because I run Steam via Flatpak, not RPM). So we do have scope to improve there. I do think testing commonly used third-party stuff is a good idea, to be clear, and I'm all in favor of adding optional test cases for it.
On Tue, Apr 25, 2023 at 12:18 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2023-04-25 at 13:49 +0200, Kamil Paral wrote:
On Thu, Apr 20, 2023 at 11:39 PM Neal Gompa ngompa13@gmail.com wrote:
Is there a way we can add regular testing for it without making it a blocker then?
For popular third-party software, I think this testing occurs quite naturally - people just use it. E.g. Steam, Chrome, VSCode, etc. I've been running Steam on F38 since Beta (and it helped me to discover an issue in mutter, which was later accepted as a blocker - but not because of Steam, but because of general issues). But if you want to have an explicit test case, Adam described how to do it. We could also have a test day for popular third-party software, if it makes sense.
To be fair, we did miss https://bugzilla.redhat.com/show_bug.cgi?id=2177287 (I missed it because I run Steam via Flatpak, not RPM). So we do have scope to improve there. I do think testing commonly used third-party stuff is a good idea, to be clear, and I'm all in favor of adding optional test cases for it.
And I'm fine with us adding some optional test cases for this. Would running a test day each cycle also be possible?
On Tue, 2023-04-25 at 12:20 -0400, Neal Gompa wrote:
On Tue, Apr 25, 2023 at 12:18 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2023-04-25 at 13:49 +0200, Kamil Paral wrote:
On Thu, Apr 20, 2023 at 11:39 PM Neal Gompa ngompa13@gmail.com wrote:
Is there a way we can add regular testing for it without making it a blocker then?
For popular third-party software, I think this testing occurs quite naturally - people just use it. E.g. Steam, Chrome, VSCode, etc. I've been running Steam on F38 since Beta (and it helped me to discover an issue in mutter, which was later accepted as a blocker - but not because of Steam, but because of general issues). But if you want to have an explicit test case, Adam described how to do it. We could also have a test day for popular third-party software, if it makes sense.
To be fair, we did miss https://bugzilla.redhat.com/show_bug.cgi?id=2177287 (I missed it because I run Steam via Flatpak, not RPM). So we do have scope to improve there. I do think testing commonly used third-party stuff is a good idea, to be clear, and I'm all in favor of adding optional test cases for it.
And I'm fine with us adding some optional test cases for this. Would running a test day each cycle also be possible?
I don't see any reason why not! Sumantro puts out a call for test days each cycle, or you could just go ahead and file a ticket for one at https://pagure.io/fedora-qa/issues now (give it the 'test days' tag and the Fedora 39 milestone). Sumantro is in charge of that process, and there's a lot of flexibility in terms of who is mainly responsible for running each individual test day; you can certainly take an active role in setting it up and running it if you'd like to.
On Thu, Apr 20, 2023 at 1:20 PM Neal Gompa ngompa13@gmail.com wrote:
Hey all,
I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2].
I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game."
For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner."
Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that.
I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma.
Any ideas welcome and appreciated!
So this is a proprietary platform, we don't even block on something like the binary nvidia drivers, so I'm not sure how we can block on something that is mostly out of our control.
If there is a change in an upstream kernel that triggers a crash do we have to await a fix from them or roll back to an older kernel?
Also are there free games that can be played for testing or reproducing bugs that don't require you to put in a credit card to be able to use them on Steam (sorry, I've never used it).
Peter
Being that Fedora is so strongly for open source sodtware, I personally think any blockers shouldn't be from or for any 3rd party proprietary software like Steam (or NVIDIA like mentioned). I would expect any blockers to be applications or packages that come with Fedora by default. I'd be happy to test Steam as I have a decent Steam library but I don't think it should be a blocker by any means. New to QA Fedora, can't wait to get involved.
On Thu, Apr 20, 2023, 16:15 Peter Robinson pbrobinson@gmail.com wrote:
On Thu, Apr 20, 2023 at 1:20 PM Neal Gompa ngompa13@gmail.com wrote:
Hey all,
I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2].
I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game."
For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner."
Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that.
I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma.
Any ideas welcome and appreciated!
So this is a proprietary platform, we don't even block on something like the binary nvidia drivers, so I'm not sure how we can block on something that is mostly out of our control.
If there is a change in an upstream kernel that triggers a crash do we have to await a fix from them or roll back to an older kernel?
Also are there free games that can be played for testing or reproducing bugs that don't require you to put in a credit card to be able to use them on Steam (sorry, I've never used it).
Peter _______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Thu, Apr 20, 2023 at 5:14 PM Peter Robinson pbrobinson@gmail.com wrote:
On Thu, Apr 20, 2023 at 1:20 PM Neal Gompa ngompa13@gmail.com wrote:
Hey all,
I would like for us to have some testing criteria around gaming and Steam so that we can ensure we're offering a working gaming experience in Fedora Linux releases. This is motivated by the issue we had in the F37 cycle where glibc broke popular multiplayer games[1]. I was reminded of this when I launched Steam today on F38 and zenity crashed[2].
I would like to propose the following criterion for Steam itself as a Beta Blocker bug: "Steam MUST be able to be installed and have its basic functionality work with no visible errors. Basic functionality for Steam includes: logging into a Steam account and installing a Windows/Proton game and a Linux/SteamOS native game."
For gaming itself, I would like to propose the following criterion as a Final Blocker bug: "Steam games identified as Deck Verified by ProtonDB.com (see https://www.protondb.com/explore?selectedFilters=whitelisted) MUST launch and let the user play the game. This criterion is not intended to judge performance, merely accessibility. At least one Windows/Proton game and one Linux/SteamOS native game MUST be tested in this manner."
Now, the tricky issue here is how to wordsmith the check for anti-cheat systems. I don't want to specifically call out just EAC, but I also don't know of a good mix of games with different anti-cheats. The important thing is to catch regressions and see if it's something we can resolve. In the EAC case from F37, it was easy for us to deal with, but if it's genuinely broken in a way we can't deal with it on the Fedora side, I don't know what we're supposed to do, so I'm wary of doing some kind of blocker criterion for that.
I'd also like this to be imposed on both release-blocking desktops: GNOME and KDE Plasma.
Any ideas welcome and appreciated!
So this is a proprietary platform, we don't even block on something like the binary nvidia drivers, so I'm not sure how we can block on something that is mostly out of our control.
If there is a change in an upstream kernel that triggers a crash do we have to await a fix from them or roll back to an older kernel?
Also are there free games that can be played for testing or reproducing bugs that don't require you to put in a credit card to be able to use them on Steam (sorry, I've never used it).
Steam has a toggle in the storefront for "free-to-play" games that you can add to your Steam library without entering a credit card.
-
Adam Williamson -
John P -
Kamil Paral -
Neal Gompa -
Peter Robinson