On Sun, Mar 31, 2024 at 04:09:36PM -0400, Ben Beasley wrote:
On 3/31/24 2:12 PM, Kevin Kofler via devel wrote:
But the fact is:
What WOULD have stopped this attack: (one or more of:)
- Deleting ALL unit tests in %prep (and then of course not trying to run
them later).
While it’s technically correct that deleting tests would have disrupted this specific attack, a policy of deleting and and never running upstream test code would have prevented me from finding and helping upstreams fix dozens and dozens of bugs due to accidentally faulty assumptions that turned out to be violated on different architectures, in different system environments, or with various allegedly-compatible dependency versions. There are even GCC bugs (miscompilations, not only failures to compile) that were discovered and fixed only because packages I maintain were running upstream unit and integration tests. Frankly, “testing the packages we ship, as built in our distribution, is actually bad” seems like a pretty strange and extreme conclusion to draw from all of this.
Deleting the tests makes no sense to me either, but it seems like a mechanism that ensures the test code can't change the build outputs (or a mechanism to detect that it's happened and abort the build) would allow upstream tests to be run without compromising the integrity of the build itself.