On Sun, Mar 31, 2024 at 04:09:36PM -0400, Ben Beasley wrote:
On 3/31/24 2:12 PM, Kevin Kofler via devel wrote:
> But the fact is:
>
> What WOULD have stopped this attack: (one or more of:)
> * Deleting ALL unit tests in %prep (and then of course not trying to run
> them later).
While it’s technically correct that deleting tests would have disrupted this
specific attack, a policy of deleting and and never running upstream test
code would have prevented me from finding and helping upstreams fix dozens
and dozens of bugs due to accidentally faulty assumptions that turned out to
be violated on different architectures, in different system environments, or
with various allegedly-compatible dependency versions. There are even GCC
bugs (miscompilations, not only failures to compile) that were discovered
and fixed only because packages I maintain were running upstream unit and
integration tests. Frankly, “testing the packages we ship, as built in our
distribution, is actually bad” seems like a pretty strange and extreme
conclusion to draw from all of this.
Deleting the tests makes no sense to me either, but it seems like a
mechanism that ensures the test code can't change the build outputs (or
a mechanism to detect that it's happened and abort the build) would
allow upstream tests to be run without compromising the integrity of the
build itself.