Am 31.03.24 um 23:02 schrieb Scott Schmit:
On Sun, Mar 31, 2024 at 04:09:36PM -0400, Ben Beasley wrote:
> On 3/31/24 2:12 PM, Kevin Kofler via devel wrote:
>> But the fact is:
>>
>> What WOULD have stopped this attack: (one or more of:)
>> * Deleting ALL unit tests in %prep (and then of course not trying to run
>> them later).
> While it’s technically correct that deleting tests would have disrupted this
> specific attack, a policy of deleting and and never running upstream test
> code would have prevented me from finding and helping upstreams fix dozens
> and dozens of bugs due to accidentally faulty assumptions that turned out to
> be violated on different architectures, in different system environments, or
> with various allegedly-compatible dependency versions. There are even GCC
> bugs (miscompilations, not only failures to compile) that were discovered
> and fixed only because packages I maintain were running upstream unit and
> integration tests. Frankly, “testing the packages we ship, as built in our
> distribution, is actually bad” seems like a pretty strange and extreme
> conclusion to draw from all of this.
Deleting the tests makes no sense to me either, but it seems like a
mechanism that ensures the test code can't change the build outputs (or
a mechanism to detect that it's happened and abort the build) would
allow upstream tests to be run without compromising the integrity of the
build itself.
It would prevent the attack from being hidden in the tests, but not in
any other part of the build.
Also, I have seen build setups which encode the status of tests in the
eventual binary and as such info page or integrated bug report
generators. Often because some distros sometimes turned them off or
ships software even with failed tests and thus pushing that headache to
upstream.
Regards
Kilian Hanich