Differences between koji and mock rawhide environments?

[Panu Matilainen]

On 11/09/2011 08:49 PM, Jim Meyering wrote:
> Richard Shaw wrote:
>> On Wed, Nov 9, 2011 at 11:18 AM, Tom Lane<tgl at redhat.com>  wrote:
>>> postgresql is currently failing to rebuild in rawhide:
>>> http://koji.fedoraproject.org/koji/taskinfo?taskID=3499379
>>> This seems quite repeatable, in koji, but the package builds fine on my
>>> workstation under mock's fedora-rawhide-x86_64 and fedora-rawhide-i386
>>> environments.  Shouldn't this have exactly the same results?  Can anyone
>>> suggest how I might reproduce the failure someplace where I can debug it?
>>
>> I'm having a similar problem with OpenImageIO. I can build against TBB
>> in a mock environment but it fails in koji.
>
> I had a similar problem with iwhd.
> All tests passed locally(F16), but "make check" in koji would always fail.
> I ended up leaving %check disabled in the spec file.
>
> Need to recheck...

I've seen some similar incidents too:

At some point rpm test-suite was failing because of DNS issues on (some 
of) the builders, causing unexpected "Could not canonicalize hostname" 
spew in test output.

Sqlite has had a couple of rather entertaining cases: one had to do with 
the filesystem and how many syncs sqlite tests were expecting to count 
on it, this was different the builder vs fedora-of-the-time era. Another 
more recent one was that sqlite test-suite tries to test failure when 
opening file descriptors, and in order to do this it opens ~2000 fds. 
This starts failing on "normal" systems but on the builders the max 
number of fds is much higher, causing the test to fail because there was 
no failure.

The only way to figure these out (that I know of) is to have some logic 
in %check to detect test-suite failure and dump out what failure logs 
you might have into koji's build.log so you have *some* clue where to 
look. Eg in rpm I have this (relying on "make check" return code wont 
work thanks to something in autofoo masking it)

%if %{with check}
%check
make check
[ "$(ls -A tests/rpmtests.dir)" ] && cat tests/rpmtests.log
%endif

	- Panu -