[Fedora-packaging] Introducing a directory for installed test programs

[Tom Callaway]

On 02/06/2012 03:58 PM, Julio Merino wrote:
> What do you mean by "multi-lib heartburn in RPM"?  (Sorry, not familiar
> from the term and a couple of Google searches did't provide much
> insight.)  FWIW, I have a preliminary atf.spec file here that provides
> subpackages for libraries, tests and binaries and things seem to be fine
> to my untrained eye.  I can share this file if it helps at all.

No, what I meant here is that RPM, when dealing with multi-arch support
(essentially the ability to simultaneously foo.i686.rpm and
foo.x86_64.rpm containing the same set of files, just built for the
different architecture targets), won't understand /usr/tests (or
/usr/libexec/tests for that matter).

However, if you want to be able to support this use-case, you'll need to
do something here to enable it, such as requiring that architecture be
embedded in the naming scheme, e.g. /usr/tests/xpdf-x86_64/

I'm not sure how much value there is in running the test suites for
supported architectures aside from the default though. If there isn't a
desire for this, we just need to explicitly state that packages
containing test-suites cannot be multi-arch.

>> You probably should open a ticket with the FHS working group:
>> https://bugs.linuxfoundation.org/ (product: FHS) and explain all of this
>> to them so that it has a chance of being officially added in the next
>> major FHS revision, whenever that may be.
> 
> OK, I can do that.  Wouldn't it be easier to illustrate the point /
> convince them if there was existing practice somewhere?  (Not sure if
> BSD flavors count in this case!)

Honestly, I don't know. Common sense says yes, but common sense has yet
to appear in the FHS revision process. ;)

Fedora does not have to wait for the FHS to approve it, but we do want
the things that we adopt to also be driven through the FHS approval
process with the hopes that we can "merge" these exceptions "upstream".

>> Above and beyond your proposal, the only other way I can think of
>> handling this would be an elaborate namespacing of the test binaries in
>> /usr/bin, e.g. /usr/bin/test-$pkg-$testname , and while that would be a
>> solution that would permit a strict compliance with the FHS as it exists
>> today, I'm not very excited about it and am not seriously recommending
>> it as an alternative.
> 
> I don't think this would be an appropriate layout, and it doesn't play
> well with the structure of atf (let aside the clutter that would appear
> in /usr/bin).  Let me provide some additional details of how the tests
> tree is structured.

[LAYOUT INFO SNIPPED]

Yes, I understand this. If we were to propose such a layout, we'd put
all the other stuff in say, /usr/share/tests/$pkg/, then provide
symlinks for the binaries into that hierarchy.

But like I said, I don't really like it.

> A little detail to note above is that in a project with a good amount of
> test coverage, there will potentially be a test program per each source
> file.  This means that the testsdir contains lots of binaries, and you
> don't want that many, potentially with conflicting names, anywhere in
> your path!  To illustrate the point, check out this output page which
> contains the results of the execution of the NetBSD test suite:

This further illustrates why I don't really like it. I was simply
stating it because inevitably, someone else would, and I wanted it on
the record has having been considered and discarded. ;)

~tom

==
Fedora Project