[RFR] Handling of %doc files

[Enrico Scholz]

Hello,

I have a Request For a new packaging Rule (RFR). Current rules are
saying that a package MUST work without %doc content. This implicates
that a package MUST work without dependencies introduced only by %doc
content.

In practice, people are packaging sample scripts as %doc files (with is
ok with me). Unfortunately, 'rpm' generates autodeps for these files and
the rpm maintainer does not want to change this. So, it may happen that
small packages with only a plain C binary get a dependency on 'perl'
just because a perl script was added into the %doc section. 'openvpn' is
an example for this.

Therefore, I suggest one of the following rules:

(a)  %doc files MUST not introduce new dependencies, or
(b)  %doc files MUST not be executable

Rule (a) is the more correct rule but more complicated to check.

Rule (b) is easy to check but might be too restrictive (e.g. a sh-script in
%doc will not hurt when /bin/sh is required by the core package already)
and abuses a more or less documented "feature" of rpm (autodeps are
generated for executable files only).

Enrico

[Enrico Scholz]

[e-mail] (Enrico Scholz) writes:
> Therefore, I suggest one of the following rules:
>
> (a)  %doc files MUST not introduce new dependencies, or

ok, I wrote an 'rpmlint' check for this rule:

http://ensc.de/fedora/DocFilesCheck.py.txt

(it is '*.txt' because the provider does not seem to like '*.py')

Example output:

| $ rpmlint openvpn-2.0.5-1.fc4.i386.rpm 
| E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/auth-pam.pl' creates additional dependency '/usr/bin/perl'
| E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/verify-cn' creates additional dependency '/usr/bin/perl'

Enrico

[Rex Dieter]

Enrico Scholz wrote:
> > Therefore, I suggest one of the following rules:
> > (a)  %doc files MUST not introduce new dependencies, or
>
> ok, I wrote an 'rpmlint' check for this rule:
> Example output:
> | $ rpmlint openvpn-2.0.5-1.fc4.i386.rpm 
> | E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/auth-pam.pl' creates additional dependency '/usr/bin/perl'
> | E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/verify-cn' creates additional dependency '/usr/bin/perl'

Excellent work and suggestion.  I agree with your proposed addition to 
rpmlint.

-- Rex

[Ville Skyttä]

On Wed, 2005-11-16 at 06:47 -0600, Rex Dieter wrote:
> Enrico Scholz wrote:
> > > Therefore, I suggest one of the following rules:
> > > (a)  %doc files MUST not introduce new dependencies, or
> >
> > ok, I wrote an 'rpmlint' check for this rule:
> > Example output:
> > | $ rpmlint openvpn-2.0.5-1.fc4.i386.rpm 
> > | E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/auth-pam.pl' creates additional dependency '/usr/bin/perl'
> > | E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/verify-cn' creates additional dependency '/usr/bin/perl'
>
> Excellent work and suggestion.  I agree with your proposed addition to 
> rpmlint.

Me too, thanks, and will do.  However, if included as is, I'm inclined
to make the message a warning instead of an error and soften the
phrasing a bit because of two things:

The check doesn't do recursive depsolving (nor do I think it should),
ie. it will generate noise about things pulled in by other dependencies.
While strictly speaking this is not noise but the real thing, in
practice packagers rely on deps pulling in things even if the software
_directly_ requires something itself (not at all limited to doc files),
so this would be seen as noise by many folks.  I don't have that strong
opinions on this though, we're talking about a lint tool anyway.

The other thing is that the check strips versions from dependencies
altogether, while in a perfect world it should evaluate them against
other dependencies in the package and see if it is really a new one (for
example, let's say a Perl script in /usr/bin results in a perl >=
0:500503 dependency, but a %doc file adds a perl >= 1:5.6.1 one -> no
message emitted, but should be).

Thoughts?  Enrico, did you send this upstream already?

[Panu Matilainen]

On Wed, 2005-11-16 at 21:12 +0200, Ville Skyttä wrote:
> On Wed, 2005-11-16 at 06:47 -0600, Rex Dieter wrote:
> > Enrico Scholz wrote:
> > > > Therefore, I suggest one of the following rules:
> > > > (a)  %doc files MUST not introduce new dependencies, or
> > >
> > > ok, I wrote an 'rpmlint' check for this rule:
> > > Example output:
> > > | $ rpmlint openvpn-2.0.5-1.fc4.i386.rpm 
> > > | E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/auth-pam.pl' creates additional dependency '/usr/bin/perl'
> > > | E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/verify-cn' creates additional dependency '/usr/bin/perl'
> >
> > Excellent work and suggestion.  I agree with your proposed addition to 
> > rpmlint.
>
> Me too, thanks, and will do.  However, if included as is, I'm inclined
> to make the message a warning instead of an error and soften the
> phrasing a bit because of two things:
>
> The check doesn't do recursive depsolving (nor do I think it should),
> ie. it will generate noise about things pulled in by other dependencies.
> While strictly speaking this is not noise but the real thing, in
> practice packagers rely on deps pulling in things even if the software
> _directly_ requires something itself (not at all limited to doc files),
> so this would be seen as noise by many folks.  I don't have that strong
> opinions on this though, we're talking about a lint tool anyway.
>
> The other thing is that the check strips versions from dependencies
> altogether, while in a perfect world it should evaluate them against
> other dependencies in the package and see if it is really a new one (for
> example, let's say a Perl script in /usr/bin results in a perl >=
> 0:500503 dependency, but a %doc file adds a perl >= 1:5.6.1 one -> no
> message emitted, but should be).
>
> Thoughts?  Enrico, did you send this upstream already?

Another idea: how about making rpm simply make anything marked as %doc
non-executable? That'd stop example scripts from accidentally pulling in
dependencies for good...

- Panu -

[Ville Skyttä]

On Wed, 2005-11-16 at 21:24 +0200, Panu Matilainen wrote:
> Another idea: how about making rpm simply make anything marked as %doc
> non-executable?

The initial message in this thread mentioned "Unfortunately, 'rpm'
generates autodeps for these files and the rpm maintainer does not want
to change this.", dunno if it covers fiddling with the executable bits.

[Enrico Scholz]

[e-mail] (Panu Matilainen) writes:
> Another idea: how about making rpm simply make anything marked as %doc
> non-executable? That'd stop example scripts from accidentally pulling in
> dependencies for good...

This is probably related to my suggested rule (b)... Else, I do not
think that the rpm maintainer wants to implement this

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=173087

Enrico

[Jose Pedro Oliveira]

Panu Matilainen wrote:
> Another idea: how about making rpm simply make anything marked as %doc
> non-executable? That'd stop example scripts from accidentally pulling in
> dependencies for good...

Unfortunately that doesn't resolve all the dependencies problems -
perl modules (non-executable .pm files) still pull new dependencies.

Example
==================================
Test.pm (file permissions 0644)
----------
package Test;
use Tk;
1;
----------

test.spec
-----------
...
%files
...
%doc Test.pm
...
-----------

Dependencies
----------
$ rpm -qpR ../RPMS/i386/test-0.1-1.i386.rpm
...
perl(Tk)
...
----------

jpo

[Enrico Scholz]

[e-mail] (Ville Skyttä) writes:
> > > > (a)  %doc files MUST not introduce new dependencies, or
> > >
> > > ok, I wrote an 'rpmlint' check for this rule:
> > > Example output:
> > > | $ rpmlint openvpn-2.0.5-1.fc4.i386.rpm 
> > > | E: openvpn %doc file '/usr/share/doc/openvpn-2.0.5/sample-scripts/auth-pam.pl' creates additional dependency '/usr/bin/perl'
>
> ...
> Me too, thanks, and will do.  However, if included as is, I'm inclined
> to make the message a warning instead of an error and soften the
> phrasing a bit because of two things:
>
> The check doesn't do recursive depsolving (nor do I think it should),
> ie. it will generate noise about things pulled in by other dependencies.

rpmlint operates on a single package only without having a view about
the available repository.

I could add a configurable whitelist, e.g. always allow /bin/sh,
/usr/bin/env or make it a warning only. But rpmlint creates other
false positives too and because there is not automatic process
behind, this would be a cosmetic effect only.
> The other thing is that the check strips versions from dependencies
> altogether, while in a perfect world it should evaluate them against
> other dependencies in the package and see if it is really a new one
> (for example, let's say a Perl script in /usr/bin results in a perl >=
> 0:500503 dependency, but a %doc file adds a perl >= 1:5.6.1 one -> no
> message emitted, but should be).

Obviously, solving this isssue would be a very interesting task. But it
would be also complicated because you have to deal with ranges; e.g. a
%doc file could require 'perl >= 5.6.1 perl < 5.9.0' while rest of the
package requires 'perl >= 5.6.0 perl <= 5.8.5'. This would pull in a dep
of 'perl > 5.8.5 perl < 5.9.0'. I will have to sleep a night over it to
see how this can be implemented...

But these versioned deps are pretty useless with the current rpm epoch
mechanism. The perl autodep generator does not know about existing
epochs and rpm does not allow to require a certain upstream version
(non-existing epoch is always assumed as '0')
> Thoughts?  Enrico, did you send this upstream already?

Yes, sent it to flepied at mandriva.

Enrico

[Ville Skyttä]

On Wed, 2005-11-16 at 21:23 +0100, Enrico Scholz wrote:
> [e-mail] (Ville Skyttä) writes:
> > The check doesn't do recursive depsolving (nor do I think it should),
> > ie. it will generate noise about things pulled in by other dependencies.
>
> rpmlint operates on a single package only without having a view about
> the available repository.

Of course.
> But rpmlint creates other
> false positives too and because there is not automatic process
> behind, this would be a cosmetic effect only.

Well, past and existing bad behaviour is IMO not a reason to add more of
it, and a lot of rpmlint _is_ about cosmetic issues :].  I have this now
in my local version:

printWarning(pkg, "%%doc file '%s' creates possible additional dependency '%s'" % (f, dep))
> Obviously, solving this isssue would be a very interesting task. But it
> would be also complicated because you have to deal with ranges;

Sure.  Dunno if it's worth spending much time on though.
> But these versioned deps are pretty useless with the current rpm epoch
> mechanism. The perl autodep generator does not know about existing
> epochs and rpm does not allow to require a certain upstream version
> (non-existing epoch is always assumed as '0')

Actually it does know some, but the implementation is so bitrotten that
it has been practically useless already for a while, see around line 200
in /usr/lib/rpm/perl.req.
> > Thoughts?  Enrico, did you send this upstream already?
>
> Yes, sent it to flepied at mandriva.

Cool, let me know if you get a response, my rpmlint patch pile is
growing painfully large and is about to grow some more, and I haven't
heard back from him in a few monthts :(

[Tom Callaway]

On Wed, 2005-11-16 at 13:14 +0100, Enrico Scholz wrote:
> > Therefore, I suggest one of the following rules:
> >
> > (a)  %doc files MUST not introduce new dependencies

Seems reasonable to me. I'll propose it to FESCO for approval.

~spot