New package license review proposal

[Jason L Tibbitts III]

I figured I'd start with this list and broaden to devel@ if people think
it's a good idea.

In doing (very) many package reviews, I've found one of the most
time-consuming things to be doing a proper license review.  Even
something simple with, say, an LGPLv2+ notice can get complicated when a
single GPLv2 file sneaks in.  It's complicated enough that I suspect in
many cases license review just isn't being done.  Plus the complexities
of licensing coupled with the complexities of our packaging guidelines
really poses a high barrier for anyone wanting to do proper license
reviews.

So I'm proposing that we separate the roles of the package reviewer from
the license reviewer, allowing someone who wants to concentrate on
licensing do participate in the review process without having to deal
with the complexities of the packaging guidelines (or even building the
software).  This isn't intended to preclude someone from taking a new
request and doing both packaging and licensing review, but simply to
allow folks to go through the existing reviews and indicate that they've
been checked for licensing issues so that someone could later go through
and review the packaging without having to struggle over the licensing.

I propose to handle this with a simple entry in the whiteboard and a
comment by the reviewer.  I can add a report under
http://fedoraproject.org/PackageReviewStatus listing tickets which need
license review, and am prepared to write a utility to facilitate things
as much as possible.  When a license question comes up, FE-Legal would
be blocked just as it is now.  (Apologies to spot.)  I would ask for
help from others to document the license review process as much as
possible.

I think in the end that with a dedicated team of folks doing license
checks, we can get the review process moving a bit quicker and cut down
on incidences of unwanted things leaking into the distro that have to be
cleaned up later.

 - J<

[Tom Callaway]

On 01/16/2010 05:37 PM, Jason L Tibbitts III wrote:
>  I figured I'd start with this list and broaden to devel@ if
> people think
>  it's a good idea.
>  
>  In doing (very) many package reviews, I've found one of the most
>  time-consuming things to be doing a proper license review.  Even
>  something simple with, say, an LGPLv2+ notice can get complicated when a
>  single GPLv2 file sneaks in.  It's complicated enough that I suspect in
>  many cases license review just isn't being done.  Plus the complexities
>  of licensing coupled with the complexities of our packaging guidelines
>  really poses a high barrier for anyone wanting to do proper license
>  reviews.
>  
>  So I'm proposing that we separate the roles of the package reviewer from
>  the license reviewer, allowing someone who wants to concentrate on
>  licensing do participate in the review process without having to deal
>  with the complexities of the packaging guidelines (or even building the
>  software).  This isn't intended to preclude someone from taking a new
>  request and doing both packaging and licensing review, but simply to
>  allow folks to go through the existing reviews and indicate that they've
>  been checked for licensing issues so that someone could later go through
>  and review the packaging without having to struggle over the licensing.
>  
>  I propose to handle this with a simple entry in the whiteboard and a
>  comment by the reviewer.  I can add a report under
>  http://fedoraproject.org/PackageReviewStatus listing tickets which need
>  license review, and am prepared to write a utility to facilitate things
>  as much as possible.  When a license question comes up, FE-Legal would
>  be blocked just as it is now.  (Apologies to spot.)  I would ask for
>  help from others to document the license review process as much as
>  possible.
>  
>  I think in the end that with a dedicated team of folks doing license
>  checks, we can get the review process moving a bit quicker and cut down
>  on incidences of unwanted things leaking into the distro that have to be
>  cleaned up later. 
Seems reasonable. We might be able to do a FAD to train some people on
looking at licenses to jump start this process.

We might also consider deploying something like FOSSology (which I've
had on my todo list for ages). Not as a replacement for this, but as an
additional helper tool.

http://fossology.org/

~spot

[Luis Villa]

On Sat, Jan 16, 2010 at 2:46 PM, Tom "spot" Callaway
<tcallawa(a)redhat.com&gt; wrote:
>  On 01/16/2010 05:37 PM, Jason L Tibbitts III wrote:
> > I figured I'd start with this list and broaden to devel@ if people think
> > it's a good idea.
> >
> > In doing (very) many package reviews, I've found one of the most
> > time-consuming things to be doing a proper license review.  Even
> > something simple with, say, an LGPLv2+ notice can get complicated when a
> > single GPLv2 file sneaks in.  It's complicated enough that I suspect in
> > many cases license review just isn't being done.  Plus the complexities
> > of licensing coupled with the complexities of our packaging guidelines
> > really poses a high barrier for anyone wanting to do proper license
> > reviews.
> >
> > So I'm proposing that we separate the roles of the package reviewer from
> > the license reviewer, allowing someone who wants to concentrate on
> > licensing do participate in the review process without having to deal
> > with the complexities of the packaging guidelines (or even building the
> > software).  This isn't intended to preclude someone from taking a new
> > request and doing both packaging and licensing review, but simply to
> > allow folks to go through the existing reviews and indicate that they've
> > been checked for licensing issues so that someone could later go through
> > and review the packaging without having to struggle over the licensing.
> >
> > I propose to handle this with a simple entry in the whiteboard and a
> > comment by the reviewer.  I can add a report under
> > http://fedoraproject.org/PackageReviewStatus listing tickets which need
> > license review, and am prepared to write a utility to facilitate things
> > as much as possible.  When a license question comes up, FE-Legal would
> > be blocked just as it is now.  (Apologies to spot.)  I would ask for
> > help from others to document the license review process as much as
> > possible.
> >
> > I think in the end that with a dedicated team of folks doing license
> > checks, we can get the review process moving a bit quicker and cut down
> > on incidences of unwanted things leaking into the distro that have to be
> > cleaned up later.
>
>  Seems reasonable. We might be able to do a FAD to train some people on
>  looking at licenses to jump start this process.
>
>  We might also consider deploying something like FOSSology (which I've
>  had on my todo list for ages). Not as a replacement for this, but as an
>  additional helper tool.
>
>  http://fossology.org/ 
Something like fossology seems like it would save everyone involved a
ton of time and pain; frankly, potentially enough that it would remove
most of the objections Jason has highlighted.

Luis

[Tom Callaway]

On 01/16/2010 05:55 PM, Luis Villa wrote:
>  Something like fossology seems like it would save everyone involved
> a
>  ton of time and pain; frankly, potentially enough that it would remove
>  most of the objections Jason has highlighted. 
Well, until I'm confident that fossology is thorough and accurate
enough, I don't want to jump to conclusions.

~spot

[Luis Villa]

On Sat, Jan 16, 2010 at 7:34 PM, Tom "spot" Callaway
<tcallawa(a)redhat.com&gt; wrote:
>  On 01/16/2010 05:55 PM, Luis Villa wrote:
> > Something like fossology seems like it would save everyone involved a
> > ton of time and pain; frankly, potentially enough that it would remove
> > most of the objections Jason has highlighted.
>
>  Well, until I'm confident that fossology is thorough and accurate
>  enough, I don't want to jump to conclusions. 
I've played with it some and been fairly impressed; it is inevitably
imperfect (like any software) but so are human reviewers. That said,
it might just be too much work to implement- it did not strike me as
easy to setup or install, and if there aren't already
automated/tool-assisted steps in the review process (I don't know) it
might be a pain to integrate into the workflow.

I know lack of reviewers is already a serious bottleneck in the
process; would having a separate cadre of license reviewers mean more
delays?

Luis

[Jason L Tibbitts III]

> >>>> "LV" == Luis Villa <luis(a)tieguy.org&gt;
> writes: 
LV> I know lack of reviewers is already a serious bottleneck in the
LV> process; would having a separate cadre of license reviewers mean
LV> more delays?

How could it possibly be so, unless a separate license review was
somehow made a blocker to the process?  That's not what's being
proposed.  At worse, nobody would do separate license reviews and the
regular package reviewers would continue as they do now.  At best, all
packages would be checked for license issues before the regular package
review happens, and package reviewers can avoid worrying about license
issues.  Reality will probably be somewhere in between.  Any separate
license review takes work off of the already far overworked package
reviewers; I can't imagine how that could hurt.

I don't know how fossology works, but if there's any way I can automate
calling it then I'll be happy to look into it.  Currently automation
would be limited to a tool that would pick a ticket which needs license
review, pull down the most recent posted srpm, unpack it and drop you
into a shell to look around, and automatically updating bugzilla.
Plenty of possibility to hang other tools off of that, except that I
don't really know of any that could be run.

 - J<

[Luis Villa]

On Sat, Jan 16, 2010 at 7:53 PM, Jason L Tibbitts III <tibbs(a)math.uh.edu&gt; wrote:
> >>>>> "LV" == Luis Villa
> <luis(a)tieguy.org&gt; writes:
>
>  LV> I know lack of reviewers is already a serious bottleneck in the
>  LV> process; would having a separate cadre of license reviewers mean
>  LV> more delays?
>
>  How could it possibly be so, unless a separate license review was
>  somehow made a blocker to the process? 
>  That's not what's being
>  proposed.  At worse, nobody would do separate license reviews and the
>  regular package reviewers would continue as they do now.  At best, all
>  packages would be checked for license issues before the regular package
>  review happens, and package reviewers can avoid worrying about license
>  issues.  Reality will probably be somewhere in between.  Any separate
>  license review takes work off of the already far overworked package
>  reviewers; I can't imagine how that could hurt. 
Ah, I understand better now- you mean this as an alternative; if the
license reviewers don't have bandwidth, the regular reviewers would
still have it on their plate before the package got submitted?

>  I don't know how fossology works, but if there's any way I
> can automate
>  calling it then I'll be happy to look into it.  Currently automation
>  would be limited to a tool that would pick a ticket which needs license
>  review, pull down the most recent posted srpm, unpack it and drop you
>  into a shell to look around, and automatically updating bugzilla.
>  Plenty of possibility to hang other tools off of that, except that I
>  don't really know of any that could be run. 
Fossology is just a pile of scripts (perl maybe? I don't recall) that
basically grep the hell out of a package and build licensing data
based on what it finds; for large codebases the reports can get fairly
elaborate. It has a large library of known license patterns, etc. So
it should be able to tell you with fairly high certainty 'this package
is licensed under license A, with a smattering of license X, Y and Z.'

What I suspect it won't do (and maybe someone should either talk with
the fossology folks to confirm) is deal with the cases of bizarre or
one-off licenses that seem to be stumbled upon fairly often here.
Perhaps they could (or already do) flag files that contain keywords
like 'copyright' or 'license' but don't contain a recognized license,
for further inspection. (I imagine they also don't have as broad a
database of licenses as Fedora does, but that is easier to fix.) If
they can be talked into adding that (or someone from fedora can hack
it in) then my guess is that it would prove a fairly efficient way to
vet packages for licensing conditions.

Luis