Re: [Fedora-legal-list] Trivy for licenses

[Maxwell G]

On Sun Mar 3, 2024 at 20:22 +0100, Philippe Ombredanne wrote:
> Hi  Maxwell:

Hi Philippe,
> On Sun, Mar 3, 2024, Maxwell G wrote:
> > Has anyone every used trivy [1] to scan for licenses? It appears more
> > robust and better maintained than askalono-cli and can detect files with
> > multiple licenses and licenses embedded in file headers.  I have been
> > running it with "trivy fs --scanners license --license-full ."
> >
> > [1] https://github.com/aquasecurity/trivy
>
> IMHO trivy is not a robust tool for license detection from me trying it.

I am not necessarily looking for the most robust tool for license
detection. I am just looking for a relatively performant and reasonably
accurate tool to scan a tree of Go modules for license files for the
go-vendor-tools [1] project that I am working on.

I evaluated askalono-cli and trivy for this purpose, and they both
fulfil that criteria. I implemented support for both of them and added
an option to choose which to use.

The Fedora legal docs describes askalono as:

It is most useful for quick analysis of packages coming out of
     ecosystems featuring projects known to have (1) highly standardized
     approaches to layout of license information (it specifically looks
     only for files that are named LICENSE or COPYING or some obvious
     variant on those), (2) generally simple license makeup, and (3)
     cultural preferences for a highly limited set of licenses (for
     example, Rust crates that don’t bundle legacy C code, Go modules,
     or Node.js npm packages).

That is exactly what I am using it for. Trivy does a better job at
detecting license files paths than asaklono and can also handle files
with multiple licenses and some license headers. My code already checks
that there is at least one license file in each Go module, so if one is
missing, the go-vendor-tools license checker will fail and require the
user to take manual action.

The Go ecosystem is relatively standardized in terms of licensing, so I
do not feel the need to use a tool like scancode which analyzes every
single file and takes a very long time to run.
> It is mostly based on google/licenseclassifier which had a single
> commit in the last 17 months, and this means this is not more
> maintained than askalono (and frankly both are fairly lightweight

<snip>
> I would not rely on
> these for anything serious and certainly not to scan code for license
> prior to its inclusion in Fedora. tools for license detection).

I am striving for "reasonably sure" that all license texts are accounted
for as opposed to spending 45 minutes performing a detailed license
files for each package.
> If you want robust license detection, consider using ScanCode [2] and
> Scancode.io [3] for more complex pipelines. Both are tools that I
> co-maintain and are considered as better tools for this. Do not
> hesitate to reach out for help!

I will definitely spend more time playing with  scancode-toolkit, but I
worry about the amount of time it takes to run on a large go vendor tree
and that it has not been packaged for Fedora yet---it has a lot of
Python dependencies. I opened [2] to track implementing a scancode
backend for go-vendor-tools. I will be sure to let you know if I have
any questions!

[1] https://gitlab.com/gotmax23/go-vendor-tools/
[2] https://gitlab.com/gotmax23/go-vendor-tools/-/issues/15

Thanks,
Maxwell