Static Analysis: some UI ideas

[David Malcolm]

I've been experimenting with some UI ideas for reporting static analysis
results: I've linked to two different UI reports below.

My hope is that we'll have a server in the Fedora infrastructure for
browsing results, marking things as false positives etc.

However, for the purposes of simplicity during experimentation I'm
simply building static HTML reports.

My #1 requirement when I'm viewing static analysis results is that I
want to *see the code* with the report, so I've attempted to simply show
the code with warnings shown inline.

Note also that when we have a server we can do all kinds of
auto-filtering behaviors so that e.g. package maintainers only see
warnings from tests that have decent signal:noise ratio (perhaps with
other warnings greyed out, or similar).


Results of an srpm build
========================
The first experimental report can be seen here:
http://fedorapeople.org/~dmalcolm/static-analysis/2013-02-01/policycoreutils-2.1.13-27.2.fc17.src.rpm-001.html

It shows warnings from 4 different static analyzers when rebuilding a
particular srpm (policycoreutils-2.1.13-27.2.fc17).  There's a summary
table at the top of the report showing for each source files in the
build which analyzers found reports (those that found any are
highlighted in red).  Each row has a <a> linking you to a report on each
source file.  Those source files that have issues have a table showing
the issues, with links to them.  The issue are shown inline within the
syntax-colored source files.

Ideally there'd by support for using "n" and "p" to move to
next/previous error (with a little javascript), but for now I've been
using "back" in the browser to navigate through the tables.

An example of an error shown inline:
http://fedorapeople.org/~dmalcolm/static-analysis/2013-02-01/policycoreutils-2.1.13-27.2.fc17.src.rpm-001.html#file-868b5c03918269eaabebfedc41eaf32e390357be-line-791
shows a true error in seunshare.c found by cppcheck ("foo =
realloc(foo, , )"  is always a mistake, since if realloc fails you get
NULL back, but still have responsibility for freeing the old foo).


Comparison report
=================
The second experimental report can be seen here:
http://fedorapeople.org/~dmalcolm/static-analysis/2013-02-04/comparison-of-python-ethtool-builds.html

It shows a comparison of the results of two different builds of a
package (python-ethtool), again running multiple analyzers.
(specifically, a comparison between 0.7 and an snapshot of upstream
git).

It's similar to the first report, but instead of showing one file at a
time, it shows a side-by-side diff of old vs new file.

Any issues found in old or new source code are shown inline, so you can
see issues that are fixed, issues that are newly introduced, and issues
that are present in both old and new code.

Both reports could use some javascript to let you use "n" and "p" to go
to next/previous errors.  Also my CSS is ugly.

Any javascript/css experts out there who can help with those areas?

(FWIW, the code that generates these are in:
https://github.com/fedora-static-analysis/mock-with-analysis/tree/master/reports
specifically make-simple-report.py and make-comparative-report.py;
they're reading the output from mock-with-analysis)

Dave