Static Analysis: results of FUDcon Lawrence hackfest

[Alec Leamas]

On 2013-01-24 22:03, David Malcolm wrote:
> On Thu, 2013-01-24 at 18:11 +0100, Alec Leamas wrote:
>> On 2013-01-24 17:44, David Malcolm wrote:
>>> Michael Hrivnak and I spent some time at FUDcon Lawrence looking at
>>> static code analysis.
>>>
>>> We hacked on the proposed common format for analysis tools (aka
>>> "firehose").
>>>
>> [cut]
>>
>>> The plan is that the interchange format can be uploaded into a web
>>> UI/database, so that we can:
>>> * scan the entire distro
>>> * compare warnings: e.g. what new warnings appear in a package rebuild?
>>> * have a consistent interface for marking warnings as false positives
>>> * come up with some subset of the warnings that we care about
>>> * etc
>>>
>> [cut]
>>
>> Probably off-topic, but just my 5c...  There are similar checks done by
>> fedora.-review, basically running spec conformance tests that doesn't
>> require a complete build  (performance reasons), boiling down to a list
>> of warnings. These are not directly tied to specific code, only the spec
>> file and never a specific line.  Still, the thought of of getting this
>> in the overall status for a package comes into my mind when I read this.
> That occurred to me as well.  It would probably be possible to run
> rpmlint during "mock-with-analysis" and capture the results.
>
> I've been trying to focus on *source code* warnings to avoid scope
> creep, but this does seem highly relevant to Fedora, so presumably the
> scope could be widened to include warnings about the specfile, and about
> artefacts that end in the package payload, and the package metadata.
>
>
>> To let fedora-review output some XML instead of current text-based
>> report. would be simple. But is there any value in it?  See the package
>> guideline violations that can be detected automatically in the same
>> database and web GUI?! Enclose not just source files but also overall
>> package analyze output (rpmlint comes to my mind)?
>>
>> Perhaps...
> Great idea.  Is it fully automated?  That's what we'd need in order to
> sanely run it within "mock-with-analysis"
>

Yes, it.s just a python script.  I see no major problem besides the 
output format, which certainly need to be something else than current 
text aimed to be pasted in bugzilla.

One thing is that f-r can make more conclusions if it has access to the 
build directory. In a normal invocation it does a complete rebuild of 
the package using mock  (and also installs it + runs rpmlint on 
installed pkg). In this scenario it works as best. Here, it would mean 
that the complete mock rebuild would be run by f-r, with proper options 
for other static analyze tools.

 From my perspective this is somewhat tempting. f-r has a plugin 
concept, so the static analyze steps could be defined as f-r plugins 
with dependencies and ordering. But this is certainly not obvious.

The other way to use f-r is to avoid the build step, running it after 
the static analyze mock rebuild. This somewhat criples f-r. We'll need 
some fixes e. g.,  to run rpmlint, nothing that tricky.

So, some discussions on the overall framework when combining the other 
tools with f-r is needed.  Nothing that complicated though, I guess.

--a