perl script to check for references to unknown Kconfig macros

[Paul Bolle]

Josh,

Here's a perl script I cobbled together that checks for references to
unknown Kconfig macros. Tested on top of current master. Invoke like
this:
    perl "$WHAT_SHALL_I_CALL_THIS.pl" $SOME_DIR/kernel-3.16.fc20/linux-3.17.0-0.rc7.git1.1.fc20.x86_64

(I stopped tracking rawhide long ago.)

I'll send a patch generated with the aid of this script shortly.

It seems overkill to invoke (something like) this every build. But
perhaps the package maintainers can include (something like) it before
they push changes into master.

The call of "glob("config-*")" is an accident waiting to happen, but I
couldn't be bothered to replace it with a call of "git ls-files
"config-*"".

Have fun!

Paul Bolle

# By Paul Bolle October 2014.
#
# Contributed to the public domain by its author.

use 5.016;
use warnings;
use autodie;

use File::Find;

my @Kconfigs;

my $Kconfigre = qr/Kconfig.*/;
my $configre = qr/^\s*(menu)?config\s+(?<config>(\w+))$/;
my $CONFIG_re = qr/\bCONFIG_(?<CONFIG_>(\w+))/;

sub match {
    push( @Kconfigs, $File::Find::name ) if ($_ =~ $Kconfigre);
}

sub parse_kconfig {
    my ($path) = @_;

my @ret;

open( my $kconfig, "<", $path );
    my $slurp = do { local $/ = undef; <$kconfig> };
    close( $kconfig );
    my @lines = split ( /\n/, $slurp );
    foreach my $line (@lines) {
    	if ($line =~ /$configre/) {
    		push( @ret, $+{config} );
    	}
    }

@ret;
}

sub parse_shipped {
    my ($path) = @_;

my @ret;

open( my $shipped, "<", $path );
    my $slurp = do { local $/ = undef; <$shipped> };
    close( $shipped );
    my @lines = split ( /\n/, $slurp );
    my $i = 1;
    foreach my $line (@lines) {
    	if ($line =~ /$CONFIG_re/) {
    		push( @ret, [$i, $+{CONFIG_}] );
    	}
    	$i++;
    }

@ret;
}

exit main ( @ARGV );

sub main {
    my %configs;

find( &match, @_ );

foreach my $Kconfig (@Kconfigs) {
    	my (@tmp) = parse_kconfig( $Kconfig );
    	foreach my $config ( @tmp ) {
    		$configs{ $config }++;
    	}
    }

foreach my $shipped (glob("config-*")) {
    	my (@tmp) = parse_shipped( $shipped );
    	foreach my $ref ( @tmp ) {
    		say( STDERR "$shipped:$ref->[0]: No Kconfig symbol matches 'CONFIG_$ref->[1]'" )
    			unless (grep( /$ref->[1]/, keys( %configs )));
    	}
    }

0;
}

[Josh Boyer]

On Thu, Oct 2, 2014 at 4:49 AM, Paul Bolle  wrote:
> Josh,
>
> Here's a perl script I cobbled together that checks for references to
> unknown Kconfig macros. Tested on top of current master. Invoke like
> this:
>     perl "$WHAT_SHALL_I_CALL_THIS.pl" $SOME_DIR/kernel-3.16.fc20/linux-3.17.0-0.rc7.git1.1.fc20.x86_64
>
> (I stopped tracking rawhide long ago.)
>
> I'll send a patch generated with the aid of this script shortly.
>
> It seems overkill to invoke (something like) this every build. But
> perhaps the package maintainers can include (something like) it before
> they push changes into master.

I would imagine it being most useful during the merge window kernels.
Perhaps running it once per -rc build would be sufficient to keep
things clean.
> The call of "glob("config-*")" is an accident waiting to happen, but I
> couldn't be bothered to replace it with a call of "git ls-files
> "config-*"".
>
> Have fun!

If you have no objections, I'll likely add this to the scripts/
directory in the kernel pkg repo.  That way it's there and people that
can actually read perl can modify it, etc.  Have a preference on a
name?  I was thinking "zombie_kconfig.pl".

josh
> Paul Bolle
>
> # By Paul Bolle October 2014.
> #
> # Contributed to the public domain by its author.
>
> use 5.016;
> use warnings;
> use autodie;
>
> use File::Find;
>
> my @Kconfigs;
>
> my $Kconfigre = qr/Kconfig.*/;
> my $configre = qr/^\s*(menu)?config\s+(?<config>(\w+))$/;
> my $CONFIG_re = qr/\bCONFIG_(?<CONFIG_>(\w+))/;
>
> sub match {
>         push( @Kconfigs, $File::Find::name ) if ($_ =~ $Kconfigre);
> }
>
> sub parse_kconfig {
>         my ($path) = @_;
>     my @ret;
>
>     open( my $kconfig, "<", $path );
>     my $slurp = do { local $/ = undef; <$kconfig> };
>     close( $kconfig );
>     my @lines = split ( /\n/, $slurp );
>     foreach my $line (@lines) {
>             if ($line =~ /$configre/) {
>                     push( @ret, $+{config} );
>             }
>     }
>
>     @ret;
>
> }
>
> sub parse_shipped {
>         my ($path) = @_;
>     my @ret;
>
>     open( my $shipped, "<", $path );
>     my $slurp = do { local $/ = undef; <$shipped> };
>     close( $shipped );
>     my @lines = split ( /\n/, $slurp );
>     my $i = 1;
>     foreach my $line (@lines) {
>             if ($line =~ /$CONFIG_re/) {
>                     push( @ret, [$i, $+{CONFIG_}] );
>             }
>             $i++;
>     }
>
>     @ret;
>
> }
>
> exit main ( @ARGV );
>
> sub main {
>         my %configs;
>     find( \&match, @_ );
>
>     foreach my $Kconfig (@Kconfigs) {
>             my (@tmp) = parse_kconfig( $Kconfig );
>             foreach my $config ( @tmp ) {
>                     $configs{ $config }++;
>             }
>     }
>
>     foreach my $shipped (glob("config-*")) {
>             my (@tmp) = parse_shipped( $shipped );
>             foreach my $ref ( @tmp ) {
>                     say( STDERR "$shipped:$ref->[0]: No Kconfig symbol matches 'CONFIG_$ref->[1]'" )
>                             unless (grep( /$ref->[1]/, keys( %configs )));
>             }
>     }
>
>     0;
>
> }

[Paul Bolle]

On Thu, 2014-10-02 at 08:41 -0400, Josh Boyer wrote:
> On Thu, Oct 2, 2014 at 4:49 AM, Paul Bolle  wrote:
> > Here's a perl script I cobbled together that checks for references to
> > unknown Kconfig macros. Tested on top of current master. Invoke like
> > this:
> >     perl "$WHAT_SHALL_I_CALL_THIS.pl" $SOME_DIR/kernel-3.16.fc20/linux-3.17.0-0.rc7.git1.1.fc20.x86_64
> >
> > (I stopped tracking rawhide long ago.)
> >
> > I'll send a patch generated with the aid of this script shortly.
> >
> > It seems overkill to invoke (something like) this every build. But
> > perhaps the package maintainers can include (something like) it before
> > they push changes into master.
>
> I would imagine it being most useful during the merge window kernels.
> Perhaps running it once per -rc build would be sufficient to keep
> things clean.

Thinking some more about this, maybe just once per merge window might do
the trick. Say after -rc3 or -rc4, when all the big changes have landed.
> > The call of "glob("config-*")" is an accident waiting to happen, but I
> > couldn't be bothered to replace it with a call of "git ls-files
> > "config-*"".
> >
> > Have fun!
>
> If you have no objections, I'll likely add this to the scripts/
> directory in the kernel pkg repo.

No, see the note about "public domain".

Note that it should be run from the package root (otherwise
"glob( "config-*" )" will return nothing). Tt would also be nice if it
was automagically supplied with the right build directory as an
argument. That requires some hooks in kernel.spec, I guess. I don't know
how to do that. kernel.spec is a scary thing... But once per merge
window is five or six times a year. That might as well be done manually.
> That way it's there and people that
> can actually read perl can modify it, etc.

... and have laugh or two about the rather peculiar style I have grown
used to. (So peculiar I can't even manage to follow my own whitespace
rules.)
> Have a preference on a
> name?  I was thinking "zombie_kconfig.pl".

I used check-configs.pl locally, but your name is funnier.

Paul Bolle

[Paul Bolle]

On Thu, 2014-10-02 at 16:36 +0200, Paul Bolle wrote:
> On Thu, 2014-10-02 at 08:41 -0400, Josh Boyer wrote:
> > If you have no objections, I'll likely add this to the scripts/
> > directory in the kernel pkg repo.
>
> No, see the note about "public domain".

This was ambiguous. I meant to say that I have no objections, it's
public domain, so do as you please.

Paul Bolle