making custom kernels easier to build

[Chris Murphy]

Since it came up at Flock: if it's possible to incorporate this recent experiencing testing Btrfs patches it'd be nice.

Two patches listed here, one is based on Btrfs integration branch, the other based on 3.16.0. I didn't realize the original patch was based on integration branch, which is apparently why it kept failing to apply with rpmbuild (and hence patch) on kernel-3.16.0-1.fc21.src.rpm. I'm told in this email that git am would have sorted this out for me. (?)

http://www.mail-archive.com/[e-mail]/msg36299.html

Request 1: if there's a more tolerant patch applying method with git that could be incorporated, even if it's just a step that converts the patch so that rpmbuild/patch will eat it, that would be nice. But if it requires a local git clone of upstream's dev branch, then ignore this part because I'm not going to do that. I'd sooner ask for a patch that applies onto something I'm actually going to build.

Request 2:  patches to just get picked up by dropping them somewhere, like rpmbuild/SOURCES/*.patch always get picked up and applied, rather than requiring two manually entered entries in kernel.spec per patch.

Thanks,

Chris Murphy

[Chris Murphy]

On Aug 11, 2014, at 3:28 PM, Chris Murphy  wrote:
> Since it came up at Flock: if it's possible to incorporate this recent experiencing testing Btrfs patches it'd be nice.
>
> Two patches listed here, one is based on Btrfs integration branch, the other based on 3.16.0. I didn't realize the original patch was based on integration branch, which is apparently why it kept failing to apply with rpmbuild (and hence patch) on kernel-3.16.0-1.fc21.src.rpm. I'm told in this email that git am would have sorted this out for me. (?)
>
> http://www.mail-archive.com/[e-mail]/msg36299.html
>
> Request 1: if there's a more tolerant patch applying method with git that could be incorporated, even if it's just a step that converts the patch so that rpmbuild/patch will eat it, that would be nice. But if it requires a local git clone of upstream's dev branch, then ignore this part because I'm not going to do that. I'd sooner ask for a patch that applies onto something I'm actually going to build.
>
> Request 2:  patches to just get picked up by dropping them somewhere, like rpmbuild/SOURCES/*.patch always get picked up and applied, rather than requiring two manually entered entries in kernel.spec per patch.

Request 3: Current instructions call for use of yum, yumdownloader, and yum-builddep. Can it be done entirely with dnf? I'm not sure what the dnf equivalent is for yumdownloader or yum-builddep.

Chris Murphy

[Josh Boyer]

On Mon, Aug 11, 2014 at 7:15 PM, Chris Murphy  wrote:
> On Aug 11, 2014, at 3:28 PM, Chris Murphy  wrote:
> > Since it came up at Flock: if it's possible to incorporate this recent experiencing testing Btrfs patches it'd be nice.
> >
> > Two patches listed here, one is based on Btrfs integration branch, the other based on 3.16.0. I didn't realize the original patch was based on integration branch, which is apparently why it kept failing to apply with rpmbuild (and hence patch) on kernel-3.16.0-1.fc21.src.rpm. I'm told in this email that git am would have sorted this out for me. (?)
> >
> > http://www.mail-archive.com/[e-mail]/msg36299.html
> >
> > Request 1: if there's a more tolerant patch applying method with git that could be incorporated, even if it's just a step that converts the patch so that rpmbuild/patch will eat it, that would be nice. But if it requires a local git clone of upstream's dev branch, then ignore this part because I'm not going to do that. I'd sooner ask for a patch that applies onto something I'm actually going to build.
> >
> > Request 2:  patches to just get picked up by dropping them somewhere, like rpmbuild/SOURCES/*.patch always get picked up and applied, rather than requiring two manually entered entries in kernel.spec per patch.
>
> Request 3: Current instructions call for use of yum, yumdownloader, and yum-builddep. Can it be done entirely with dnf? I'm not sure what the dnf equivalent is for yumdownloader or yum-builddep.

I have no idea.  None of us use dnf.  If someone would like to
experiment with what it would take to do this with dnf, that would be
a great way for the community to help out.  Post the directions to the
kernel list and we'll see how they look.

josh

[Josh Boyer]

On Mon, Aug 11, 2014 at 5:28 PM, Chris Murphy  wrote:
> Since it came up at Flock: if it's possible to incorporate this recent experiencing testing Btrfs patches it'd be nice.

Please send requests for the kernel to the kernel list in the future.
We don't read devel as frequently.
> Two patches listed here, one is based on Btrfs integration branch, the other based on 3.16.0. I didn't realize the original patch was based on integration branch, which is apparently why it kept failing to apply with rpmbuild (and hence patch) on kernel-3.16.0-1.fc21.src.rpm. I'm told in this email that git am would have sorted this out for me. (?)
>
> http://www.mail-archive.com/[e-mail]/msg36299.html
>
> Request 1: if there's a more tolerant patch applying method with git that could be incorporated, even if it's just a step that converts the patch so that rpmbuild/patch will eat it, that would be nice. But if it requires a local git clone of upstream's dev branch, then ignore this part because I'm not going to do that. I'd sooner ask for a patch that applies onto something I'm actually going to build.

I'll look this over later.  I'm far too jet lagged to do it now.
> Request 2:  patches to just get picked up by dropping them somewhere, like rpmbuild/SOURCES/*.patch always get picked up and applied, rather than requiring two manually entered entries in kernel.spec per patch.

Patches require ordering.  At a minimum you're going to have to list
them in the order they should apply.  The first line in the spec is to
enumerate them (e.g. Patch25100).  We might be able to do something
about the application, but if we do it's going to make things more
obfuscated, either by keeping the patches in a series file elsewhere
or using something else.  We default to explicit and dumb so you can
see exactly what is applied where.

josh

[ottos]

Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Josh Boyer
Sent: Monday, August 11, 2014 8:19 PM
To: Development discussions related to Fedora
Subject: Re: making custom kernels easier to build

On Mon, Aug 11, 2014 at 5:28 PM, Chris Murphy  wrote:
> Since it came up at Flock: if it's possible to incorporate this recent experiencing testing Btrfs patches it'd be nice.

Please send requests for the kernel to the kernel list in the future.
We don't read devel as frequently.
> Two patches listed here, one is based on Btrfs integration branch, the 
> other based on 3.16.0. I didn't realize the original patch was based 
> on integration branch, which is apparently why it kept failing to 
> apply with rpmbuild (and hence patch) on kernel-3.16.0-1.fc21.src.rpm. 
> I'm told in this email that git am would have sorted this out for me. 
> (?)
>
> http://www.mail-archive.com/[e-mail]/msg36299.html
>
> Request 1: if there's a more tolerant patch applying method with git that could be incorporated, even if it's just a step that converts the patch so that rpmbuild/patch will eat it, that would be nice. But if it requires a local git clone of upstream's dev branch, then ignore this part because I'm not going to do that. I'd sooner ask for a patch that applies onto something I'm actually going to build.

I'll look this over later.  I'm far too jet lagged to do it now.
> Request 2:  patches to just get picked up by dropping them somewhere, like rpmbuild/SOURCES/*.patch always get picked up and applied, rather than requiring two manually entered entries in kernel.spec per patch.

Patches require ordering.  At a minimum you're going to have to list them in the order they should apply.  The first line in the spec is to enumerate them (e.g. Patch25100).  We might be able to do something about the application, but if we do it's going to make things more obfuscated, either by keeping the patches in a series file elsewhere or using something else.  We default to explicit and dumb so you can see exactly what is applied where.

josh
--
devel mailing list
[e-mail]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

[Josh Boyer]

On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
> Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.

That doesn't explain much.  Why is that a problem?  Which two lines?
It's clearly not erroring in when it is called or the build would
fail.

Also, please don't top post.

josh

[ottos]

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Josh Boyer
Sent: Tuesday, August 12, 2014 7:02 AM
To: Development discussions related to Fedora
Subject: Re: making custom kernels easier to build

On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
> Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.

That doesn't explain much.  Why is that a problem?  Which two lines?
It's clearly not erroring in when it is called or the build would fail.

Also, please don't top post.

Josh
It does fail!!!!   Try it
--
devel mailing list
[e-mail]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

[Josh Boyer]

On Tue, Aug 12, 2014 at 8:04 AM, otto  wrote:
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Josh Boyer
> Sent: Tuesday, August 12, 2014 7:02 AM
> To: Development discussions related to Fedora
> Subject: Re: making custom kernels easier to build
>
> On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
> > Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.
>
> That doesn't explain much.  Why is that a problem?  Which two lines?
> It's clearly not erroring in when it is called or the build would fail.
>
> Also, please don't top post.
>
> Josh
> It does fail!!!!   Try it

We build kernels daily.  They call make modules_install.  It doesn't
fail.  You haven't given enough clear context on what you're talking
about.  It would be better for you to just post the error message
you're seeing.

josh

[Przemek Klosowski]

On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
> Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.

Typically, something like that might happen if you are compiling in a 
pre-existing environment, where either an old compilation or someone 
manually created a directory with the same name as a file from the 
current compilation. Such error tends not to self-correct. The kernel 
build system builds from scratch so it doesn't see the problem---I bet 
if you rm -rf the entire build tree before rebuilding it'll work for you 
as well.

NB, when you report errors, please provide more context: what exactly 
are you comparing (version/release numbers, where you got them from) and 
what errors are you seeing (the exact copy of the error message); 
otherwise all others can offer is guesses.

[ottos]

From: [e-mail] [mailto:[e-mail]] On Behalf Of Przemek Klosowski
Sent: Tuesday, August 12, 2014 12:02 PM
To: [e-mail]
Subject: Re: making custom kernels easier to build

On Tue, Aug 12, 2014 at 4:29 AM, ottos    wrote:

Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.

Typically, something like that might happen if you are compiling in a pre-existing environment, where either an old compilation or someone manually created a directory with the same name as a file from the current compilation. Such error tends not to self-correct.  The kernel build system builds from scratch so it doesn't see the problem---I bet if you rm -rf the entire build tree before rebuilding it'll work for you as well.

NB, when you report errors, please provide more context: what exactly are you comparing (version/release numbers, where you got them from) and what errors are you seeing (the exact copy of the error message); otherwise all others can offer is guesses.

The following code is the defective code from Makefile.  The lines with the ** besides them is the lines in particular that are defective.

modules_install: _modinst_ _modinst_post

PHONY += _modinst_

_modinst_:

@rm -rf $(MODLIB)/kernel

**      @rm -f $(MODLIB)/source

@mkdir -p $(MODLIB)/kernel

@ln -s $(srctree) $(MODLIB)/source

@if [ ! $(objtree) -ef  $(MODLIB)/build ]; then \

**               rm -f $(MODLIB)/build ; \

ln -s $(objtree) $(MODLIB)/build ; \

fi

if you execute a “make modules_install”   notice that you are trying delete a directive with a file delete.  I edit this file each time I build a custom kernel and put a “r” in front of the “f” making it –rf.  Is this enough.

The commands I use are make bzImage;make modules;make modules_install;make install

[Josh Boyer]

On Tue, Aug 12, 2014 at 1:34 PM, ottos  wrote:
> From: [e-mail]
> [mailto:[e-mail]] On Behalf Of Przemek
> Klosowski
> Sent: Tuesday, August 12, 2014 12:02 PM
> To: [e-mail]
>
> Subject: Re: making custom kernels easier to build
>
> On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
>
> Since this topic is here.  There is an error in Makefile when you do a make
> modules_install.  It attempts to delete a directory with a delete file
> command.  This occurs in two places.  If you are fixing fix this problem.
>
> Typically, something like that might happen if you are compiling in a
> pre-existing environment, where either an old compilation or someone
> manually created a directory with the same name as a file from the current
> compilation. Such error tends not to self-correct.  The kernel build system
> builds from scratch so it doesn't see the problem---I bet if you rm -rf the
> entire build tree before rebuilding it'll work for you as well.
>
> NB, when you report errors, please provide more context: what exactly are
> you comparing (version/release numbers, where you got them from) and what
> errors are you seeing (the exact copy of the error message); otherwise all
> others can offer is guesses.
>
> The following code is the defective code from Makefile.  The lines with the
> ** besides them is the lines in particular that are defective.
>
> modules_install: _modinst_ _modinst_post
>
> PHONY += _modinst_
>
> _modinst_:
>     @rm -rf $(MODLIB)/kernel
>
> **      @rm -f $(MODLIB)/source
>     @mkdir -p $(MODLIB)/kernel
>
>     @ln -s $(srctree) $(MODLIB)/source
>
>     @if [ ! $(objtree) -ef  $(MODLIB)/build ]; then \
>
> **               rm -f $(MODLIB)/build ; \
>             ln -s $(objtree) $(MODLIB)/build ; \
>
>     fi
>
>
>
> if you execute a "make modules_install"   notice that you are trying delete
> a directive with a file delete.  I edit this file each time I build a custom
> kernel and put a "r" in front of the "f" making it -rf.  Is this enough.

$(MODLIB)/source and $(MODLIB)/build are symlinks.  It's removing the
link, not the directory.  If they aren't links on your machine then
something went wrong somewhere for you, but this is not a problem
Fedora or upstream has.

josh

[ottos]

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Josh Boyer
Sent: Tuesday, August 12, 2014 12:43 PM
To: Development discussions related to Fedora
Subject: Re: making custom kernels easier to build

On Tue, Aug 12, 2014 at 1:34 PM, ottos  wrote:
> From: [e-mail]
> [mailto:[e-mail]] On Behalf Of Przemek 
> Klosowski
> Sent: Tuesday, August 12, 2014 12:02 PM
> To: [e-mail]
>
> Subject: Re: making custom kernels easier to build
>
> On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
>
> Since this topic is here.  There is an error in Makefile when you do a 
> make modules_install.  It attempts to delete a directory with a delete 
> file command.  This occurs in two places.  If you are fixing fix this problem.
>
> Typically, something like that might happen if you are compiling in a 
> pre-existing environment, where either an old compilation or someone 
> manually created a directory with the same name as a file from the 
> current compilation. Such error tends not to self-correct.  The kernel 
> build system builds from scratch so it doesn't see the problem---I bet 
> if you rm -rf the entire build tree before rebuilding it'll work for you as well.
>
> NB, when you report errors, please provide more context: what exactly 
> are you comparing (version/release numbers, where you got them from) 
> and what errors are you seeing (the exact copy of the error message); 
> otherwise all others can offer is guesses.
>
> The following code is the defective code from Makefile.  The lines 
> with the
> ** besides them is the lines in particular that are defective.
>
> modules_install: _modinst_ _modinst_post
>
> PHONY += _modinst_
>
> _modinst_:
>     @rm -rf $(MODLIB)/kernel
>
> **      @rm -f $(MODLIB)/source
>     @mkdir -p $(MODLIB)/kernel
>
>     @ln -s $(srctree) $(MODLIB)/source
>
>     @if [ ! $(objtree) -ef  $(MODLIB)/build ]; then \
>
> **               rm -f $(MODLIB)/build ; \
>             ln -s $(objtree) $(MODLIB)/build ; \
>
>     fi
>
>
>
> if you execute a "make modules_install"   notice that you are trying delete
> a directive with a file delete.  I edit this file each time I build a 
> custom kernel and put a "r" in front of the "f" making it -rf.  Is this enough.

$(MODLIB)/source and $(MODLIB)/build are symlinks.  It's removing the link, not the directory.  If they aren't links on your machine then something went wrong somewhere for you, but this is not a problem Fedora or upstream has.

Josh

I would accept this explanation except notice the line  @rm -rf $(MODLIB)/kernel which by your def is a symlink but has the -rf for removing a directory and on my machine all of the above are directories so it is intending to delete files in those directories and it works when I correct it.
--
devel mailing list
[e-mail]
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

[Nico Kadel-Garcia]

> On Aug 12, 2014, at 8:04, "otto"  wrote:
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Josh Boyer
> Sent: Tuesday, August 12, 2014 7:02 AM
> To: Development discussions related to Fedora
> Subject: Re: making custom kernels easier to build
> > On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
> > Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.
>
> That doesn't explain much.  Why is that a problem?  Which two lines?
> It's clearly not erroring in when it is called or the build would fail.
>
> Also, please don't top post.
>
> Josh
> It does fail!!!!   Try it

This sort if failure occurs when someone does a "cp -r" or better yet, a "scp -r" to copy the build tree elsewhere and the directory symlink gets replicated as a directory, instead.

From long, harsh experience: do not try to decide, in a Makefile, whether a directory is a symlink and get cutesy about it. Use the "rm -r" hammer to make sure.

[ottos]

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Nico Kadel-Garcia
Sent: Tuesday, August 12, 2014 7:28 PM
To: Development discussions related to Fedora
Subject: Re: making custom kernels easier to build
> On Aug 12, 2014, at 8:04, "otto"  wrote:
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Josh Boyer
> Sent: Tuesday, August 12, 2014 7:02 AM
> To: Development discussions related to Fedora
> Subject: Re: making custom kernels easier to build
> > On Tue, Aug 12, 2014 at 4:29 AM, ottos  wrote:
> > Since this topic is here.  There is an error in Makefile when you do a make modules_install.  It attempts to delete a directory with a delete file command.  This occurs in two places.  If you are fixing fix this problem.
>
> That doesn't explain much.  Why is that a problem?  Which two lines?
> It's clearly not erroring in when it is called or the build would fail.
>
> Also, please don't top post.
>
> Josh
> It does fail!!!!   Try it

This sort if failure occurs when someone does a "cp -r" or better yet, a "scp -r" to copy the build tree elsewhere and the directory symlink gets replicated as a directory, instead.
> From long, harsh experience: do not try to decide, in a Makefile, whether a directory is a symlink and get cutesy about it. Use the "rm -r" hammer to make sure.