RFC: Repos with vanilla kernel packages
Thorsten Leemhuis
fedora at leemhuis.info
Sun Jan 22 17:43:44 UTC 2012
Hi!
I know a lot about packaging from my work on Fedora in its early years
and quite closely watch the kernel for my day job these days. So I many
moons ago thought "why not combine the two interests and maintain a
small special interest repository that provides vanilla kernel packages
for Fedora", but never realized that idea.
But I finally got things running now and set up such a repo with Linux
3.2.1. Find details on
https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories
Repo:
http://repos.fedorapeople.org/repos/thl/kernel-vanilla-stable/
Git:
http://fedorapeople.org/gitweb?p=thl/public_git/kernel.git;a=summary
Patch to kernel.spec from F16
http://fedorapeople.org/gitweb?p=thl/public_git/kernel.git;a=commitdiff;h=7df221fa0c8d7f11f0ef511a5a75e5d5bd062ce5
I included the source for above wiki page and the patch kernel.spec
below. I hope that makes yelling in my direction ^w^w^w^w commenting on
it easier, as I suppose some people will not like the idea as a whole
^w^w^w^w^w some of the decisions I made when adjusting the spec file for
my purposes. if that's the case please let me know, as I do not plan to
mention the effort anywhere else over the next few days; that way I
should be able to adjust details if there are good reasons to do so,
before to many people get aware of those repos.
Note, the next step I plan is to set up a second repo with packages that
have the latest mainline development kernels (3.3-rc). But I thought I
collect options on the current state of things first before moving on.
CU
knurd
Text from the wiki page:
> {{admon/warning|This is a draft document that describes a package
> repository that is neither announced not ready for public
> consumption, as some of the details might change during the
> boot-up-phase the repositories are in at the moment. And this page
> definitely needs someone that proof reads it…}}
>
> = kernel-vanilla packages for Fedora =
>
> == Overview ==
>
> This page contains information about a
> [http://repos.fedorapeople.org/repos/thl/ set of package
> repositories] on fedorapeople.org that contain RPMs of vanilla
> kernels built for Fedora. Vanilla in this scope means "unmodified",
> hence these packages do not contain any of those enhancements the
> Fedora developers integrate into the kernel packages that Fedora
> normally contains.
>
> Currently there is only one repository:
>
> *
> [http://repos.fedorapeople.org/repos/thl/kernel-vanilla-stable/fedora-16/
> kernel-vanilla-stable for F16]
> ([http://repos.fedorapeople.org/repos/thl/kernel-vanilla-stable/ repo
> file]) -- Latest kernel from the stable series (3.x.y)
>
> There will be one more soon:
>
> * kernel-vanilla-mainline for F16 -- latest mainline development
> kernel (aka 3.x-rc series)
>
> There is the rough idea for more:
>
> * kernel-vanilla-mainline for F17 -- latest mainline development
> kernel (aka 3.x-rc series) * kernel-vanilla-stable-testing for F16 --
> 3.x.y-rc kernels could go here; might also be a good place to put new
> releases from the mainline (3.x) or stable series (3.x.y) here for a
> short while before moving them to kernel-vanilla-stable
>
> To use those kernels download the repo file and put it in
> /etc/yum.repos.d/ and install the kernel you want with yum; the
> packages are signed with
> [http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD7927A2FCC9DBCAB
> this key].
>
> Please note a few important things:
>
> * none of the developers that maintain the Fedora kernel is involved
> in this effort * most systems work better and are run in a more
> secure manner with the official Fedora kernels * the usage
> instructions are brief on purpose; if you don't understand them, then
> you likely should not use these packages
>
> For more information see the FAQ.
>
> == FAQ ==
>
> === For users ===
>
> ==== Who is behind this effort? Can we trust the people behind it?
> ====
>
> Right now the kernel-vanilla repositories are maintained by
> [[user:thl|Thorsten Leemhuis (aka "knurd")]] only. Hopefully a few
> other people join to help, that's why part of this text is written as
> if a team is keeping care of the repositories.
>
> You have to decide yourself if you can trust the packages from these
> repositories. If it is any help: Some of those that use or contribute
> to Fedora since a while will know that Thorsten has quite a history
> of Fedora contributions, even if he was not that much active in the
> past two years. You can assume he has no interest in ruin his
> reputation quickly by providing crap in these repositories. On the
> other hand you should know that Thorsten started these repositories
> to learn more about the kernel and kernel development; so expect
> he'll make some mistakes along the way. And the vanilla kernels have
> some known downsides (see below)
>
> ==== What's the goal? ====
>
> The main ideas is to help upstream development, which in the end will
> be of benefit for Fedora as well. With the packages from the mainline
> repositories it for example is quite easy to test kernels that are
> still under development and report bugs early. The kernels from the
> stable repositories on the other hand make it easy to check if a bug
> that happens with a kernel from Fedora is specific to it or also
> present in the newest vanilla kernel; if that is the case users can
> directly work with upstream on solutions for the problem and don't
> have to go through the overworked and quite busy developers that
> maintain the kernel packages in Fedora.
>
> ==== Are the kernels from the kernel-vanilla repositories as good as
> those Fedora provides? ====
>
> No. There are several reasons for why not; the most important ones:
>
> * the developers that take care of the kernel package in Fedora are
> far more experienced in packaging kernels and kernel development than
> those that take care of the kernel-vanilla repositories * the kernels
> that get used in Fedora or released as proper update get a lot of
> testing; the kernels from the kernel-vanilla repositories get nearly
> no testing * some of the kernel-vanilla repositories contain kernels
> that are still under heavy development and sometimes are known to
> have serious bugs * the official Fedora kernels sometimes contain
> changes that fix security problems before they get upstream
>
> But the non-development kernels found in the kernel-vanilla
> repositories should work fine for a lot of situations and often be
> round about as good as a self-compiled kernel.
>
> ==== Are the kernels safe to use? ====
>
> Depends on your definition of "safe".
>
> The Linux kernel is a complex piece of software which contains bugs.
> Those can lead to data loss or in very rare situations even lead to
> hardware defects. Those bugs might only show up under specific
> circumstances -- for example when a specific mix of hardware is used
> with a specific kernel version that was built with a specific
> configuration. It might be unlikely that such a bug is triggered by
> one of the non-development kernels from the kernel-vanilla
> repositories, but it's definitely possible. Self compiled kernels
> bear the same risk; chances of hitting serious bugs are lower for
> kernels that have undergone widespread testing already.
>
> In other words: The kernels from the kernel-vanilla repositories will
> work just fine for most people. But use them on your own risk and
> have backups handy, as there is a small risk those kernels might
> damage your data or your system.
>
> ==== Should everything work with those kernels that works with the
> official Fedora kernels? ====
>
> No. Vanilla kernels are not that different from the kernels Fedora
> provides, but the latter include a few enhancements. Each of them is
> there for a good reason.
>
> When this text was written Fedora for example included utrace in
> their kernels to support userspace tracing with systemtap; that won't
> work with the kernels from the kernel-vanilla repositories. The
> kernels from Fedora sometimes include fresher drivers which some
> systems will require to work properly. And sometimes there are
> inter-dependencies between drivers in kernel and userland. The
> nouveau driver for graphics hardware from Nvidia is one such driver,
> as it has no stable API yet; that's why the DRM/KMS driver in the
> kernel is marked as "staging". The Mesa 3D or X.org drivers included
> in a particular Fedora release therefore might depend on the exact
> nouveau DRM/KMS driver which is part of the kernels Fedora ships for
> this release; thus the nouveau drivers for Mesa 3D and X.org that are
> part of a certain Fedora release might not work properly with kernels
> found in the kernel-vanilla repositories, as the latter might contain
> an older or newer nouveau DRM/KMS driver which are incompatible.
>
> The non-development kernels found in the kernel-vanilla repositories
> therefore should work on a lot of systems, but on some systems they
> will be worse than the kernels Fedora provides.
>
> ==== Will this repository also ship updates userland components like
> drivers or udev that match the kernels in the repositories? ====
>
> No, as there should be no need to, as the interfaces between the
> kernel and userland software should never change in an incompatible
> way; Linus Torvalds makes this pretty clear every now and then.
>
> That is the long story short. There are a lot of situations where the
> world is more complicated:
>
> * above mentioned rule does not apply to staging drivers, so
> situations might arise where the vanilla kernels are not usable for
> people that need staging drivers for their system. Apart from the
> nouveau drivers that shouldn't bother to many people; and time will
> tell how bad the situation is for nouveau. * Fedora sometimes might
> contains software that depends on bits that are not upstream
>
> And even with this rule sometimes a new mainline kernel versions
> brings changes that require updates userland software. Three
> examples:
>
> * the version number jump from 2.6.39 to 3.0 confused some software *
> in rare cases fixing security problems was only possible my changing
> the interfaces in an incompatible way * sometimes nobody notices
> early enough that interfaces have changed
>
> It remains to be seen how often we hit such issues and how bad they
> are; how we deal with them will need to get discussed on a case by
> case basis. In some cases we might have to other solution then to add
> new versions of other software to the repositories. But the plan is
> to avoid this.
>
> ==== Do you plan to provide packages for "linux-next" or "linux-rt"
> as well? ====
>
> For now: No. I know there is some interest in packages for them, but
> maintaining those will consume a lot of time regularly and we have
> not enough resources to do it properly right now. But if you want to
> step up and help, [[user:thl|get in contact]].
>
> ==== Do you plan to provide vanilla kernels for RHEL and derivatives
> like CentOS and SL? ====
>
> Sounds like a good addition. But it will result in additional work,
> too; and we currently have no one that would regularly run such
> kernels. So for now we won't get our feet wet in that area. But if
> you want to step up and help, [[User:thl|get in contact]].
>
> ==== Do you plan to provide packages for longterm kernels ====
>
> Unlikely. Mail goal of the kernel-vanilla repositories is to help
> upstream kernel development; but longterm kernels are a dead end and
> quite far away from mainline development, so they would not fit that
> well. But it might make sense for RHEL and derivatives, if those will
> ever be supported.
>
> ==== What configuration do those kernels use? ====
>
> Basically the same configuration the Fedora kernels use. Maybe a few
> staging drivers might get turned on to help their development, but
> apart from it the plan is to stick closely to what Fedora does.
>
> ==== Why don't you put these kernels in Fedoras main repositories
> ====
>
> I tend to think it's not a good idea, as that would make them more
> "official" and people might simply use those packages without knowing
> what their downsides are.
>
> That's the long story rough and short. And sure, there are reasons
> why having vanilla kernels in the main repositories make sense. Feel
> free to start a discussion on
> [https://admin.fedoraproject.org/mailman/listinfo/devel Fedoras devel
> mailing list], we'll watch and might jump in.
>
> Putting the kernels in a well know 3rd party add-on repository for
> Fedora might make sense, but some of the problems would be similar;
> and there are others problems,as then users might ask to build add-on
> modules for those kernels, too. In other words: Would need discussion
> and careful evaluation.
>
> ==== Are those kernels really unpatched? ====
>
> No, they contain a handful of very few changes that are needed for
> packaging.
>
> ==== How up2date will those repositories be? ====
>
> Time will tell, but we do the work in our spare time. Sometimes the
> day job and this strange thing called "real life" leave not much time
> to work on these kernels, so there will be a lag.
>
> === For contributors and developers ===
>
> ==== Can you please include this patch? ====
>
> No. Get your patch merged upstream, then the change you are
> interested in will automatically show up in these packages (and in
> Fedora, too)
>
> ==== Is there a Git tree somewhere? ====
>
> [http://fedorapeople.org/gitweb?p=thl/public_git/kernel.git;a=summary
> Sure].
>
> Let us know if we should do modifications to allow others to
> contribute to or benefit from this git tree better.
>
> ==== What Fedora versions will be supported ====
>
> The plan is to support the most recent Fedora version for the stable
> and mainline repositories. The latter will also be provides for the
> distribution that is currently under development (rawhide on the
> first half of a devel cycle iteration or the alpha/beta/rcs in the
> second half)
>
> ==== Why are there no debug kernels and not even debuginfo packages
> ====
>
> The space on repos.fedoraprople.org is limited, hence limiting the
> number of kernels we can provide. The debuginfo packages are also
> quite big, which makes them hard to handle. Hopefully in the short or
> medium timeframe solutions can be found to provide these packages.
>
> ==== Why don't you commit your changes to Fedora's kernel git repo on
> pkgs.fedoraproject.org? ====
>
> Maybe that would make sense. But it bears the risk that a commit is
> done to a wrong branch and disturbs the work of the Fedora kernels
> maintainer. Further: Not all of those that contribute to Fedora can
> commit there. That's similar with the fedorapeople git repository,
> but the docs indicate others can be given access with the help of
> ACLs.
>
> But whatever: Git is made for distributed development and it's likely
> not a big deal where the git repo lives.
>
> ==== Can I help ====
>
> Of course. [[user:thl|Talk to Thorsten]]; best if you come with some
> ideas what you can and want to do.
>
> ==== Do you plan to work together with those that take care of the
> kernel packages in Fedora? ====
>
> Definitely. But it remains to be seen how it looks like in practice.
>
>
> ==== Please stop providing alternative kernel packages, they take
> attention away from the kernels Fedora provide and thus harm Fedora!
> ====
>
> That's a valid concern, but I think the benefits outweigh the
> downsides.
>
> That again is the long story short. Just to get a little bit deeper
> into it: Similar arguments could be used to argue that Fedora should
> stop shipping patched kernels, as those take attention away from the
> upstream kernel. Up to a point such an argument is valid, too, but
> there are good reasons why Fedora patches its kernels.
>
> Maybe in the long term Fedora can ship vanilla kernels as regular
> kernel. That would be best for all, but a goal that is would take
> quite some effort to reach. Might be worth start a discussion on
> [https://admin.fedoraproject.org/mailman/listinfo/devel Fedoras devel
> mailing list] how to get closer to that goal.
>
> ==== Why did you drop the "-vanilla" postfix that normally gets added
> to the "name" macro when you build a vanilla kernel RPM locally?
> ====
>
> I've thought about dropping or leaving it for a while, as both
> schemes have various benefits and downsides. In the end I went for
> dropping it due to reasons like this:
>
> * nearly every other repository in Fedoraland that ships variants of
> a packages that is included in Fedora does not change the name * the
> postfix breaks things like "fedpkg srpm" on the git checkout *
> external solutions that heavily depend on the naming scheme fedora
> used (like the kmod stuff used in some external repositories) should
> just work * yum would not recognize kernel packages with a "-vanilla"
> postfix as "installonly" and thus would perform a regular update for
> vanilla packages instead of installing them parallel to the current
> one
Patch
> --- a/kernel.spec
> +++ b/kernel.spec
> @@ -23,7 +23,7 @@ Summary: The Linux kernel
> #
> # (Uncomment the '#' and both spaces below to set the buildid.)
> #
> -# % define buildid .local
> +%define buildid .vanilla.knurd
> ###################################################################
>
> # The buildid can also be specified on the rpmbuild command line
> @@ -109,7 +109,7 @@ Summary: The Linux kernel
> # kernel-PAE (only valid for i686)
> %define with_pae %{?_without_pae: 0} %{?!_without_pae: 1}
> # kernel-debug
> -%define with_debug %{?_without_debug: 0} %{?!_without_debug: 1}
> +%define with_debug %{?_with_debug: 1} %{?!_with_debug: 0}
> # kernel-doc
> %define with_doc %{?_without_doc: 0} %{?!_without_doc: 1}
> # kernel-headers
> @@ -119,7 +119,7 @@ Summary: The Linux kernel
> # tools
> %define with_tools %{?_without_tools: 0} %{?!_without_tools: 1}
> # kernel-debuginfo
> -%define with_debuginfo %{?_without_debuginfo: 0} %{?!_without_debuginfo: 1}
> +%define with_debuginfo %{?_with_debuginfo: 1} %{?!_with_debuginfo: 0}
> # kernel-bootwrapper (for creating zImages from kernel + initrd)
> %define with_bootwrapper %{?_without_bootwrapper: 0} %{?!_without_bootwrapper: 1}
> # Want to build a the vsdo directories installed
> @@ -164,7 +164,7 @@ Summary: The Linux kernel
> %define with_sparse %{?_with_sparse: 1} %{?!_with_sparse: 0}
>
> # Include driver backports (e.g. compat-wireless) in the kernel build.
> -%define with_backports %{?_without_backports: 0} %{?!_without_backports: 1}
> +%define with_backports %{?_with_backports: 1} %{?!_with_backports: 0}
>
> # Set debugbuildsenabled to 1 for production (build separate debug kernels)
> # and 0 for rawhide (all kernels are debug kernels).
> @@ -234,7 +234,7 @@ Summary: The Linux kernel
>
> %if %{nopatches}
> %define with_bootwrapper 0
> -%define variant -vanilla
> +#define variant -vanilla
> %else
> %define variant_fedora -fedora
> %endif
> @@ -2274,6 +2274,12 @@ fi
> # and build.
>
> %changelog
> +* Fri Jan 20 2012 Thorsten Leemhuis <fedora at leemhuis.info>
> +- build vanilla with "buildid .vanilla.knurd" and without debug
> +- disable compat-wireless
> +- disable debuginfo creation, to big to handle for me right now
> +- do not set the variant macro as that adds "-vanilla" to %%Name
> +
> * Fri Jan 20 2012 Dave Jones <davej at redhat.com>
> - net: reintroduce missing rcu_assign_pointer() calls
>
More information about the kernel
mailing list