Product lifecycles and kernel impacts

[Josh Boyer]

Hi All,

I've seen a lot of discussion around release cycles and lifetimes.
I've not heard any specific requests for what impacts any of the
current ideas would have to the kernel, but that doesn't mean they
won't be coming.  We should probably start thinking about various
things we can support today, what we could possibly support in the
future, and what we'd need to do it.  I've CC'd the Product liaisons
so they can see this.  Please keep them on CC.

The first scenario I can think of is probably some kind of "long-term"
release.  Exactly what long-term means here is undefined, but we
already support a Fedora release for 13 months or so.  We do that
today with rolling kernel releases.  We could possibly continue, but I
expect the Products to desire something more "stable".

What that implies is that we'd likely need to base on an upstream
longterm kernel.  That means:

1) No new kernel features.
2) No new hardware enablement outside of things acceptable for an
upstream longterm commit (basically just adding a new USB/PCI id to an
existing driver).
3) Bugfixes will primarily come from the upstream longterm tree
4) There will be no alternative kernels supported on the Fedora
longterm release.
5) There are no guarantees on bugfixes, response time, etc.

If any of the above are desired, people are better off picking an
Enterprise distro where you have contracted support.

So if that's the case, the feasibility of this is going to hinge a lot
on timing.  I would think from a kernel perspective we'd know up-front
when a Fedora longterm was going to happen, and we'd try and align
that release with the newest official longterm stable kernel.  Those
are maintained for 2 years, which seems to be a good fit for a Fedora
longterm release.  Any longer than that and you're back in the
Enterprise space again.

The second scenario I can think of is products with mismatched release
cycles.  E.g. Server doing a 2 year longterm and developing the
Server.next release on top of Base, which releases every 6 months.  Or
Workstation doing a release once a year while Base moves every 6
months.  Etc.  It's hard to come up with a concrete scenario since
none of the products have gotten this far.  It's worth noting that
FESCo is basically disallowing this for the first releases, so this is
a secondary concern.

I would propose that for Base, we continue our current method of
rebasing with each kernel release.  That has been working very well
for the past few Fedora releases, and gets us the most "bang for the
buck" in terms of features, hardware support, and fixes. A year long
release for e.g. Workstation would likely also desire the newer
support and features, so they could probably just update their kernel
in the same way.  This keeps our maintenance costs down to today's and
gives products flexibility.

In the Server LTS + Server.next development scenario, I would expect
the LTS to use the longterm kernel as suggested above which means
whenever they cut over to LTS mode it would need to coincide with an
upstream longterm kernel.  That would be something we have to watch
and plan for.  Server.next would continue to use Base until it was
ready to cut over and repeat.  I hope that makes sense.

In terms of people, the only major change I see would be the longterm
addition.  That might be something we can tuck, but having additional
QA and maintainer resources would allow us to cover development and
support better.  If there was any major deviation in which kernel was
used in the various products, we'd likely need either the Product
teams themselves to pick up maintenance or additional people on the
kernel team to handle that.  The desire here is to keep the kernel
common among as many Products and releases as we can.

This is mostly just some thoughts I've had on the topics so far.  Let
me know what you all are thinking and please ask questions or propose
alternatives as you see fit.

josh

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/21/2013 09:18 AM, Josh Boyer wrote:
>  Hi All,
>  
>  I've seen a lot of discussion around release cycles and lifetimes. 
>  I've not heard any specific requests for what impacts any of the 
>  current ideas would have to the kernel, but that doesn't mean they 
>  won't be coming.  We should probably start thinking about various 
>  things we can support today, what we could possibly support in the 
>  future, and what we'd need to do it.  I've CC'd the Product
>  liaisons so they can see this.  Please keep them on CC.
>  
>  The first scenario I can think of is probably some kind of
>  "long-term" release.  Exactly what long-term means here is
>  undefined, but we already support a Fedora release for 13 months or
>  so.  We do that today with rolling kernel releases.  We could
>  possibly continue, but I expect the Products to desire something
>  more "stable".
>  
>  What that implies is that we'd likely need to base on an upstream 
>  longterm kernel.  That means:
>  
>  1) No new kernel features. 2) No new hardware enablement outside of
>  things acceptable for an upstream longterm commit (basically just
>  adding a new USB/PCI id to an existing driver). 3) Bugfixes will
>  primarily come from the upstream longterm tree 4) There will be no
>  alternative kernels supported on the Fedora longterm release. 5)
>  There are no guarantees on bugfixes, response time, etc.
>  
>  If any of the above are desired, people are better off picking an 
>  Enterprise distro where you have contracted support.
>  
>  So if that's the case, the feasibility of this is going to hinge a
>  lot on timing.  I would think from a kernel perspective we'd know
>  up-front when a Fedora longterm was going to happen, and we'd try
>  and align that release with the newest official longterm stable
>  kernel.  Those are maintained for 2 years, which seems to be a good
>  fit for a Fedora longterm release.  Any longer than that and you're
>  back in the Enterprise space again.
>  
>  The second scenario I can think of is products with mismatched
>  release cycles.  E.g. Server doing a 2 year longterm and developing
>  the Server.next release on top of Base, which releases every 6
>  months.  Or Workstation doing a release once a year while Base
>  moves every 6 months.  Etc.  It's hard to come up with a concrete
>  scenario since none of the products have gotten this far.  It's
>  worth noting that FESCo is basically disallowing this for the first
>  releases, so this is a secondary concern.
>  
>  I would propose that for Base, we continue our current method of 
>  rebasing with each kernel release.  That has been working very
>  well for the past few Fedora releases, and gets us the most "bang
>  for the buck" in terms of features, hardware support, and fixes. A
>  year long release for e.g. Workstation would likely also desire the
>  newer support and features, so they could probably just update
>  their kernel in the same way.  This keeps our maintenance costs
>  down to today's and gives products flexibility.
>  
>  In the Server LTS + Server.next development scenario, I would
>  expect the LTS to use the longterm kernel as suggested above which
>  means whenever they cut over to LTS mode it would need to coincide
>  with an upstream longterm kernel.  That would be something we have
>  to watch and plan for.  Server.next would continue to use Base
>  until it was ready to cut over and repeat.  I hope that makes
>  sense.
>   
Would you believe I was typing up an email to you earlier today (that
I didn't finish) to suggest this exact thing?

My thought was that we would naturally tie the Fedora Server release
cycle to the Kernel LTS cycle, as you suggest. This should keep the
maintenance costs on the kernel down.

I suspect that Server is the only one of the three products for which
the LTS kernel makes sense, though. Both the Workstation and Cloud
projects are likely going to want to take advantage of newer
capabilities far more often.



>  In terms of people, the only major change I see would be the
>  longterm addition.  That might be something we can tuck, but having
>  additional QA and maintainer resources would allow us to cover
>  development and support better.  If there was any major deviation
>  in which kernel was used in the various products, we'd likely need
>  either the Product teams themselves to pick up maintenance or
>  additional people on the kernel team to handle that.  The desire
>  here is to keep the kernel common among as many Products and
>  releases as we can.
>   
What do you think you would need for a resource here, if we make the
following assumptions (not final, just ideas):

1) The Fedora Server stable release is the only Product running the
LTS kernel

2) We limit updates to the kernel package in the stable release to a
regular schedule (excepting only security fixes). See my lifecycle
proposal plan for how I suggest we might want to do Fedora Server 1.1,
1.2, etc. releases. If we limited kernel patch roll-ups to a six-month
cycle, would that reduce your resource needs?



>  This is mostly just some thoughts I've had on the topics so far.
>  Let me know what you all are thinking and please ask questions or
>  propose alternatives as you see fit.
>  
>  josh
>   
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.15 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlKOPRQACgkQeiVVYja6o6NEYgCfReM2infj7YK30IA6BLx3G7OC
> lgcAniXhJs28TzXTZ+oHagDPbHMiMPo2
> =+EUR
> -----END PGP SIGNATURE-----

[Josh Boyer]

On Thu, Nov 21, 2013 at 12:04 PM, Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>
>  On 11/21/2013 09:18 AM, Josh Boyer wrote:
> > Hi All,
> >
> > I've seen a lot of discussion around release cycles and lifetimes.
> > I've not heard any specific requests for what impacts any of the
> > current ideas would have to the kernel, but that doesn't mean they
> > won't be coming.  We should probably start thinking about various
> > things we can support today, what we could possibly support in the
> > future, and what we'd need to do it.  I've CC'd the Product
> > liaisons so they can see this.  Please keep them on CC.
> >
> > The first scenario I can think of is probably some kind of
> > "long-term" release.  Exactly what long-term means here is
> > undefined, but we already support a Fedora release for 13 months or
> > so.  We do that today with rolling kernel releases.  We could
> > possibly continue, but I expect the Products to desire something
> > more "stable".
> >
> > What that implies is that we'd likely need to base on an upstream
> > longterm kernel.  That means:
> >
> > 1) No new kernel features. 2) No new hardware enablement outside of
> > things acceptable for an upstream longterm commit (basically just
> > adding a new USB/PCI id to an existing driver). 3) Bugfixes will
> > primarily come from the upstream longterm tree 4) There will be no
> > alternative kernels supported on the Fedora longterm release. 5)
> > There are no guarantees on bugfixes, response time, etc.
> >
> > If any of the above are desired, people are better off picking an
> > Enterprise distro where you have contracted support.
> >
> > So if that's the case, the feasibility of this is going to hinge a
> > lot on timing.  I would think from a kernel perspective we'd know
> > up-front when a Fedora longterm was going to happen, and we'd try
> > and align that release with the newest official longterm stable
> > kernel.  Those are maintained for 2 years, which seems to be a good
> > fit for a Fedora longterm release.  Any longer than that and you're
> > back in the Enterprise space again.
> >
> > The second scenario I can think of is products with mismatched
> > release cycles.  E.g. Server doing a 2 year longterm and developing
> > the Server.next release on top of Base, which releases every 6
> > months.  Or Workstation doing a release once a year while Base
> > moves every 6 months.  Etc.  It's hard to come up with a concrete
> > scenario since none of the products have gotten this far.  It's
> > worth noting that FESCo is basically disallowing this for the first
> > releases, so this is a secondary concern.
> >
> > I would propose that for Base, we continue our current method of
> > rebasing with each kernel release.  That has been working very
> > well for the past few Fedora releases, and gets us the most "bang
> > for the buck" in terms of features, hardware support, and fixes. A
> > year long release for e.g. Workstation would likely also desire the
> > newer support and features, so they could probably just update
> > their kernel in the same way.  This keeps our maintenance costs
> > down to today's and gives products flexibility.
> >
> > In the Server LTS + Server.next development scenario, I would
> > expect the LTS to use the longterm kernel as suggested above which
> > means whenever they cut over to LTS mode it would need to coincide
> > with an upstream longterm kernel.  That would be something we have
> > to watch and plan for.  Server.next would continue to use Base
> > until it was ready to cut over and repeat.  I hope that makes
> > sense.
> >
>
>  Would you believe I was typing up an email to you earlier today (that
>  I didn't finish) to suggest this exact thing? 
I would believe it.  It's almost like I read the WG lists and pay
attention to stuff coming up there or something.

>  My thought was that we would naturally tie the Fedora Server release
>  cycle to the Kernel LTS cycle, as you suggest. This should keep the
>  maintenance costs on the kernel down. 
Down, yes.  Not non-existent but smaller.  And to be clear, I'm
referring to the official longterm kernels that GregKH supports and
uses for his LTSI stuff.  He commits to doing those for 2 years and so
far has done a good job with 3.0 and now 3.10.  In the past, others
have volunteered to do longterm kernels and they generally fail
because they lose interest.  We'll have to keep an eye on this because
if that becomes a problem, we aren't staffed to carry a longterm by
ourselves.

>  I suspect that Server is the only one of the three products for
> which
>  the LTS kernel makes sense, though. Both the Workstation and Cloud
>  projects are likely going to want to take advantage of newer
>  capabilities far more often. 
Probably.  In fact, anything more than one Product doing an LTS would
really make me nervous unless it was using the same cycle and kernel.
I don't think doing disjoint cycles with different kernels for LTS is
feasible.

> > In terms of people, the only major change I see would be the
> > longterm addition.  That might be something we can tuck, but having
> > additional QA and maintainer resources would allow us to cover
> > development and support better.  If there was any major deviation
> > in which kernel was used in the various products, we'd likely need
> > either the Product teams themselves to pick up maintenance or
> > additional people on the kernel team to handle that.  The desire
> > here is to keep the kernel common among as many Products and
> > releases as we can.
> >
>
>  What do you think you would need for a resource here, if we make the
>  following assumptions (not final, just ideas):
>
>  1) The Fedora Server stable release is the only Product running the
>  LTS kernel
>
>  2) We limit updates to the kernel package in the stable release to a
>  regular schedule (excepting only security fixes). See my lifecycle
>  proposal plan for how I suggest we might want to do Fedora Server 1.1,
>  1.2, etc. releases. If we limited kernel patch roll-ups to a six-month
>  cycle, would that reduce your resource needs? 
I'm not quite following you on this part.

If Server is doing an LTS, it follows the latest upstream longterm
kernel.  We agree there.  Which means that whenever upstream longterm
does a release (which is actually pretty regular), we do a kernel
update containing that.  E.g. 3.13.1, 3.13.2, 3.13.3, etc.  Those are
typically on the order of 1-3 weeks between releases.  Not months.
They also contain more than security fixes, but they are limited to
_fixes_ not features.  I would think that's acceptable.

So were you referring to Server.next (aka the development version)
with 6 month rollups?  I would think Server.next would just roll with
whatever is in Base until it cut off.  If you're talking about a
planned 6 month update bundle for the LTS of some kind, then I guess
you could do that.  I don't think it would change how often we update
the LTS kernel in koji and whatnot.  If you want bundled updates, we'd
probably just build the kernel and let the Server team pick it up in
whatever update form they want.

So from a maintainer resource standpoint, it would essentially be
another release to do with and a different kernel but mostly
hands-off.  If upstream lost interest in that particular longterm for
whatever reason, we'd likely have to pick up that support and we can't
currently do that.  Having another maintainer would help.  From a QA
standpoint, I'd probably push that back to the Product (Server) that
wanted to do the LTS, particularly if that Product wants to do some
kind of 6month roll-up release.  Make sense?

josh

[Josh Boyer]

On Thu, Nov 21, 2013 at 1:25 PM, Josh Boyer <jwboyer(a)fedoraproject.org&gt; wrote:
>  If Server is doing an LTS, it follows the latest upstream longterm
>  kernel.  We agree there.  Which means that whenever upstream longterm
>  does a release (which is actually pretty regular), we do a kernel
>  update containing that.  E.g. 3.13.1, 3.13.2, 3.13.3, etc.  Those are 
I should probably clarify that 3.13 was an example number there.  3.13
won't be a longterm upstream kernel.  Given the current longterm
schedules, I wouldn't expect another one to be picked until about a
year from now.  Something to keep in mind going forward.

josh

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/21/2013 01:25 PM, Josh Boyer wrote:
>  On Thu, Nov 21, 2013 at 12:04 PM, Stephen Gallagher
>  <sgallagh(a)redhat.com&gt; wrote:
> > -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
> > 
> > On 11/21/2013 09:18 AM, Josh Boyer wrote:
> >> Hi All,
> >> 
> >> I've seen a lot of discussion around release cycles and
> >> lifetimes. I've not heard any specific requests for what
> >> impacts any of the current ideas would have to the kernel, but
> >> that doesn't mean they won't be coming.  We should probably
> >> start thinking about various things we can support today, what
> >> we could possibly support in the future, and what we'd need to
> >> do it.  I've CC'd the Product liaisons so they can see this.
> >> Please keep them on CC.
> >> 
> >> The first scenario I can think of is probably some kind of 
> >> "long-term" release.  Exactly what long-term means here is 
> >> undefined, but we already support a Fedora release for 13
> >> months or so.  We do that today with rolling kernel releases.
> >> We could possibly continue, but I expect the Products to desire
> >> something more "stable".
> >> 
> >> What that implies is that we'd likely need to base on an
> >> upstream longterm kernel.  That means:
> >> 
> >> 1) No new kernel features. 2) No new hardware enablement
> >> outside of things acceptable for an upstream longterm commit
> >> (basically just adding a new USB/PCI id to an existing driver).
> >> 3) Bugfixes will primarily come from the upstream longterm tree
> >> 4) There will be no alternative kernels supported on the Fedora
> >> longterm release. 5) There are no guarantees on bugfixes,
> >> response time, etc.
> >> 
> >> If any of the above are desired, people are better off picking
> >> an Enterprise distro where you have contracted support.
> >> 
> >> So if that's the case, the feasibility of this is going to
> >> hinge a lot on timing.  I would think from a kernel perspective
> >> we'd know up-front when a Fedora longterm was going to happen,
> >> and we'd try and align that release with the newest official
> >> longterm stable kernel.  Those are maintained for 2 years,
> >> which seems to be a good fit for a Fedora longterm release.
> >> Any longer than that and you're back in the Enterprise space
> >> again.
> >> 
> >> The second scenario I can think of is products with mismatched 
> >> release cycles.  E.g. Server doing a 2 year longterm and
> >> developing the Server.next release on top of Base, which
> >> releases every 6 months.  Or Workstation doing a release once a
> >> year while Base moves every 6 months.  Etc.  It's hard to come
> >> up with a concrete scenario since none of the products have
> >> gotten this far.  It's worth noting that FESCo is basically
> >> disallowing this for the first releases, so this is a secondary
> >> concern.
> >> 
> >> I would propose that for Base, we continue our current method
> >> of rebasing with each kernel release.  That has been working
> >> very well for the past few Fedora releases, and gets us the
> >> most "bang for the buck" in terms of features, hardware
> >> support, and fixes. A year long release for e.g. Workstation
> >> would likely also desire the newer support and features, so
> >> they could probably just update their kernel in the same way.
> >> This keeps our maintenance costs down to today's and gives
> >> products flexibility.
> >> 
> >> In the Server LTS + Server.next development scenario, I would 
> >> expect the LTS to use the longterm kernel as suggested above
> >> which means whenever they cut over to LTS mode it would need to
> >> coincide with an upstream longterm kernel.  That would be
> >> something we have to watch and plan for.  Server.next would
> >> continue to use Base until it was ready to cut over and repeat.
> >> I hope that makes sense.
> >> 
> > 
> > Would you believe I was typing up an email to you earlier today
> > (that I didn't finish) to suggest this exact thing?
>  
>  I would believe it.  It's almost like I read the WG lists and pay 
>  attention to stuff coming up there or something.
>  
> > My thought was that we would naturally tie the Fedora Server
> > release cycle to the Kernel LTS cycle, as you suggest. This
> > should keep the maintenance costs on the kernel down.
>  
>  Down, yes.  Not non-existent but smaller.  And to be clear, I'm 
>  referring to the official longterm kernels that GregKH supports
>  and uses for his LTSI stuff.  He commits to doing those for 2 years
>  and so far has done a good job with 3.0 and now 3.10.  In the past,
>  others have volunteered to do longterm kernels and they generally
>  fail because they lose interest.  We'll have to keep an eye on this
>  because if that becomes a problem, we aren't staffed to carry a
>  longterm by ourselves.
>  
> > I suspect that Server is the only one of the three products for
> > which the LTS kernel makes sense, though. Both the Workstation
> > and Cloud projects are likely going to want to take advantage of
> > newer capabilities far more often.
>  
>  Probably.  In fact, anything more than one Product doing an LTS
>  would really make me nervous unless it was using the same cycle and
>  kernel. I don't think doing disjoint cycles with different kernels
>  for LTS is feasible.
>  
> >> In terms of people, the only major change I see would be the 
> >> longterm addition.  That might be something we can tuck, but
> >> having additional QA and maintainer resources would allow us to
> >> cover development and support better.  If there was any major
> >> deviation in which kernel was used in the various products,
> >> we'd likely need either the Product teams themselves to pick up
> >> maintenance or additional people on the kernel team to handle
> >> that.  The desire here is to keep the kernel common among as
> >> many Products and releases as we can.
> >> 
> > 
> > What do you think you would need for a resource here, if we make
> > the following assumptions (not final, just ideas):
> > 
> > 1) The Fedora Server stable release is the only Product running
> > the LTS kernel
> > 
> > 2) We limit updates to the kernel package in the stable release
> > to a regular schedule (excepting only security fixes). See my
> > lifecycle proposal plan for how I suggest we might want to do
> > Fedora Server 1.1, 1.2, etc. releases. If we limited kernel patch
> > roll-ups to a six-month cycle, would that reduce your resource
> > needs?
>  
>  I'm not quite following you on this part.
>  
>  If Server is doing an LTS, it follows the latest upstream longterm 
>  kernel.  We agree there.  Which means that whenever upstream
>  longterm does a release (which is actually pretty regular), we do a
>  kernel update containing that.  E.g. 3.13.1, 3.13.2, 3.13.3, etc.
>  Those are typically on the order of 1-3 weeks between releases.
>  Not months. They also contain more than security fixes, but they
>  are limited to _fixes_ not features.  I would think that's
>  acceptable.
>   
Ok, I was actually trying to make your life easier by reducing the
number of updates you had to package up, but if it's easier to follow
the 1-3 week schedule than a 3-6 month schedule, I'm not going to argue.


>  So were you referring to Server.next (aka the development version) 
>  with 6 month rollups?  I would think Server.next would just roll
>  with whatever is in Base until it cut off.  If you're talking about
>  a planned 6 month update bundle for the LTS of some kind, then I
>  guess you could do that.  I don't think it would change how often
>  we update the LTS kernel in koji and whatnot.  If you want bundled
>  updates, we'd probably just build the kernel and let the Server
>  team pick it up in whatever update form they want.
>   
Well, if you're going to do the work to release the updated kernels
more often, we'd gladly take them. I was just trying to reduce the
amount of work you would have to do for the LTS branch.


>  So from a maintainer resource standpoint, it would essentially be 
>  another release to do with and a different kernel but mostly 
>  hands-off.  If upstream lost interest in that particular longterm
>  for whatever reason, we'd likely have to pick up that support and
>  we can't currently do that.  Having another maintainer would help.
>  From a QA 
If upstream loses interest, we'll deal with it at that point. The
server cycle is hopefully going to be 18-24 months, so I'd like to
believe that our exposure on the back-end would be limited.


>  standpoint, I'd probably push that back to the Product (Server)
>  that wanted to do the LTS, particularly if that Product wants to do
>  some kind of 6month roll-up release.  Make sense? 
I agree.



I was kind of thinking that we actually might want to offer two
choices of kernels in general: the classic 'kernel', which is kept at
the latest upstream release and 'kernel-stable', which follows the
stable release. The Server would tie itself to 'kernel-stable', but
theoretically a user could opt to install the 'kernel' package
instead, if they had an urgent need for a new feature. (This operation
would not be recommended or supported in any way by the Fedora Server
product, just providing live rounds for shooting oneself in the foot).
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.15 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlKTfvcACgkQeiVVYja6o6PiawCcDoyqQ2bv+M5QByS5ryfV6kVO
> c1MAmwRqMfwm0ZL4ODRACQpfNbgoOcBZ
> =vcrX
> -----END PGP SIGNATURE-----

[Josh Boyer]

On Mon, Nov 25, 2013 at 11:46 AM, Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
> >> What do you think you would need for a resource here, if we
> make
> >> the following assumptions (not final, just ideas):
> >>
> >> 1) The Fedora Server stable release is the only Product running
> >> the LTS kernel
> >>
> >> 2) We limit updates to the kernel package in the stable release
> >> to a regular schedule (excepting only security fixes). See my
> >> lifecycle proposal plan for how I suggest we might want to do
> >> Fedora Server 1.1, 1.2, etc. releases. If we limited kernel patch
> >> roll-ups to a six-month cycle, would that reduce your resource
> >> needs?
> >
> > I'm not quite following you on this part.
> >
> > If Server is doing an LTS, it follows the latest upstream longterm
> > kernel.  We agree there.  Which means that whenever upstream
> > longterm does a release (which is actually pretty regular), we do a
> > kernel update containing that.  E.g. 3.13.1, 3.13.2, 3.13.3, etc.
> > Those are typically on the order of 1-3 weeks between releases.
> > Not months. They also contain more than security fixes, but they
> > are limited to _fixes_ not features.  I would think that's
> > acceptable.
> >
>
>  Ok, I was actually trying to make your life easier by reducing the
>  number of updates you had to package up, but if it's easier to follow
>  the 1-3 week schedule than a 3-6 month schedule, I'm not going to argue. 
My point was more that you might be underestimating the number of
fixes (some of which are CVE fixes) that flow into even the longterm
kernels.  Yes, most of them are fairly minor, but some of them aren't.
 So we'd bump and build, but not necessarily file updates unless
there's something important.

>  I was kind of thinking that we actually might want to offer two
>  choices of kernels in general: the classic 'kernel', which is kept at
>  the latest upstream release and 'kernel-stable', which follows the
>  stable release. The Server would tie itself to 'kernel-stable', but 
Nope.  That would require two kernel SRPMs or some horrible
abomination of packaging two different trees in the same SRPM.  That
leads to even more work for us, unless...

>  theoretically a user could opt to install the 'kernel'
> package
>  instead, if they had an urgent need for a new feature. (This operation
>  would not be recommended or supported in any way by the Fedora Server
>  product, just providing live rounds for shooting oneself in the foot). 
Then they can just install the kernel from the Base repo, which would
be this classic kernel you referred to above.  I'm not at all fond of
doing that same work twice in Server LTS just because people want
Server-LTS-except-for-the-kernel.  This is yet another use case that
is served by Enterprise distros already, so doing anything beyond "use
Base" is (imo) out-of-scope here.  If the Base kernel somehow becomes
incompatible with Server LTS userspace (which would be very rare),
then they get to keep all the broken pieces.

josh

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/25/2013 11:57 AM, Josh Boyer wrote:
>  On Mon, Nov 25, 2013 at 11:46 AM, Stephen Gallagher
>  <sgallagh(a)redhat.com&gt; wrote:
> >>> What do you think you would need for a resource here, if we
> >>> make the following assumptions (not final, just ideas):
> >>> 
> >>> 1) The Fedora Server stable release is the only Product
> >>> running the LTS kernel
> >>> 
> >>> 2) We limit updates to the kernel package in the stable
> >>> release to a regular schedule (excepting only security
> >>> fixes). See my lifecycle proposal plan for how I suggest we
> >>> might want to do Fedora Server 1.1, 1.2, etc. releases. If we
> >>> limited kernel patch roll-ups to a six-month cycle, would
> >>> that reduce your resource needs?
> >> 
> >> I'm not quite following you on this part.
> >> 
> >> If Server is doing an LTS, it follows the latest upstream
> >> longterm kernel.  We agree there.  Which means that whenever
> >> upstream longterm does a release (which is actually pretty
> >> regular), we do a kernel update containing that.  E.g. 3.13.1,
> >> 3.13.2, 3.13.3, etc. Those are typically on the order of 1-3
> >> weeks between releases. Not months. They also contain more than
> >> security fixes, but they are limited to _fixes_ not features.
> >> I would think that's acceptable.
> >> 
> > 
> > Ok, I was actually trying to make your life easier by reducing
> > the number of updates you had to package up, but if it's easier
> > to follow the 1-3 week schedule than a 3-6 month schedule, I'm
> > not going to argue.
>  
>  My point was more that you might be underestimating the number of 
>  fixes (some of which are CVE fixes) that flow into even the
>  longterm kernels.  Yes, most of them are fairly minor, but some of
>  them aren't. So we'd bump and build, but not necessarily file
>  updates unless there's something important.
>  
> > I was kind of thinking that we actually might want to offer two 
> > choices of kernels in general: the classic 'kernel', which is
> > kept at the latest upstream release and 'kernel-stable', which
> > follows the stable release. The Server would tie itself to
> > 'kernel-stable', but
>  
>  Nope.  That would require two kernel SRPMs or some horrible 
>  abomination of packaging two different trees in the same SRPM.
>  That leads to even more work for us, unless...
>  
> > theoretically a user could opt to install the 'kernel' package 
> > instead, if they had an urgent need for a new feature. (This
> > operation would not be recommended or supported in any way by the
> > Fedora Server product, just providing live rounds for shooting
> > oneself in the foot).
>  
>  Then they can just install the kernel from the Base repo, which
>  would be this classic kernel you referred to above.  I'm not at all
>  fond of doing that same work twice in Server LTS just because
>  people want Server-LTS-except-for-the-kernel.  This is yet another
>  use case that is served by Enterprise distros already, so doing
>  anything beyond "use Base" is (imo) out-of-scope here.  If the Base
>  kernel somehow becomes incompatible with Server LTS userspace
>  (which would be very rare), then they get to keep all the broken
>  pieces.
>   
Sorry, "use Base" was what I was essentially trying to say here. I
think we're pretty much in agreement. I did *not* mean we should build
an extra unstable kernel for the Server.

> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.15 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlKTgi0ACgkQeiVVYja6o6MQ7QCgqdp+eWsl1Lhwgfb6XNwHJHDD
> iHsAniuut4EcnBHN+0jSsAbcoToa+Syb
> =lXYY
> -----END PGP SIGNATURE-----

[Lance Lassetter]

On Thu, Nov 21, 2013 at 09:18:45AM -0500, Josh Boyer wrote:
>  Hi All,
>  
>  I've seen a lot of discussion around release cycles and lifetimes.
>  I've not heard any specific requests for what impacts any of the
>  current ideas would have to the kernel, but that doesn't mean they
>  won't be coming.  We should probably start thinking about various
>  things we can support today, what we could possibly support in the
>  future, and what we'd need to do it.  I've CC'd the Product liaisons
>  so they can see this.  Please keep them on CC.
>  
>  The first scenario I can think of is probably some kind of "long-term"
>  release.  Exactly what long-term means here is undefined, but we
>  already support a Fedora release for 13 months or so.  We do that
>  today with rolling kernel releases.  We could possibly continue, but I
>  expect the Products to desire something more "stable".
>  
>  What that implies is that we'd likely need to base on an upstream
>  longterm kernel.  That means:
>  
>  1) No new kernel features.
>  2) No new hardware enablement outside of things acceptable for an
>  upstream longterm commit (basically just adding a new USB/PCI id to an
>  existing driver).
>  3) Bugfixes will primarily come from the upstream longterm tree
>  4) There will be no alternative kernels supported on the Fedora
>  longterm release.
>  5) There are no guarantees on bugfixes, response time, etc.
>  
>  If any of the above are desired, people are better off picking an
>  Enterprise distro where you have contracted support.
>  
>  So if that's the case, the feasibility of this is going to hinge a lot
>  on timing.  I would think from a kernel perspective we'd know up-front
>  when a Fedora longterm was going to happen, and we'd try and align
>  that release with the newest official longterm stable kernel.  Those
>  are maintained for 2 years, which seems to be a good fit for a Fedora
>  longterm release.  Any longer than that and you're back in the
>  Enterprise space again.
>  
>  The second scenario I can think of is products with mismatched release
>  cycles.  E.g. Server doing a 2 year longterm and developing the
>  Server.next release on top of Base, which releases every 6 months.  Or
>  Workstation doing a release once a year while Base moves every 6
>  months.  Etc.  It's hard to come up with a concrete scenario since
>  none of the products have gotten this far.  It's worth noting that
>  FESCo is basically disallowing this for the first releases, so this is
>  a secondary concern.
>  
>  I would propose that for Base, we continue our current method of
>  rebasing with each kernel release.  That has been working very well
>  for the past few Fedora releases, and gets us the most "bang for the
>  buck" in terms of features, hardware support, and fixes. A year long
>  release for e.g. Workstation would likely also desire the newer
>  support and features, so they could probably just update their kernel
>  in the same way.  This keeps our maintenance costs down to today's and
>  gives products flexibility.
>  
>  In the Server LTS + Server.next development scenario, I would expect
>  the LTS to use the longterm kernel as suggested above which means
>  whenever they cut over to LTS mode it would need to coincide with an
>  upstream longterm kernel.  That would be something we have to watch
>  and plan for.  Server.next would continue to use Base until it was
>  ready to cut over and repeat.  I hope that makes sense.
>  
>  In terms of people, the only major change I see would be the longterm
>  addition.  That might be something we can tuck, but having additional
>  QA and maintainer resources would allow us to cover development and
>  support better.  If there was any major deviation in which kernel was
>  used in the various products, we'd likely need either the Product
>  teams themselves to pick up maintenance or additional people on the
>  kernel team to handle that.  The desire here is to keep the kernel
>  common among as many Products and releases as we can.
>  
>  This is mostly just some thoughts I've had on the topics so far.  Let
>  me know what you all are thinking and please ask questions or propose
>  alternatives as you see fit.
>  
>  josh
>  _______________________________________________
>  kernel mailing list
>  kernel(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/kernel 
I would agree with this. I really love Fedora and have been using it since the free Red
Hat days...probably since around 1998...

The rapid release cycle that Fedora adopted just doesn't seem for for servers at all.

If Fedora takes this route then why not cut out all server software altogether and just
make it a desktop OS.  

What about a Fedora-server version that may last 2 or 3 years?  Before an upgrade is
imminent.

In the long run this would seem a good selling point for Fedora in general and if there
was an optional donate page before downloading this longer version....

Lance Lassetter

[Reindl Harald]

Am 21.11.2013 18:23, schrieb Lance Lassetter:
>  If Fedora takes this route then why not cut out all server software
> altogether and just make it a desktop OS 
because people including me loving to have the same OS on their servers as on
their desktop machine to have F-1 in production and a few weeks after release
the recent version to find out rough edges, report bugs and find workarounds
as needed to get the next polished version  in production

this may not be the majority but i prove since 2009 and F9 on around 20 servers
that it is possible to do so without re-installing and with "cut out all server
software" you would rent Fedora unuseable at all because it would no longer
be a useful distribution for development