2:05 p.m.
At FLOCK this year, I did a short workshop on what was labeled EPEL.Next.
At that we went over a bit of what EPEL has done in the past, what its
current challenges are, and what could be its future. Toshio Kuratomi was
great in capturing what was said at the meeting and
EPEL
* Extra packages for enterprise linux
* Lots of name recognition now
* Formed as rhel5 was being released
people needed more packages. at the time there were a ton of addon repos.
None of the m were compatible with each other necessarily.
All the other repos, dag, freshrpms, etc overrode the base os in some
places.
Decided for EPEL, overriding RHEL was something we wouldn't do.
Started with RHEL3,4 and soon after release we had 5.
RHEL was supported 5-7years and we figured we could support EPEL packages
for the same length of time.
Early controversy - repotags (shortened name that can be in the packages.
[graph of Fedora machines measured by unique ips that hit mirrormanager vs
EPEL machines]
[graph shows steady growth in EPEL. Larger initial Fedora machines that
rose and then plateaued]
[graph shows a noticable dip every weekend. Can also see the dip in Fedora
numbers for the Incident.]
[graph is probably conservative in numbers as many enterprises
Second graph:
[Graph shows combined growth same as last graph.]
[Graph shows EPEL5 that has risen but plateaued about the time EPEL6 came
out.]
[graph shows that EPEL6 has been growing and has surpassed the EPEL5
plateau.]
Where are we now:
We support 3 arches.
RHEL4: 1258 srpms. EOL
RHEL5: 3651 srpms
RHEL6: 5449 srpms
RHEL7: 3100 srpms and growing fastest
For CENTOS7: the epel repofile will be shipped directly with centos for the
first time.
Current Problems:
* 10+ year support for RHEL now. Hard for EPEL packagers to commit to
doing the same.
* We have requests for both newer and older conflicting packages.
- Currently policy is not to have unpleasant surprises regarding
compatibility.
- puppet2.x vs puppet3.x
* Some people need major changes to build new software
* Some people need no changes in order to keep existing software running
* Developer burnout: People come in and after a few years, they are burnt
out because they can't upgrade the things that they want due to the
compatibility policies.
Where are we going?
Pain Points:
* Inability to yum downgrade because only include one old package in the
updates tree
* Harder for EPEL than Fedora because EPEL users have more need to
rollback if an incompatibility creeps in and because EPEL users may
schedule their releases
* Not every part of package repo is suitable for inclusion in anaconda
because we may not have dep closure in the subset of the repo.
* Which RHEL point release can also make a difference because the base
RHEL sometimes upgrades incompatibly and we don't keep a separate build for
both point releases.
* Can take a long time to push packages through bodhi. Can take 7 days to
push a CVE through bodhi. (global EPEL and Fedora problem)
* Not all maintainers interpret "Don't break compatibility" in the same
way.
* Many guidelines are verbal, not formal.
* Some maintainers are mainly Fedora maintainers and don't know what people
are using it for in EPEL. They respond to requests to install or update a
package out of a willingness to help but don't know what users need.
* No guideline enforcement.
* Takes a while for new EPEL releases to be brought out once a new RHEL
release is made (because CentOS hasn't released yet).
* Traditionally, we wait because contributors may not have RHEL
entitilements so they need to have CentOS to test.
* Need entitlements for contributors to run RHEL (Easiest: Use RHEL,
SciLinux, Fermi Linux).
* Packages get added to one of the RH base repositories and then we have to
pull them from the EPEL repositories.
* Some people have extra layered products from RH and do not want us to
conflict with those. Other people do not have the layered products and
therefore want to be able to get those packages from EPEL.
* Overlap between EPEL and CentOS Extras.
Ideas to deal with Pain
* Get RHEL licenses for contributors. There is a process but it takes a
long while because of the tax problems for giving it to people. Current
procedure is that we get requests, we give the names to a person inside of
RH and then they eventually get back to us with licenses.
* Let's create EPIC: separate repo. 3 year lifespan instead of 10 year,
rhel life cycle.
* Debian Volatile repository and also Debian Backports. These repos
contain newer versions of packages, even for Debian Stable. Good for
packages which change all the time.
* Debian also has protection mechanism that says no major or no major.minor
version updates in apt (their yum equivalent).
* Red Hat already releases different lifecycle repositories (layered
products). Why not replicate what they do.
* What about implementing EPEL as a set of projects like Fedora Playground?
* Would change the guarantees and expectations of EPEL *quite* a bit.
* Maybe as the way to implement EPIC rather than EPEL.
* Policy change that allows for regular major changes in EPEL.
* Can't update at any times.
* Can make incompatible changes on the next point release of RHEL->CentOS.
* Example: When RHEL7.1 comes out we have a 30 day window to get packages
updated and new packages in that make incompatible changes.
* Archive 7.0. The toplevel 7 tree is a symlink to whatever point
release is latest.
* Formalize the governance of EPEL.
* Written policies of some sort for decision making
* Elections?
* Problems to solve:
* Don't want to just listen to the people who are loudest
* Don't want to just listen to the people who have been around the
longest
* Do we want automated testing of EPEL? yes. get it working on whatever
subset you can and we'll go on from there.
* Move to CentOS as build system? Gives us additional arches.
* Could we do an ISV program like quaid does?
--
Stephen J Smoogen.
2:10 p.m.
New subject: EPEL Fwd: Notes from .next discussion.
---------- Forwarded message ----------
From: Stephen John Smoogen <smooge(a)gmail.com>
Date: 12 August 2014 21:05
Subject: EPEL Notes from .next discussion.
To: EPEL development disccusion <epel-devel-list(a)redhat.com>,
epel-users-list(a)fedoraproject.org
At FLOCK this year, I did a short workshop on what was labeled EPEL.Next.
At that we went over a bit of what EPEL has done in the past, what its
current challenges are, and what could be its future. Toshio Kuratomi was
great in capturing what was said at the meeting and
EPEL
* Extra packages for enterprise linux
* Lots of name recognition now
* Formed as rhel5 was being released
people needed more packages. at the time there were a ton of addon repos.
None of the m were compatible with each other necessarily.
All the other repos, dag, freshrpms, etc overrode the base os in some
places.
Decided for EPEL, overriding RHEL was something we wouldn't do.
Started with RHEL3,4 and soon after release we had 5.
RHEL was supported 5-7years and we figured we could support EPEL packages
for the same length of time.
Early controversy - repotags (shortened name that can be in the packages.
[graph of Fedora machines measured by unique ips that hit mirrormanager vs
EPEL machines]
[graph shows steady growth in EPEL. Larger initial Fedora machines that
rose and then plateaued]
[graph shows a noticable dip every weekend. Can also see the dip in Fedora
numbers for the Incident.]
[graph is probably conservative in numbers as many enterprises
Second graph:
[Graph shows combined growth same as last graph.]
[Graph shows EPEL5 that has risen but plateaued about the time EPEL6 came
out.]
[graph shows that EPEL6 has been growing and has surpassed the EPEL5
plateau.]
Where are we now:
We support 3 arches.
RHEL4: 1258 srpms. EOL
RHEL5: 3651 srpms
RHEL6: 5449 srpms
RHEL7: 3100 srpms and growing fastest
For CENTOS7: the epel repofile will be shipped directly with centos for the
first time.
Current Problems:
* 10+ year support for RHEL now. Hard for EPEL packagers to commit to
doing the same.
* We have requests for both newer and older conflicting packages.
- Currently policy is not to have unpleasant surprises regarding
compatibility.
- puppet2.x vs puppet3.x
* Some people need major changes to build new software
* Some people need no changes in order to keep existing software running
* Developer burnout: People come in and after a few years, they are burnt
out because they can't upgrade the things that they want due to the
compatibility policies.
Where are we going?
Pain Points:
* Inability to yum downgrade because only include one old package in the
updates tree
* Harder for EPEL than Fedora because EPEL users have more need to
rollback if an incompatibility creeps in and because EPEL users may
schedule their releases
* Not every part of package repo is suitable for inclusion in anaconda
because we may not have dep closure in the subset of the repo.
* Which RHEL point release can also make a difference because the base
RHEL sometimes upgrades incompatibly and we don't keep a separate build for
both point releases.
* Can take a long time to push packages through bodhi. Can take 7 days to
push a CVE through bodhi. (global EPEL and Fedora problem)
* Not all maintainers interpret "Don't break compatibility" in the same
way.
* Many guidelines are verbal, not formal.
* Some maintainers are mainly Fedora maintainers and don't know what people
are using it for in EPEL. They respond to requests to install or update a
package out of a willingness to help but don't know what users need.
* No guideline enforcement.
* Takes a while for new EPEL releases to be brought out once a new RHEL
release is made (because CentOS hasn't released yet).
* Traditionally, we wait because contributors may not have RHEL
entitilements so they need to have CentOS to test.
* Need entitlements for contributors to run RHEL (Easiest: Use RHEL,
SciLinux, Fermi Linux).
* Packages get added to one of the RH base repositories and then we have to
pull them from the EPEL repositories.
* Some people have extra layered products from RH and do not want us to
conflict with those. Other people do not have the layered products and
therefore want to be able to get those packages from EPEL.
* Overlap between EPEL and CentOS Extras.
Ideas to deal with Pain
* Get RHEL licenses for contributors. There is a process but it takes a
long while because of the tax problems for giving it to people. Current
procedure is that we get requests, we give the names to a person inside of
RH and then they eventually get back to us with licenses.
* Let's create EPIC: separate repo. 3 year lifespan instead of 10 year,
rhel life cycle.
* Debian Volatile repository and also Debian Backports. These repos
contain newer versions of packages, even for Debian Stable. Good for
packages which change all the time.
* Debian also has protection mechanism that says no major or no major.minor
version updates in apt (their yum equivalent).
* Red Hat already releases different lifecycle repositories (layered
products). Why not replicate what they do.
* What about implementing EPEL as a set of projects like Fedora Playground?
* Would change the guarantees and expectations of EPEL *quite* a bit.
* Maybe as the way to implement EPIC rather than EPEL.
* Policy change that allows for regular major changes in EPEL.
* Can't update at any times.
* Can make incompatible changes on the next point release of RHEL->CentOS.
* Example: When RHEL7.1 comes out we have a 30 day window to get packages
updated and new packages in that make incompatible changes.
* Archive 7.0. The toplevel 7 tree is a symlink to whatever point
release is latest.
* Formalize the governance of EPEL.
* Written policies of some sort for decision making
* Elections?
* Problems to solve:
* Don't want to just listen to the people who are loudest
* Don't want to just listen to the people who have been around the
longest
* Do we want automated testing of EPEL? yes. get it working on whatever
subset you can and we'll go on from there.
* Move to CentOS as build system? Gives us additional arches.
* Could we do an ISV program like quaid does?
--
Stephen J Smoogen.
_______________________________________________
epel-devel mailing list
epel-devel(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/epel-devel
--
Stephen J Smoogen.
2:56 p.m.
On Tue, 12 Aug 2014 21:05:07 +0200
Stephen John Smoogen <smooge(a)gmail.com> wrote:
At FLOCK this year, I did a short workshop on what was labeled
EPEL.Next. At that we went over a bit of what EPEL has done in the
past, what its current challenges are, and what could be its future.
Toshio Kuratomi was great in capturing what was said at the meeting
and
EPEL
* Extra packages for enterprise linux
* Lots of name recognition now
* Formed as rhel5 was being released
people needed more packages. at the time there were a ton of addon
repos. None of the m were compatible with each other necessarily.
All the other repos, dag, freshrpms, etc overrode the base os in some
places.
Decided for EPEL, overriding RHEL was something we wouldn't do.
Started with RHEL3,4 and soon after release we had 5.
RHEL was supported 5-7years and we figured we could support EPEL
packages for the same length of time.
Early controversy - repotags (shortened name that can be in the
packages.
[graph of Fedora machines measured by unique ips that hit
mirrormanager vs EPEL machines]
[graph shows steady growth in EPEL. Larger initial Fedora machines
that rose and then plateaued]
[graph shows a noticable dip every weekend. Can also see the dip in
Fedora numbers for the Incident.]
[graph is probably conservative in numbers as many enterprises
Second graph:
[Graph shows combined growth same as last graph.]
[Graph shows EPEL5 that has risen but plateaued about the time EPEL6
came out.]
[graph shows that EPEL6 has been growing and has surpassed the EPEL5
plateau.]
Where are we now:
We support 3 arches.
I think you mean 3 releases. we have 3 arches for epel5 and
epel6 and 2
for epel7
RHEL4: 1258 srpms. EOL
RHEL5: 3651 srpms
RHEL6: 5449 srpms
RHEL7: 3100 srpms and growing fastest
For CENTOS7: the epel repofile will be shipped directly with centos
for the first time.
Current Problems:
* 10+ year support for RHEL now. Hard for EPEL packagers to commit to
doing the same.
* We have requests for both newer and older conflicting packages.
- Currently policy is not to have unpleasant surprises regarding
compatibility.
- puppet2.x vs puppet3.x
* Some people need major changes to build new software
* Some people need no changes in order to keep existing software
running
* Developer burnout: People come in and after a few years, they are
burnt out because they can't upgrade the things that they want due to
the compatibility policies.
I think i would like to see us add an epel-extras repo where things can
change faster. Things like mediawiki, puppet, etc could go in extras.
Where are we going?
Pain Points:
* Inability to yum downgrade because only include one old package in
the updates tree
* Harder for EPEL than Fedora because EPEL users have more need to
rollback if an incompatibility creeps in and because EPEL users may
schedule their releases
* Not every part of package repo is suitable for inclusion in anaconda
because we may not have dep closure in the subset of the repo.
* Which RHEL point release can also make a difference because the
base RHEL sometimes upgrades incompatibly and we don't keep a
separate build for both point releases.
* Can take a long time to push packages through bodhi. Can take 7
days to push a CVE through bodhi. (global EPEL and Fedora problem)
* Not all maintainers interpret "Don't break compatibility" in the
same way.
* Many guidelines are verbal, not formal.
* Some maintainers are mainly Fedora maintainers and don't know what
people are using it for in EPEL. They respond to requests to install
or update a package out of a willingness to help but don't know what
users need.
* No guideline enforcement.
* Takes a while for new EPEL releases to be brought out once a new
RHEL release is made (because CentOS hasn't released yet).
* Traditionally, we wait because contributors may not have RHEL
entitilements so they need to have CentOS to test.
* Need entitlements for contributors to run RHEL (Easiest: Use RHEL,
SciLinux, Fermi Linux).
* Packages get added to one of the RH base repositories and then we
have to pull them from the EPEL repositories.
* Some people have extra layered products from RH and do not want us
to conflict with those. Other people do not have the layered
products and therefore want to be able to get those packages from
EPEL.
* Overlap between EPEL and CentOS Extras.
Ideas to deal with Pain
* Get RHEL licenses for contributors. There is a process but it
takes a long while because of the tax problems for giving it to
people. Current procedure is that we get requests, we give the names
to a person inside of RH and then they eventually get back to us with
licenses.
* Let's create EPIC: separate repo. 3 year lifespan instead of 10
year, rhel life cycle.
* Debian Volatile repository and also Debian Backports. These repos
contain newer versions of packages, even for Debian Stable. Good for
packages which change all the time.
* Debian also has protection mechanism that says no major or no
major.minor version updates in apt (their yum equivalent).
* Red Hat already releases different lifecycle repositories (layered
products). Why not replicate what they do.
* What about implementing EPEL as a set of projects like Fedora
Playground?
* Would change the guarantees and expectations of EPEL *quite* a
bit.
* Maybe as the way to implement EPIC rather than EPEL.
* Policy change that allows for regular major changes in EPEL.
* Can't update at any times.
* Can make incompatible changes on the next point release of
RHEL->CentOS.
* Example: When RHEL7.1 comes out we have a 30 day window to get
packages updated and new packages in that make incompatible changes.
* Archive 7.0. The toplevel 7 tree is a symlink to whatever point
release is latest.
* Formalize the governance of EPEL.
* Written policies of some sort for decision making
* Elections?
* Problems to solve:
* Don't want to just listen to the people who are loudest
* Don't want to just listen to the people who have been around the
longest
* Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
* Move to CentOS as build system? Gives us additional arches.
* Could we do an ISV program like quaid does?
something to consider is moving epel to /opt/fedora/epel or somewhere
like it, then dealing with overlap etc should be simpler, but it would
take a massive change in packaging and could really only be done in
epel8
Dennis
6:25 p.m.
On 12 August 2014 21:56, Dennis Gilmore <dennis(a)ausil.us> wrote:
On Tue, 12 Aug 2014 21:05:07 +0200
Stephen John Smoogen <smooge(a)gmail.com> wrote:
> Where are we now:
> We support 3 arches.
I think you mean 3 releases. we have 3 arches for epel5 and epel6 and 2
for epel7
I actually meant arches as I forgot that i386 isn't a 7 item.
> RHEL4: 1258 srpms. EOL
> RHEL5: 3651 srpms
> RHEL6: 5449 srpms
> RHEL7: 3100 srpms and growing fastest
>
> * Maybe as the way to implement EPIC rather than EPEL.
> * Policy change that allows for regular major changes in EPEL.
> * Can't update at any times.
> * Can make incompatible changes on the next point release of
> RHEL->CentOS.
> * Example: When RHEL7.1 comes out we have a 30 day window to get
> packages updated and new packages in that make incompatible changes.
> * Archive 7.0. The toplevel 7 tree is a symlink to whatever point
> release is latest.
> * Formalize the governance of EPEL.
> * Written policies of some sort for decision making
> * Elections?
> * Problems to solve:
> * Don't want to just listen to the people who are loudest
> * Don't want to just listen to the people who have been around the
> longest
> * Do we want automated testing of EPEL? yes. get it working on
> whatever subset you can and we'll go on from there.
> * Move to CentOS as build system? Gives us additional arches.
>
> * Could we do an ISV program like quaid does?
something to consider is moving epel to /opt/fedora/epel or somewhere
like it, then dealing with overlap etc should be simpler, but it would
take a massive change in packaging and could really only be done in
epel8
I think that would be something with softwarecollections.org as that seems
in line with their way of packaging items. I think that for some sorts of
packages and libraries it makes sense for that, but I don't know if EPEL
would mix and match like that. What I would like to do is the following:
We move from our moving distribution to a point release cycle with a disk
layout like the following:
epel
epel/development/
epel/development/5
epel/development/6
epel/development/7
epel/beta/5.12
epel/beta/6.7
epel/beta/7.1
epel/releases
epel/releases/6.6
epel/releases/5.11
epel/releases/7.0
epel/updates
epel/updates/6.6
epel/updates/5.11
epel/updates/7.0
epel/updates/testing/
epel/updates/testing/6.6
epel/updates/testing/5.11
epel/updates/testing/7.0
epel/development would be like the current EPEL directories but without the
stringent requirements that packages are locked to a specific version for
the lifetime of the overall RHEL release. Instead whenever RHEL releases a
new dot release (7.0, 6.6, 5.11), EPEL would branch off the releases from
development to beta/7.0 or beta/6.6 etc. Packages would be built against
the point release and would need to be tested to get sufficient karma for
'release'. Once 6 weeks have passed from the RHEL point release, all
packages which had gotten enough karma and that had completed repository
trees would be put into epel/releases/6.6. Updates to that package would be
put into updates/testing/6.6 and then promoted to updates/6.6 when enough
karma had been given for an update. New packages which were added after the
point release would show up in updates following the process Fedora does
for new packages between point releases.
Later when 7.1, 6.7, 5.12 (or whatever they are called) comes out then the
process is repeated which will make sure that packages that aren't needed
enough to have sponsors will not be in EPEL and potentially broken and
large updates are possible so if python34 is in 7.0 but python35 is ready
for 7.2 it can replace it without problems (or similarly with ruby23 etc
etc). Once the next point release is made ready, the old point release will
be archived off to keep storage levels within reason.
I know that this proposal needs a lot more fleshing out, but I think it
covers the use cases many users of EPEL need for long term usage of
packages.
--
Stephen J Smoogen.
7:52 p.m.
On Aug 13 01:25, Stephen John Smoogen wrote:
I think that would be something with softwarecollections.org as that
seems
in line with their way of packaging items. I think that for some sorts of
packages and libraries it makes sense for that, but I don't know if EPEL
would mix and match like that. What I would like to do is the following:
We move from our moving distribution to a point release cycle with a disk
layout like the following:
epel
epel/development/
epel/development/5
epel/development/6
epel/development/7
epel/beta/5.12
epel/beta/6.7
epel/beta/7.1
epel/releases
epel/releases/6.6
epel/releases/5.11
epel/releases/7.0
epel/updates
epel/updates/6.6
epel/updates/5.11
epel/updates/7.0
epel/updates/testing/
epel/updates/testing/6.6
epel/updates/testing/5.11
epel/updates/testing/7.0
epel/development would be like the current EPEL directories but without the
stringent requirements that packages are locked to a specific version for
the lifetime of the overall RHEL release. Instead whenever RHEL releases a
new dot release (7.0, 6.6, 5.11), EPEL would branch off the releases from
development to beta/7.0 or beta/6.6 etc. Packages would be built against
the point release and would need to be tested to get sufficient karma for
'release'. Once 6 weeks have passed from the RHEL point release, all
packages which had gotten enough karma and that had completed repository
trees would be put into epel/releases/6.6. Updates to that package would be
put into updates/testing/6.6 and then promoted to updates/6.6 when enough
karma had been given for an update. New packages which were added after the
point release would show up in updates following the process Fedora does
for new packages between point releases.
Later when 7.1, 6.7, 5.12 (or whatever they are called) comes out then the
process is repeated which will make sure that packages that aren't needed
enough to have sponsors will not be in EPEL and potentially broken and
large updates are possible so if python34 is in 7.0 but python35 is ready
for 7.2 it can replace it without problems (or similarly with ruby23 etc
etc). Once the next point release is made ready, the old point release will
be archived off to keep storage levels within reason.
I know that this proposal needs a lot more fleshing out, but I think it
covers the use cases many users of EPEL need for long term usage of
packages.
--
Stephen J Smoogen.
Were there plans made (at flock or elsewhere) for regular EPEL-related meetings? I would
like
to chip in and help where I can. I think this proposal strikes a happy medium
between stability (within a point-release) and updated features.
--
Brian Stinson
bstinson(a)ksu.edu | IRC: bstinson | Bitbucket/Twitter: bstinsonmhk
8:37 p.m.
On 12 August 2014 18:52, Brian Stinson <bstinson(a)ksu.edu> wrote:
> I know that this proposal needs a lot more fleshing out, but I think it
> covers the use cases many users of EPEL need for long term usage of
> packages.
>
>
> --
> Stephen J Smoogen.
Were there plans made (at flock or elsewhere) for regular EPEL-related
meetings? I would like
to chip in and help where I can. I think this proposal strikes a happy
medium
between stability (within a point-release) and updated features.
It was brought up but not decided as not everyone can attend FLOCK and it
would seem unfair to people who could not attend.
I would like to get a fleshed out 'we are doing X' by August 19th 2014 so
that we can move forward.
--
Stephen J Smoogen.
7:28 a.m.
On Wed, 13 Aug 2014 01:25:04 +0200
Stephen John Smoogen <smooge(a)gmail.com> wrote:
On 12 August 2014 21:56, Dennis Gilmore <dennis(a)ausil.us>
wrote:
> On Tue, 12 Aug 2014 21:05:07 +0200
> Stephen John Smoogen <smooge(a)gmail.com> wrote:
>
>
> > Where are we now:
> > We support 3 arches.
> I think you mean 3 releases. we have 3 arches for epel5 and epel6
> and 2 for epel7
>
>
I actually meant arches as I forgot that i386 isn't a 7 item.
> > RHEL4: 1258 srpms. EOL
> > RHEL5: 3651 srpms
> > RHEL6: 5449 srpms
> > RHEL7: 3100 srpms and growing fastest
> >
> > * Maybe as the way to implement EPIC rather than EPEL.
> > * Policy change that allows for regular major changes in EPEL.
> > * Can't update at any times.
> > * Can make incompatible changes on the next point release of
> > RHEL->CentOS.
> > * Example: When RHEL7.1 comes out we have a 30 day window to get
> > packages updated and new packages in that make incompatible
> > changes.
What is the problem you're trying to solve here?
> > * Archive 7.0. The toplevel 7 tree is a symlink to
whatever
> > point release is latest.
> > * Formalize the governance of EPEL.
> > * Written policies of some sort for decision making
> > * Elections?
> > * Problems to solve:
> > * Don't want to just listen to the people who are loudest
> > * Don't want to just listen to the people who have been
> > around the longest
> > * Do we want automated testing of EPEL? yes. get it working on
> > whatever subset you can and we'll go on from there.
> > * Move to CentOS as build system? Gives us additional arches.
It doesn't actually give us any extra arches. I would like to work out
a way to enable EPEL contribution via a CentOS path and have EPEL be
part of both Fedora and CentOS.
> > * Could we do an ISV program like quaid does?
>
> something to consider is moving epel to /opt/fedora/epel or
> somewhere like it, then dealing with overlap etc should be simpler,
> but it would take a massive change in packaging and could really
> only be done in epel8
>
I think that would be something with softwarecollections.org as that
seems in line with their way of packaging items. I think that for
some sorts of packages and libraries it makes sense for that, but I
don't know if EPEL would mix and match like that. What I would like
to do is the following:
I disagree. FHS says that anything not part of the OS should go
in /opt/ EPEL is an add on and not part of the OS.
We move from our moving distribution to a point release cycle with a
disk layout like the following:
epel
epel/development/
epel/development/5
epel/development/6
epel/development/7
epel/beta/5.12
epel/beta/6.7
epel/beta/7.1
epel/releases
epel/releases/6.6
epel/releases/5.11
epel/releases/7.0
epel/updates
epel/updates/6.6
epel/updates/5.11
epel/updates/7.0
epel/updates/testing/
epel/updates/testing/6.6
epel/updates/testing/5.11
epel/updates/testing/7.0
epel/development would be like the current EPEL directories but
without the stringent requirements that packages are locked to a
specific version for the lifetime of the overall RHEL release.
Instead whenever RHEL releases a new dot release (7.0, 6.6, 5.11),
EPEL would branch off the releases from development to beta/7.0 or
beta/6.6 etc. Packages would be built against the point release and
would need to be tested to get sufficient karma for 'release'. Once 6
weeks have passed from the RHEL point release, all packages which had
gotten enough karma and that had completed repository trees would be
put into epel/releases/6.6. Updates to that package would be put into
updates/testing/6.6 and then promoted to updates/6.6 when enough
karma had been given for an update. New packages which were added
after the point release would show up in updates following the
process Fedora does for new packages between point releases.
This will make a much larger work load for releng. what extra resources
will we have? I find it quite confusing. I honestly do not like the
idea of having a static snapshot for releases. we have a few options
available to make older versions available. we can mash the tree with
every package ever released. or we can work on fixing the patches in
mash to have say the last 2 or 3 builds. I think we are better off
having epel as somthing stable where you can not put incompatible
changes for the life of the rhel release and epel-extras where things
are able to change quickly and in incompatible ways.
Later when 7.1, 6.7, 5.12 (or whatever they are called) comes out
then the process is repeated which will make sure that packages that
aren't needed enough to have sponsors will not be in EPEL and
potentially broken and large updates are possible so if python34 is
in 7.0 but python35 is ready for 7.2 it can replace it without
problems (or similarly with ruby23 etc etc). Once the next point
release is made ready, the old point release will be archived off to
keep storage levels within reason.
I know that this proposal needs a lot more fleshing out, but I think
it covers the use cases many users of EPEL need for long term usage of
packages.
Dennis
9:23 a.m.
On Tue, 12 Aug 2014 21:05:07 +0200
Stephen John Smoogen <smooge(a)gmail.com> wrote:
At FLOCK this year, I did a short workshop on what was labeled
EPEL.Next. At that we went over a bit of what EPEL has done in the
past, what its current challenges are, and what could be its future.
Toshio Kuratomi was great in capturing what was said at the meeting
and
...snip...
For CENTOS7: the epel repofile will be shipped directly with centos
for the first time.
Note that this will be in centos-extras (but thats enabled by default,
so just a nitpick)
...snip...
Where are we going?
Pain Points:
* Inability to yum downgrade because only include one old package in
the updates tree
Yeah, there were some recent mash patches that might help here.
Allowing us to keep 1 or 2 older ones too.
...snip lots of problems...
Ideas to deal with Pain
* Get RHEL licenses for contributors. There is a process but it
takes a long while because of the tax problems for giving it to
people. Current procedure is that we get requests, we give the names
to a person inside of RH and then they eventually get back to us with
licenses.
We are working to revamp/improve this.
* Let's create EPIC: separate repo. 3 year lifespan instead of
10
year, rhel life cycle.
* Debian Volatile repository and also Debian Backports. These repos
contain newer versions of packages, even for Debian Stable. Good for
packages which change all the time.
* Debian also has protection mechanism that says no major or no
major.minor version updates in apt (their yum equivalent).
* Red Hat already releases different lifecycle repositories (layered
products). Why not replicate what they do.
* What about implementing EPEL as a set of projects like Fedora
Playground?
* Would change the guarantees and expectations of EPEL *quite* a
bit.
* Maybe as the way to implement EPIC rather than EPEL.
I think we really do need to look at another repo for things that are
so rapid. It's going to be very hard to change user expectations for
EPEL now, since it's already entrenched.
I guess for my uses the parallel installable stuff works fine. I know
thats not the case for others though, so a more rapidly moving repo
would be better for them. I wonder if it wouldn't be good to coordinate
with CentOS folks and see where such a repo would make the most sense?
* Policy change that allows for regular major changes in EPEL.
* Can't update at any times.
* Can make incompatible changes on the next point release of
RHEL->CentOS.
* Example: When RHEL7.1 comes out we have a 30 day window to get
packages updated and new packages in that make incompatible changes.
The problem with that is that it's different than what we have always
done before. I suspect many people will be burned by incompatible
changes at that window.
* Archive 7.0. The toplevel 7 tree is a symlink to whatever
point
release is latest.
The idea of having a release tree was an interesting one. However, it's
going to be a lot more work. If we are saying "here's EPEL 7.0 release
repo" we would need to make sure it's well tested and has no issues.
ie, we would need to go through much of what Fedora does for a release.
Do we have any QA folks at all? ;)
I think more practical might be the idea of shipping multiple old
versions in the existing repos.
* Formalize the governance of EPEL.
* Written policies of some sort for decision making
* Elections?
* Problems to solve:
* Don't want to just listen to the people who are loudest
* Don't want to just listen to the people who have been around the
longest
I'd prefer to avoid voting.... those that do the work should have the
most input. Or perhaps those that show up. ;) There was some talk about
trying to do meetings again. I gave up in dispair last time I tried,
but would happily attend if someone else wants to organize them.
* Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
Yeah, I would love some automated testing.
* Move to CentOS as build system? Gives us additional arches.
Yeah, if they do a 32bit x86 and 32bit arm, that would give us more
arches.
kevin
11:11 a.m.
On 13 August 2014 06:28, Dennis Gilmore <dennis(a)ausil.us> wrote:
On Wed, 13 Aug 2014 01:25:04 +0200
Stephen John Smoogen <smooge(a)gmail.com> wrote:
> On 12 August 2014 21:56, Dennis Gilmore <dennis(a)ausil.us> wrote:
> > > * Example: When RHEL7.1 comes out we have a 30 day window to get
> > > packages updated and new packages in that make incompatible
> > > changes.
What is the problem you're trying to solve here?
The problem I am trying to solve is that we have packages that require
large deltas to stay patched and relevant to users. However users do not
want to have these appearing 'randomly' during a release cycle. So by
keying such large deltas to the Red Hat point releases we have a place
where the updates can occur and if old stuff is still wanted with new stuff
how to maintain both.
> > > * Archive 7.0. The toplevel 7 tree is a symlink
to whatever
> > > point release is latest.
> > > * Formalize the governance of EPEL.
> > > * Written policies of some sort for decision making
> > > * Elections?
> > > * Problems to solve:
> > > * Don't want to just listen to the people who are loudest
> > > * Don't want to just listen to the people who have been
> > > around the longest
> > > * Do we want automated testing of EPEL? yes. get it working on
> > > whatever subset you can and we'll go on from there.
> > > * Move to CentOS as build system? Gives us additional arches.
It doesn't actually give us any extra arches. I would like to work out
a way to enable EPEL contribution via a CentOS path and have EPEL be
part of both Fedora and CentOS.
if CentOS gets ARM32 and i386 then it would add 2 arches that people are
'wanting' but RHEL does not support.
> > > * Could we do an ISV program like quaid does?
> >
> > something to consider is moving epel to /opt/fedora/epel or
> > somewhere like it, then dealing with overlap etc should be simpler,
> > but it would take a massive change in packaging and could really
> > only be done in epel8
> >
>
> I think that would be something with softwarecollections.org as that
> seems in line with their way of packaging items. I think that for
> some sorts of packages and libraries it makes sense for that, but I
> don't know if EPEL would mix and match like that. What I would like
> to do is the following:
I disagree. FHS says that anything not part of the OS should go
in /opt/ EPEL is an add on and not part of the OS.
In the past we did not do this because we were bound by the Fedora
Packaging Rules. I don't have the energy or want to try and fight that
Windmill but I am not going to stand in someone who wants to.
> We move from our moving distribution to a point release cycle
with a
> disk layout like the following:
>
> epel
> epel/development/
> epel/development/5
> epel/development/6
> epel/development/7
> epel/beta/5.12
> epel/beta/6.7
> epel/beta/7.1
> epel/releases
> epel/releases/6.6
> epel/releases/5.11
> epel/releases/7.0
> epel/updates
> epel/updates/6.6
> epel/updates/5.11
> epel/updates/7.0
> epel/updates/testing/
> epel/updates/testing/6.6
> epel/updates/testing/5.11
> epel/updates/testing/7.0
>
> epel/development would be like the current EPEL directories but
> without the stringent requirements that packages are locked to a
> specific version for the lifetime of the overall RHEL release.
> Instead whenever RHEL releases a new dot release (7.0, 6.6, 5.11),
> EPEL would branch off the releases from development to beta/7.0 or
> beta/6.6 etc. Packages would be built against the point release and
> would need to be tested to get sufficient karma for 'release'. Once 6
> weeks have passed from the RHEL point release, all packages which had
> gotten enough karma and that had completed repository trees would be
> put into epel/releases/6.6. Updates to that package would be put into
> updates/testing/6.6 and then promoted to updates/6.6 when enough
> karma had been given for an update. New packages which were added
> after the point release would show up in updates following the
> process Fedora does for new packages between point releases.
This will make a much larger work load for releng. what extra resources
will we have? I find it quite confusing. I honestly do not like the
What extra resources do you need? I am sorry it is confusing.. I am trying
to communicate it while severely jetlagged. Part of this is to get a
conversation going so that we can find out why this would not work or would
work.
idea of having a static snapshot for releases. we have a few options
available to make older versions available. we can mash the tree with
every package ever released. or we can work on fixing the patches in
mash to have say the last 2 or 3 builds. I think we are better off
having epel as somthing stable where you can not put incompatible
changes for the life of the rhel release and epel-extras where things
are able to change quickly and in incompatible ways.
That was an alternative but I wasn't sure how much koji/bodhi/pkg changes
it would require or what packages would actually stay in EPEL versus
EPEL-extras as most packages have had to require a major change over a 10+
year lifetime that RHEL is going to.
Dennis
--
Stephen J Smoogen.
12:15 p.m.
On 08/13/2014 09:23 AM, Kevin Fenzi wrote:
On Tue, 12 Aug 2014 21:05:07 +0200
Stephen John Smoogen <smooge(a)gmail.com> wrote:
> At FLOCK this year, I did a short workshop on what was labeled
> EPEL.Next. At that we went over a bit of what EPEL has done in the
> past, what its current challenges are, and what could be its future.
> Toshio Kuratomi was great in capturing what was said at the meeting
> and
...snip...
> For CENTOS7: the epel repofile will be shipped directly with centos
> for the first time.
Note that this will be in centos-extras (but thats enabled by default,
so just a nitpick)
...snip...
> Where are we going?
> Pain Points:
> * Inability to yum downgrade because only include one old package in
> the updates tree
Yeah, there were some recent mash patches that might help here.
Allowing us to keep 1 or 2 older ones too.
...snip lots of problems...
> Ideas to deal with Pain
> * Get RHEL licenses for contributors. There is a process but it
> takes a long while because of the tax problems for giving it to
> people. Current procedure is that we get requests, we give the names
> to a person inside of RH and then they eventually get back to us with
> licenses.
We are working to revamp/improve this.
> * Let's create EPIC: separate repo. 3 year lifespan instead of 10
> year, rhel life cycle.
> * Debian Volatile repository and also Debian Backports. These repos
> contain newer versions of packages, even for Debian Stable. Good for
> packages which change all the time.
> * Debian also has protection mechanism that says no major or no
> major.minor version updates in apt (their yum equivalent).
> * Red Hat already releases different lifecycle repositories (layered
> products). Why not replicate what they do.
> * What about implementing EPEL as a set of projects like Fedora
> Playground?
> * Would change the guarantees and expectations of EPEL *quite* a
> bit.
> * Maybe as the way to implement EPIC rather than EPEL.
I think we really do need to look at another repo for things that are
so rapid. It's going to be very hard to change user expectations for
EPEL now, since it's already entrenched.
I don't think these two options are exclusive.
A second repository would certainly help keep original expectations of
epel in place, and is a good idea.
EPEL itself still provides a base for many projects, and could improve
upon existing user expectations.
I guess for my uses the parallel installable stuff works fine. I
know
thats not the case for others though, so a more rapidly moving repo
would be better for them. I wonder if it wouldn't be good to coordinate
with CentOS folks and see where such a repo would make the most sense?
I certainly think we should do this. It would benefit both our SIG
communities as well as other interested users.
The idea of having a release tree was an interesting one. However,
it's
going to be a lot more work. If we are saying "here's EPEL 7.0 release
repo" we would need to make sure it's well tested and has no issues.
ie, we would need to go through much of what Fedora does for a release.
Do we have any QA folks at all? ;)
I believe there's an effort for common QA underway. This would be an
area where we (CentOS and Fedora/EPEL) could coordinate.
I think more practical might be the idea of shipping multiple old
versions in the existing repos.
> * Formalize the governance of EPEL.
> * Written policies of some sort for decision making
> * Elections?
> * Problems to solve:
> * Don't want to just listen to the people who are loudest
> * Don't want to just listen to the people who have been around the
> longest
I'd prefer to avoid voting.... those that do the work should have the
most input. Or perhaps those that show up. ;) There was some talk about
trying to do meetings again. I gave up in dispair last time I tried,
but would happily attend if someone else wants to organize them.
> * Do we want automated testing of EPEL? yes. get it working on
> whatever subset you can and we'll go on from there.
Yeah, I would love some automated testing.
This should be possible in the reasonably near future.
> * Move to CentOS as build system? Gives us additional arches.
Yeah, if they do a 32bit x86 and 32bit arm, that would give us more
arches.
32bit x86 is definitely happening. I'm not certain about 32bit arm,
largely because of the largely varied hardware support requirements and
uboot.
64bit arm(uefi backed) is certainly on our roadmap.
--
Jim Perrin
The CentOS Project | http://www.centos.org
twitter: @BitIntegrity | GPG Key: FA09AD77
12:52 p.m.
Once upon a time, Stephen John Smoogen <smooge(a)gmail.com> said:
On 13 August 2014 06:28, Dennis Gilmore <dennis(a)ausil.us>
wrote:
> I disagree. FHS says that anything not part of the OS should go
> in /opt/ EPEL is an add on and not part of the OS.
>
In the past we did not do this because we were bound by the Fedora
Packaging Rules. I don't have the energy or want to try and fight that
Windmill but I am not going to stand in someone who wants to.
"part of the OS" is somewhat vague. My personal feeling (for the zero
it is worth) is that anything that is installed from yum repos
(especially non-conflicting repos) can be considered "part of the OS".
I usually consider /opt and /usr/local for stuff installed outside of
the OS-provided package management tools (even if said tools are
managing third-party repos).
The problem I'd see with moving stuff to /opt in EPEL is that it doesn't
get tested like that in Fedora, and causes definate diversion from
Fedora's "norms", so it puts more burden on packages. Also, it makes it
somewhat more difficult to move something from a Fedora install to a
RHEL/CentOS+EPEL install (because paths would be different).
--
Chris Adams <linux(a)cmadams.net>
1:04 a.m.
<snip lots of stuff>
Smooge thanks for the write-up.
Great points are being brought up by all parties here.
I think I'm +1 on a second repo that moves faster and can be incompatible
with the right type of announce-list. (maybe even put the uri for that list
in the .repo file or something). It also might be acceptable to allow
builds against SCL, or bundle libs if we want real speed and end-user
convenience. Obviously there's more than a little hand-waving there, so
don't consider that anything more than a spitball idea.
I like the idea of pairing with the CentOS folks for that, but I don't
really know where to start there. This ecosystem has lots of information if
you know exactly where to look. Otherwise, navigating it can be tricky.
As for EPEL meetings, if we could have them again, I'd really try to be a
part of it. Timing is crazy for a world-wide crew like this one.
I'm completely -1 on /opt/blah for EPEL.
8:49 a.m.
On Wed, Aug 13, 2014 at 11:04 PM, Michael Stahnke <stahnma(a)puppetlabs.com>
wrote:
><snip lots of stuff>
Smooge thanks for the write-up.
Great points are being brought up by all parties here.
I think I'm +1 on a second repo that moves faster and can be incompatible
with the right type of announce-list. (maybe even put the uri for that list
in the .repo file or something). It also might be acceptable to allow
builds against SCL, or bundle libs if we want real speed and end-user
convenience. Obviously there's more than a little hand-waving there, so
don't consider that anything more than a spitball idea.
I like the idea of pairing with the CentOS folks for that, but I don't
really know where to start there. This ecosystem has lots of information if
you know exactly where to look. Otherwise, navigating it can be tricky.
As for EPEL meetings, if we could have them again, I'd really try to be a
part of it. Timing is crazy for a world-wide crew like this one.
I'm completely -1 on /opt/blah for EPEL.
Thanks Michael for saving me writing an email -- I'm +1 on everything you
said here.
Regarding working with CentOS -- I know Jim's watching this list closely
and seems he plans to be at the proposed meeting this week, so it should be
easy to get some dialogue going.
-Jeff
3539
days inactive
3545
days old
epel-devel@lists.fedoraproject.org
12 comments
8 participants
participants (8)
-
Brian Stinson -
Chris Adams -
Dennis Gilmore -
Jeff Sheltren -
Jim Perrin -
Kevin Fenzi -
Michael Stahnke -
Stephen John Smoogen