At FLOCK this year, I did a short workshop on what was labeled EPEL.Next. At that we went over a bit of what EPEL has done in the past, what its current challenges are, and what could be its future. Toshio Kuratomi was great in capturing what was said at the meeting and
EPEL
* Extra packages for enterprise linux * Lots of name recognition now * Formed as rhel5 was being released people needed more packages. at the time there were a ton of addon repos. None of the m were compatible with each other necessarily. All the other repos, dag, freshrpms, etc overrode the base os in some places.
Decided for EPEL, overriding RHEL was something we wouldn't do.
Started with RHEL3,4 and soon after release we had 5.
RHEL was supported 5-7years and we figured we could support EPEL packages for the same length of time.
Early controversy - repotags (shortened name that can be in the packages.
[graph of Fedora machines measured by unique ips that hit mirrormanager vs EPEL machines] [graph shows steady growth in EPEL. Larger initial Fedora machines that rose and then plateaued] [graph shows a noticable dip every weekend. Can also see the dip in Fedora numbers for the Incident.] [graph is probably conservative in numbers as many enterprises
Second graph: [Graph shows combined growth same as last graph.] [Graph shows EPEL5 that has risen but plateaued about the time EPEL6 came out.] [graph shows that EPEL6 has been growing and has surpassed the EPEL5 plateau.]
Where are we now: We support 3 arches.
RHEL4: 1258 srpms. EOL RHEL5: 3651 srpms RHEL6: 5449 srpms RHEL7: 3100 srpms and growing fastest
For CENTOS7: the epel repofile will be shipped directly with centos for the first time.
Current Problems: * 10+ year support for RHEL now. Hard for EPEL packagers to commit to doing the same. * We have requests for both newer and older conflicting packages. - Currently policy is not to have unpleasant surprises regarding compatibility. - puppet2.x vs puppet3.x * Some people need major changes to build new software * Some people need no changes in order to keep existing software running * Developer burnout: People come in and after a few years, they are burnt out because they can't upgrade the things that they want due to the compatibility policies.
Where are we going? Pain Points: * Inability to yum downgrade because only include one old package in the updates tree * Harder for EPEL than Fedora because EPEL users have more need to rollback if an incompatibility creeps in and because EPEL users may schedule their releases * Not every part of package repo is suitable for inclusion in anaconda because we may not have dep closure in the subset of the repo. * Which RHEL point release can also make a difference because the base RHEL sometimes upgrades incompatibly and we don't keep a separate build for both point releases. * Can take a long time to push packages through bodhi. Can take 7 days to push a CVE through bodhi. (global EPEL and Fedora problem) * Not all maintainers interpret "Don't break compatibility" in the same way. * Many guidelines are verbal, not formal. * Some maintainers are mainly Fedora maintainers and don't know what people are using it for in EPEL. They respond to requests to install or update a package out of a willingness to help but don't know what users need. * No guideline enforcement. * Takes a while for new EPEL releases to be brought out once a new RHEL release is made (because CentOS hasn't released yet). * Traditionally, we wait because contributors may not have RHEL entitilements so they need to have CentOS to test. * Need entitlements for contributors to run RHEL (Easiest: Use RHEL, SciLinux, Fermi Linux). * Packages get added to one of the RH base repositories and then we have to pull them from the EPEL repositories. * Some people have extra layered products from RH and do not want us to conflict with those. Other people do not have the layered products and therefore want to be able to get those packages from EPEL. * Overlap between EPEL and CentOS Extras.
Ideas to deal with Pain * Get RHEL licenses for contributors. There is a process but it takes a long while because of the tax problems for giving it to people. Current procedure is that we get requests, we give the names to a person inside of RH and then they eventually get back to us with licenses. * Let's create EPIC: separate repo. 3 year lifespan instead of 10 year, rhel life cycle. * Debian Volatile repository and also Debian Backports. These repos contain newer versions of packages, even for Debian Stable. Good for packages which change all the time. * Debian also has protection mechanism that says no major or no major.minor version updates in apt (their yum equivalent). * Red Hat already releases different lifecycle repositories (layered products). Why not replicate what they do. * What about implementing EPEL as a set of projects like Fedora Playground? * Would change the guarantees and expectations of EPEL *quite* a bit. * Maybe as the way to implement EPIC rather than EPEL. * Policy change that allows for regular major changes in EPEL. * Can't update at any times. * Can make incompatible changes on the next point release of RHEL->CentOS. * Example: When RHEL7.1 comes out we have a 30 day window to get packages updated and new packages in that make incompatible changes. * Archive 7.0. The toplevel 7 tree is a symlink to whatever point release is latest. * Formalize the governance of EPEL. * Written policies of some sort for decision making * Elections? * Problems to solve: * Don't want to just listen to the people who are loudest * Don't want to just listen to the people who have been around the longest * Do we want automated testing of EPEL? yes. get it working on whatever subset you can and we'll go on from there. * Move to CentOS as build system? Gives us additional arches.
* Could we do an ISV program like quaid does?
---------- Forwarded message ---------- From: Stephen John Smoogen smooge@gmail.com Date: 12 August 2014 21:05 Subject: EPEL Notes from .next discussion. To: EPEL development disccusion epel-devel-list@redhat.com, epel-users-list@fedoraproject.org
At FLOCK this year, I did a short workshop on what was labeled EPEL.Next. At that we went over a bit of what EPEL has done in the past, what its current challenges are, and what could be its future. Toshio Kuratomi was great in capturing what was said at the meeting and
EPEL
* Extra packages for enterprise linux * Lots of name recognition now * Formed as rhel5 was being released people needed more packages. at the time there were a ton of addon repos. None of the m were compatible with each other necessarily. All the other repos, dag, freshrpms, etc overrode the base os in some places.
Decided for EPEL, overriding RHEL was something we wouldn't do.
Started with RHEL3,4 and soon after release we had 5.
RHEL was supported 5-7years and we figured we could support EPEL packages for the same length of time.
Early controversy - repotags (shortened name that can be in the packages.
[graph of Fedora machines measured by unique ips that hit mirrormanager vs EPEL machines] [graph shows steady growth in EPEL. Larger initial Fedora machines that rose and then plateaued] [graph shows a noticable dip every weekend. Can also see the dip in Fedora numbers for the Incident.] [graph is probably conservative in numbers as many enterprises
Second graph: [Graph shows combined growth same as last graph.] [Graph shows EPEL5 that has risen but plateaued about the time EPEL6 came out.] [graph shows that EPEL6 has been growing and has surpassed the EPEL5 plateau.]
Where are we now: We support 3 arches.
RHEL4: 1258 srpms. EOL RHEL5: 3651 srpms RHEL6: 5449 srpms RHEL7: 3100 srpms and growing fastest
For CENTOS7: the epel repofile will be shipped directly with centos for the first time.
Current Problems: * 10+ year support for RHEL now. Hard for EPEL packagers to commit to doing the same. * We have requests for both newer and older conflicting packages. - Currently policy is not to have unpleasant surprises regarding compatibility. - puppet2.x vs puppet3.x * Some people need major changes to build new software * Some people need no changes in order to keep existing software running * Developer burnout: People come in and after a few years, they are burnt out because they can't upgrade the things that they want due to the compatibility policies.
Where are we going? Pain Points: * Inability to yum downgrade because only include one old package in the updates tree * Harder for EPEL than Fedora because EPEL users have more need to rollback if an incompatibility creeps in and because EPEL users may schedule their releases * Not every part of package repo is suitable for inclusion in anaconda because we may not have dep closure in the subset of the repo. * Which RHEL point release can also make a difference because the base RHEL sometimes upgrades incompatibly and we don't keep a separate build for both point releases. * Can take a long time to push packages through bodhi. Can take 7 days to push a CVE through bodhi. (global EPEL and Fedora problem) * Not all maintainers interpret "Don't break compatibility" in the same way. * Many guidelines are verbal, not formal. * Some maintainers are mainly Fedora maintainers and don't know what people are using it for in EPEL. They respond to requests to install or update a package out of a willingness to help but don't know what users need. * No guideline enforcement. * Takes a while for new EPEL releases to be brought out once a new RHEL release is made (because CentOS hasn't released yet). * Traditionally, we wait because contributors may not have RHEL entitilements so they need to have CentOS to test. * Need entitlements for contributors to run RHEL (Easiest: Use RHEL, SciLinux, Fermi Linux). * Packages get added to one of the RH base repositories and then we have to pull them from the EPEL repositories. * Some people have extra layered products from RH and do not want us to conflict with those. Other people do not have the layered products and therefore want to be able to get those packages from EPEL. * Overlap between EPEL and CentOS Extras.
Ideas to deal with Pain * Get RHEL licenses for contributors. There is a process but it takes a long while because of the tax problems for giving it to people. Current procedure is that we get requests, we give the names to a person inside of RH and then they eventually get back to us with licenses. * Let's create EPIC: separate repo. 3 year lifespan instead of 10 year, rhel life cycle. * Debian Volatile repository and also Debian Backports. These repos contain newer versions of packages, even for Debian Stable. Good for packages which change all the time. * Debian also has protection mechanism that says no major or no major.minor version updates in apt (their yum equivalent). * Red Hat already releases different lifecycle repositories (layered products). Why not replicate what they do. * What about implementing EPEL as a set of projects like Fedora Playground? * Would change the guarantees and expectations of EPEL *quite* a bit. * Maybe as the way to implement EPIC rather than EPEL. * Policy change that allows for regular major changes in EPEL. * Can't update at any times. * Can make incompatible changes on the next point release of RHEL->CentOS. * Example: When RHEL7.1 comes out we have a 30 day window to get packages updated and new packages in that make incompatible changes. * Archive 7.0. The toplevel 7 tree is a symlink to whatever point release is latest. * Formalize the governance of EPEL. * Written policies of some sort for decision making * Elections? * Problems to solve: * Don't want to just listen to the people who are loudest * Don't want to just listen to the people who have been around the longest * Do we want automated testing of EPEL? yes. get it working on whatever subset you can and we'll go on from there. * Move to CentOS as build system? Gives us additional arches.
* Could we do an ISV program like quaid does?
On Tue, 12 Aug 2014 21:05:07 +0200 Stephen John Smoogen smooge@gmail.com wrote:
At FLOCK this year, I did a short workshop on what was labeled EPEL.Next. At that we went over a bit of what EPEL has done in the past, what its current challenges are, and what could be its future. Toshio Kuratomi was great in capturing what was said at the meeting and
EPEL
- Extra packages for enterprise linux
- Lots of name recognition now
- Formed as rhel5 was being released
people needed more packages. at the time there were a ton of addon repos. None of the m were compatible with each other necessarily. All the other repos, dag, freshrpms, etc overrode the base os in some places.
Decided for EPEL, overriding RHEL was something we wouldn't do.
Started with RHEL3,4 and soon after release we had 5.
RHEL was supported 5-7years and we figured we could support EPEL packages for the same length of time.
Early controversy - repotags (shortened name that can be in the packages.
[graph of Fedora machines measured by unique ips that hit mirrormanager vs EPEL machines] [graph shows steady growth in EPEL. Larger initial Fedora machines that rose and then plateaued] [graph shows a noticable dip every weekend. Can also see the dip in Fedora numbers for the Incident.] [graph is probably conservative in numbers as many enterprises
Second graph: [Graph shows combined growth same as last graph.] [Graph shows EPEL5 that has risen but plateaued about the time EPEL6 came out.] [graph shows that EPEL6 has been growing and has surpassed the EPEL5 plateau.]
Where are we now: We support 3 arches.
I think you mean 3 releases. we have 3 arches for epel5 and epel6 and 2 for epel7
RHEL4: 1258 srpms. EOL RHEL5: 3651 srpms RHEL6: 5449 srpms RHEL7: 3100 srpms and growing fastest
For CENTOS7: the epel repofile will be shipped directly with centos for the first time.
Current Problems:
- 10+ year support for RHEL now. Hard for EPEL packagers to commit to
doing the same.
- We have requests for both newer and older conflicting packages.
- Currently policy is not to have unpleasant surprises regarding
compatibility.
- puppet2.x vs puppet3.x
- Some people need major changes to build new software
- Some people need no changes in order to keep existing software
running
- Developer burnout: People come in and after a few years, they are
burnt out because they can't upgrade the things that they want due to the compatibility policies.
I think i would like to see us add an epel-extras repo where things can change faster. Things like mediawiki, puppet, etc could go in extras.
Where are we going? Pain Points:
- Inability to yum downgrade because only include one old package in
the updates tree
- Harder for EPEL than Fedora because EPEL users have more need to
rollback if an incompatibility creeps in and because EPEL users may schedule their releases
- Not every part of package repo is suitable for inclusion in anaconda
because we may not have dep closure in the subset of the repo.
- Which RHEL point release can also make a difference because the
base RHEL sometimes upgrades incompatibly and we don't keep a separate build for both point releases.
- Can take a long time to push packages through bodhi. Can take 7
days to push a CVE through bodhi. (global EPEL and Fedora problem)
- Not all maintainers interpret "Don't break compatibility" in the
same way.
- Many guidelines are verbal, not formal.
- Some maintainers are mainly Fedora maintainers and don't know what
people are using it for in EPEL. They respond to requests to install or update a package out of a willingness to help but don't know what users need.
- No guideline enforcement.
- Takes a while for new EPEL releases to be brought out once a new
RHEL release is made (because CentOS hasn't released yet).
- Traditionally, we wait because contributors may not have RHEL
entitilements so they need to have CentOS to test.
- Need entitlements for contributors to run RHEL (Easiest: Use RHEL,
SciLinux, Fermi Linux).
- Packages get added to one of the RH base repositories and then we
have to pull them from the EPEL repositories.
- Some people have extra layered products from RH and do not want us
to conflict with those. Other people do not have the layered products and therefore want to be able to get those packages from EPEL.
- Overlap between EPEL and CentOS Extras.
Ideas to deal with Pain
- Get RHEL licenses for contributors. There is a process but it
takes a long while because of the tax problems for giving it to people. Current procedure is that we get requests, we give the names to a person inside of RH and then they eventually get back to us with licenses.
- Let's create EPIC: separate repo. 3 year lifespan instead of 10
year, rhel life cycle.
- Debian Volatile repository and also Debian Backports. These repos
contain newer versions of packages, even for Debian Stable. Good for packages which change all the time.
- Debian also has protection mechanism that says no major or no
major.minor version updates in apt (their yum equivalent).
- Red Hat already releases different lifecycle repositories (layered
products). Why not replicate what they do.
- What about implementing EPEL as a set of projects like Fedora
Playground?
- Would change the guarantees and expectations of EPEL *quite* a
bit.
- Maybe as the way to implement EPIC rather than EPEL.
- Policy change that allows for regular major changes in EPEL.
- Can't update at any times.
- Can make incompatible changes on the next point release of
RHEL->CentOS.
- Example: When RHEL7.1 comes out we have a 30 day window to get
packages updated and new packages in that make incompatible changes. * Archive 7.0. The toplevel 7 tree is a symlink to whatever point release is latest.
- Formalize the governance of EPEL.
- Written policies of some sort for decision making
- Elections?
- Problems to solve:
- Don't want to just listen to the people who are loudest
- Don't want to just listen to the people who have been around the
longest
- Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
Move to CentOS as build system? Gives us additional arches.
Could we do an ISV program like quaid does?
something to consider is moving epel to /opt/fedora/epel or somewhere like it, then dealing with overlap etc should be simpler, but it would take a massive change in packaging and could really only be done in epel8
Dennis
On 12 August 2014 21:56, Dennis Gilmore dennis@ausil.us wrote:
On Tue, 12 Aug 2014 21:05:07 +0200 Stephen John Smoogen smooge@gmail.com wrote:
Where are we now: We support 3 arches.
I think you mean 3 releases. we have 3 arches for epel5 and epel6 and 2 for epel7
I actually meant arches as I forgot that i386 isn't a 7 item.
RHEL4: 1258 srpms. EOL RHEL5: 3651 srpms RHEL6: 5449 srpms RHEL7: 3100 srpms and growing fastest
- Maybe as the way to implement EPIC rather than EPEL.
- Policy change that allows for regular major changes in EPEL.
- Can't update at any times.
- Can make incompatible changes on the next point release of
RHEL->CentOS.
- Example: When RHEL7.1 comes out we have a 30 day window to get
packages updated and new packages in that make incompatible changes. * Archive 7.0. The toplevel 7 tree is a symlink to whatever point release is latest.
- Formalize the governance of EPEL.
- Written policies of some sort for decision making
- Elections?
- Problems to solve:
- Don't want to just listen to the people who are loudest
- Don't want to just listen to the people who have been around the
longest
- Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
Move to CentOS as build system? Gives us additional arches.
Could we do an ISV program like quaid does?
something to consider is moving epel to /opt/fedora/epel or somewhere like it, then dealing with overlap etc should be simpler, but it would take a massive change in packaging and could really only be done in epel8
I think that would be something with softwarecollections.org as that seems in line with their way of packaging items. I think that for some sorts of packages and libraries it makes sense for that, but I don't know if EPEL would mix and match like that. What I would like to do is the following:
We move from our moving distribution to a point release cycle with a disk layout like the following:
epel epel/development/ epel/development/5 epel/development/6 epel/development/7 epel/beta/5.12 epel/beta/6.7 epel/beta/7.1 epel/releases epel/releases/6.6 epel/releases/5.11 epel/releases/7.0 epel/updates epel/updates/6.6 epel/updates/5.11 epel/updates/7.0 epel/updates/testing/ epel/updates/testing/6.6 epel/updates/testing/5.11 epel/updates/testing/7.0
epel/development would be like the current EPEL directories but without the stringent requirements that packages are locked to a specific version for the lifetime of the overall RHEL release. Instead whenever RHEL releases a new dot release (7.0, 6.6, 5.11), EPEL would branch off the releases from development to beta/7.0 or beta/6.6 etc. Packages would be built against the point release and would need to be tested to get sufficient karma for 'release'. Once 6 weeks have passed from the RHEL point release, all packages which had gotten enough karma and that had completed repository trees would be put into epel/releases/6.6. Updates to that package would be put into updates/testing/6.6 and then promoted to updates/6.6 when enough karma had been given for an update. New packages which were added after the point release would show up in updates following the process Fedora does for new packages between point releases.
Later when 7.1, 6.7, 5.12 (or whatever they are called) comes out then the process is repeated which will make sure that packages that aren't needed enough to have sponsors will not be in EPEL and potentially broken and large updates are possible so if python34 is in 7.0 but python35 is ready for 7.2 it can replace it without problems (or similarly with ruby23 etc etc). Once the next point release is made ready, the old point release will be archived off to keep storage levels within reason.
I know that this proposal needs a lot more fleshing out, but I think it covers the use cases many users of EPEL need for long term usage of packages.
On Aug 13 01:25, Stephen John Smoogen wrote:
I think that would be something with softwarecollections.org as that seems in line with their way of packaging items. I think that for some sorts of packages and libraries it makes sense for that, but I don't know if EPEL would mix and match like that. What I would like to do is the following:
We move from our moving distribution to a point release cycle with a disk layout like the following:
epel epel/development/ epel/development/5 epel/development/6 epel/development/7 epel/beta/5.12 epel/beta/6.7 epel/beta/7.1 epel/releases epel/releases/6.6 epel/releases/5.11 epel/releases/7.0 epel/updates epel/updates/6.6 epel/updates/5.11 epel/updates/7.0 epel/updates/testing/ epel/updates/testing/6.6 epel/updates/testing/5.11 epel/updates/testing/7.0
epel/development would be like the current EPEL directories but without the stringent requirements that packages are locked to a specific version for the lifetime of the overall RHEL release. Instead whenever RHEL releases a new dot release (7.0, 6.6, 5.11), EPEL would branch off the releases from development to beta/7.0 or beta/6.6 etc. Packages would be built against the point release and would need to be tested to get sufficient karma for 'release'. Once 6 weeks have passed from the RHEL point release, all packages which had gotten enough karma and that had completed repository trees would be put into epel/releases/6.6. Updates to that package would be put into updates/testing/6.6 and then promoted to updates/6.6 when enough karma had been given for an update. New packages which were added after the point release would show up in updates following the process Fedora does for new packages between point releases.
Later when 7.1, 6.7, 5.12 (or whatever they are called) comes out then the process is repeated which will make sure that packages that aren't needed enough to have sponsors will not be in EPEL and potentially broken and large updates are possible so if python34 is in 7.0 but python35 is ready for 7.2 it can replace it without problems (or similarly with ruby23 etc etc). Once the next point release is made ready, the old point release will be archived off to keep storage levels within reason.
I know that this proposal needs a lot more fleshing out, but I think it covers the use cases many users of EPEL need for long term usage of packages.
-- Stephen J Smoogen.
Were there plans made (at flock or elsewhere) for regular EPEL-related meetings? I would like to chip in and help where I can. I think this proposal strikes a happy medium between stability (within a point-release) and updated features.
-- Brian Stinson bstinson@ksu.edu | IRC: bstinson | Bitbucket/Twitter: bstinsonmhk
On 12 August 2014 18:52, Brian Stinson bstinson@ksu.edu wrote:
I know that this proposal needs a lot more fleshing out, but I think it covers the use cases many users of EPEL need for long term usage of packages.
-- Stephen J Smoogen.
Were there plans made (at flock or elsewhere) for regular EPEL-related meetings? I would like to chip in and help where I can. I think this proposal strikes a happy medium between stability (within a point-release) and updated features.
It was brought up but not decided as not everyone can attend FLOCK and it would seem unfair to people who could not attend.
I would like to get a fleshed out 'we are doing X' by August 19th 2014 so that we can move forward.
On Wed, 13 Aug 2014 01:25:04 +0200 Stephen John Smoogen smooge@gmail.com wrote:
On 12 August 2014 21:56, Dennis Gilmore dennis@ausil.us wrote:
On Tue, 12 Aug 2014 21:05:07 +0200 Stephen John Smoogen smooge@gmail.com wrote:
Where are we now: We support 3 arches.
I think you mean 3 releases. we have 3 arches for epel5 and epel6 and 2 for epel7
I actually meant arches as I forgot that i386 isn't a 7 item.
RHEL4: 1258 srpms. EOL RHEL5: 3651 srpms RHEL6: 5449 srpms RHEL7: 3100 srpms and growing fastest
- Maybe as the way to implement EPIC rather than EPEL.
- Policy change that allows for regular major changes in EPEL.
- Can't update at any times.
- Can make incompatible changes on the next point release of
RHEL->CentOS.
- Example: When RHEL7.1 comes out we have a 30 day window to get
packages updated and new packages in that make incompatible changes.
What is the problem you're trying to solve here?
* Archive 7.0. The toplevel 7 tree is a symlink to whatever
point release is latest.
- Formalize the governance of EPEL.
- Written policies of some sort for decision making
- Elections?
- Problems to solve:
- Don't want to just listen to the people who are loudest
- Don't want to just listen to the people who have been
around the longest
- Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
- Move to CentOS as build system? Gives us additional arches.
It doesn't actually give us any extra arches. I would like to work out a way to enable EPEL contribution via a CentOS path and have EPEL be part of both Fedora and CentOS.
- Could we do an ISV program like quaid does?
something to consider is moving epel to /opt/fedora/epel or somewhere like it, then dealing with overlap etc should be simpler, but it would take a massive change in packaging and could really only be done in epel8
I think that would be something with softwarecollections.org as that seems in line with their way of packaging items. I think that for some sorts of packages and libraries it makes sense for that, but I don't know if EPEL would mix and match like that. What I would like to do is the following:
I disagree. FHS says that anything not part of the OS should go in /opt/ EPEL is an add on and not part of the OS.
We move from our moving distribution to a point release cycle with a disk layout like the following:
epel epel/development/ epel/development/5 epel/development/6 epel/development/7 epel/beta/5.12 epel/beta/6.7 epel/beta/7.1 epel/releases epel/releases/6.6 epel/releases/5.11 epel/releases/7.0 epel/updates epel/updates/6.6 epel/updates/5.11 epel/updates/7.0 epel/updates/testing/ epel/updates/testing/6.6 epel/updates/testing/5.11 epel/updates/testing/7.0
epel/development would be like the current EPEL directories but without the stringent requirements that packages are locked to a specific version for the lifetime of the overall RHEL release. Instead whenever RHEL releases a new dot release (7.0, 6.6, 5.11), EPEL would branch off the releases from development to beta/7.0 or beta/6.6 etc. Packages would be built against the point release and would need to be tested to get sufficient karma for 'release'. Once 6 weeks have passed from the RHEL point release, all packages which had gotten enough karma and that had completed repository trees would be put into epel/releases/6.6. Updates to that package would be put into updates/testing/6.6 and then promoted to updates/6.6 when enough karma had been given for an update. New packages which were added after the point release would show up in updates following the process Fedora does for new packages between point releases.
This will make a much larger work load for releng. what extra resources will we have? I find it quite confusing. I honestly do not like the idea of having a static snapshot for releases. we have a few options available to make older versions available. we can mash the tree with every package ever released. or we can work on fixing the patches in mash to have say the last 2 or 3 builds. I think we are better off having epel as somthing stable where you can not put incompatible changes for the life of the rhel release and epel-extras where things are able to change quickly and in incompatible ways.
Later when 7.1, 6.7, 5.12 (or whatever they are called) comes out then the process is repeated which will make sure that packages that aren't needed enough to have sponsors will not be in EPEL and potentially broken and large updates are possible so if python34 is in 7.0 but python35 is ready for 7.2 it can replace it without problems (or similarly with ruby23 etc etc). Once the next point release is made ready, the old point release will be archived off to keep storage levels within reason.
I know that this proposal needs a lot more fleshing out, but I think it covers the use cases many users of EPEL need for long term usage of packages.
Dennis
On 13 August 2014 06:28, Dennis Gilmore dennis@ausil.us wrote:
On Wed, 13 Aug 2014 01:25:04 +0200 Stephen John Smoogen smooge@gmail.com wrote:
On 12 August 2014 21:56, Dennis Gilmore dennis@ausil.us wrote:
- Example: When RHEL7.1 comes out we have a 30 day window to get
packages updated and new packages in that make incompatible changes.
What is the problem you're trying to solve here?
The problem I am trying to solve is that we have packages that require large deltas to stay patched and relevant to users. However users do not want to have these appearing 'randomly' during a release cycle. So by keying such large deltas to the Red Hat point releases we have a place where the updates can occur and if old stuff is still wanted with new stuff how to maintain both.
* Archive 7.0. The toplevel 7 tree is a symlink to whatever
point release is latest.
- Formalize the governance of EPEL.
- Written policies of some sort for decision making
- Elections?
- Problems to solve:
- Don't want to just listen to the people who are loudest
- Don't want to just listen to the people who have been
around the longest
- Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
- Move to CentOS as build system? Gives us additional arches.
It doesn't actually give us any extra arches. I would like to work out a way to enable EPEL contribution via a CentOS path and have EPEL be part of both Fedora and CentOS.
if CentOS gets ARM32 and i386 then it would add 2 arches that people are 'wanting' but RHEL does not support.
- Could we do an ISV program like quaid does?
something to consider is moving epel to /opt/fedora/epel or somewhere like it, then dealing with overlap etc should be simpler, but it would take a massive change in packaging and could really only be done in epel8
I think that would be something with softwarecollections.org as that seems in line with their way of packaging items. I think that for some sorts of packages and libraries it makes sense for that, but I don't know if EPEL would mix and match like that. What I would like to do is the following:
I disagree. FHS says that anything not part of the OS should go in /opt/ EPEL is an add on and not part of the OS.
In the past we did not do this because we were bound by the Fedora Packaging Rules. I don't have the energy or want to try and fight that Windmill but I am not going to stand in someone who wants to.
We move from our moving distribution to a point release cycle with a disk layout like the following:
epel epel/development/ epel/development/5 epel/development/6 epel/development/7 epel/beta/5.12 epel/beta/6.7 epel/beta/7.1 epel/releases epel/releases/6.6 epel/releases/5.11 epel/releases/7.0 epel/updates epel/updates/6.6 epel/updates/5.11 epel/updates/7.0 epel/updates/testing/ epel/updates/testing/6.6 epel/updates/testing/5.11 epel/updates/testing/7.0
epel/development would be like the current EPEL directories but without the stringent requirements that packages are locked to a specific version for the lifetime of the overall RHEL release. Instead whenever RHEL releases a new dot release (7.0, 6.6, 5.11), EPEL would branch off the releases from development to beta/7.0 or beta/6.6 etc. Packages would be built against the point release and would need to be tested to get sufficient karma for 'release'. Once 6 weeks have passed from the RHEL point release, all packages which had gotten enough karma and that had completed repository trees would be put into epel/releases/6.6. Updates to that package would be put into updates/testing/6.6 and then promoted to updates/6.6 when enough karma had been given for an update. New packages which were added after the point release would show up in updates following the process Fedora does for new packages between point releases.
This will make a much larger work load for releng. what extra resources will we have? I find it quite confusing. I honestly do not like the
What extra resources do you need? I am sorry it is confusing.. I am trying to communicate it while severely jetlagged. Part of this is to get a conversation going so that we can find out why this would not work or would work.
idea of having a static snapshot for releases. we have a few options available to make older versions available. we can mash the tree with every package ever released. or we can work on fixing the patches in mash to have say the last 2 or 3 builds. I think we are better off having epel as somthing stable where you can not put incompatible changes for the life of the rhel release and epel-extras where things are able to change quickly and in incompatible ways.
That was an alternative but I wasn't sure how much koji/bodhi/pkg changes it would require or what packages would actually stay in EPEL versus EPEL-extras as most packages have had to require a major change over a 10+ year lifetime that RHEL is going to.
Dennis
Once upon a time, Stephen John Smoogen smooge@gmail.com said:
On 13 August 2014 06:28, Dennis Gilmore dennis@ausil.us wrote:
I disagree. FHS says that anything not part of the OS should go in /opt/ EPEL is an add on and not part of the OS.
In the past we did not do this because we were bound by the Fedora Packaging Rules. I don't have the energy or want to try and fight that Windmill but I am not going to stand in someone who wants to.
"part of the OS" is somewhat vague. My personal feeling (for the zero it is worth) is that anything that is installed from yum repos (especially non-conflicting repos) can be considered "part of the OS". I usually consider /opt and /usr/local for stuff installed outside of the OS-provided package management tools (even if said tools are managing third-party repos).
The problem I'd see with moving stuff to /opt in EPEL is that it doesn't get tested like that in Fedora, and causes definate diversion from Fedora's "norms", so it puts more burden on packages. Also, it makes it somewhat more difficult to move something from a Fedora install to a RHEL/CentOS+EPEL install (because paths would be different).
<snip lots of stuff>
Smooge thanks for the write-up.
Great points are being brought up by all parties here.
I think I'm +1 on a second repo that moves faster and can be incompatible with the right type of announce-list. (maybe even put the uri for that list in the .repo file or something). It also might be acceptable to allow builds against SCL, or bundle libs if we want real speed and end-user convenience. Obviously there's more than a little hand-waving there, so don't consider that anything more than a spitball idea.
I like the idea of pairing with the CentOS folks for that, but I don't really know where to start there. This ecosystem has lots of information if you know exactly where to look. Otherwise, navigating it can be tricky.
As for EPEL meetings, if we could have them again, I'd really try to be a part of it. Timing is crazy for a world-wide crew like this one.
I'm completely -1 on /opt/blah for EPEL.
On Wed, Aug 13, 2014 at 11:04 PM, Michael Stahnke stahnma@puppetlabs.com wrote:
<snip lots of stuff>
Smooge thanks for the write-up.
Great points are being brought up by all parties here.
I think I'm +1 on a second repo that moves faster and can be incompatible with the right type of announce-list. (maybe even put the uri for that list in the .repo file or something). It also might be acceptable to allow builds against SCL, or bundle libs if we want real speed and end-user convenience. Obviously there's more than a little hand-waving there, so don't consider that anything more than a spitball idea.
I like the idea of pairing with the CentOS folks for that, but I don't really know where to start there. This ecosystem has lots of information if you know exactly where to look. Otherwise, navigating it can be tricky.
As for EPEL meetings, if we could have them again, I'd really try to be a part of it. Timing is crazy for a world-wide crew like this one.
I'm completely -1 on /opt/blah for EPEL.
Thanks Michael for saving me writing an email -- I'm +1 on everything you said here.
Regarding working with CentOS -- I know Jim's watching this list closely and seems he plans to be at the proposed meeting this week, so it should be easy to get some dialogue going.
-Jeff
On Tue, 12 Aug 2014 21:05:07 +0200 Stephen John Smoogen smooge@gmail.com wrote:
At FLOCK this year, I did a short workshop on what was labeled EPEL.Next. At that we went over a bit of what EPEL has done in the past, what its current challenges are, and what could be its future. Toshio Kuratomi was great in capturing what was said at the meeting and
...snip...
For CENTOS7: the epel repofile will be shipped directly with centos for the first time.
Note that this will be in centos-extras (but thats enabled by default, so just a nitpick)
...snip...
Where are we going? Pain Points:
- Inability to yum downgrade because only include one old package in
the updates tree
Yeah, there were some recent mash patches that might help here. Allowing us to keep 1 or 2 older ones too.
...snip lots of problems...
Ideas to deal with Pain
- Get RHEL licenses for contributors. There is a process but it
takes a long while because of the tax problems for giving it to people. Current procedure is that we get requests, we give the names to a person inside of RH and then they eventually get back to us with licenses.
We are working to revamp/improve this.
- Let's create EPIC: separate repo. 3 year lifespan instead of 10
year, rhel life cycle.
- Debian Volatile repository and also Debian Backports. These repos
contain newer versions of packages, even for Debian Stable. Good for packages which change all the time.
- Debian also has protection mechanism that says no major or no
major.minor version updates in apt (their yum equivalent).
- Red Hat already releases different lifecycle repositories (layered
products). Why not replicate what they do.
- What about implementing EPEL as a set of projects like Fedora
Playground?
- Would change the guarantees and expectations of EPEL *quite* a
bit.
- Maybe as the way to implement EPIC rather than EPEL.
I think we really do need to look at another repo for things that are so rapid. It's going to be very hard to change user expectations for EPEL now, since it's already entrenched.
I guess for my uses the parallel installable stuff works fine. I know thats not the case for others though, so a more rapidly moving repo would be better for them. I wonder if it wouldn't be good to coordinate with CentOS folks and see where such a repo would make the most sense?
- Policy change that allows for regular major changes in EPEL.
- Can't update at any times.
- Can make incompatible changes on the next point release of
RHEL->CentOS.
- Example: When RHEL7.1 comes out we have a 30 day window to get
packages updated and new packages in that make incompatible changes.
The problem with that is that it's different than what we have always done before. I suspect many people will be burned by incompatible changes at that window.
* Archive 7.0. The toplevel 7 tree is a symlink to whatever point
release is latest.
The idea of having a release tree was an interesting one. However, it's going to be a lot more work. If we are saying "here's EPEL 7.0 release repo" we would need to make sure it's well tested and has no issues. ie, we would need to go through much of what Fedora does for a release. Do we have any QA folks at all? ;)
I think more practical might be the idea of shipping multiple old versions in the existing repos.
- Formalize the governance of EPEL.
- Written policies of some sort for decision making
- Elections?
- Problems to solve:
- Don't want to just listen to the people who are loudest
- Don't want to just listen to the people who have been around the
longest
I'd prefer to avoid voting.... those that do the work should have the most input. Or perhaps those that show up. ;) There was some talk about trying to do meetings again. I gave up in dispair last time I tried, but would happily attend if someone else wants to organize them.
- Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
Yeah, I would love some automated testing.
- Move to CentOS as build system? Gives us additional arches.
Yeah, if they do a 32bit x86 and 32bit arm, that would give us more arches.
kevin
On 08/13/2014 09:23 AM, Kevin Fenzi wrote:
On Tue, 12 Aug 2014 21:05:07 +0200 Stephen John Smoogen smooge@gmail.com wrote:
At FLOCK this year, I did a short workshop on what was labeled EPEL.Next. At that we went over a bit of what EPEL has done in the past, what its current challenges are, and what could be its future. Toshio Kuratomi was great in capturing what was said at the meeting and
...snip...
For CENTOS7: the epel repofile will be shipped directly with centos for the first time.
Note that this will be in centos-extras (but thats enabled by default, so just a nitpick)
...snip...
Where are we going? Pain Points:
- Inability to yum downgrade because only include one old package in
the updates tree
Yeah, there were some recent mash patches that might help here. Allowing us to keep 1 or 2 older ones too.
...snip lots of problems...
Ideas to deal with Pain
- Get RHEL licenses for contributors. There is a process but it
takes a long while because of the tax problems for giving it to people. Current procedure is that we get requests, we give the names to a person inside of RH and then they eventually get back to us with licenses.
We are working to revamp/improve this.
- Let's create EPIC: separate repo. 3 year lifespan instead of 10
year, rhel life cycle.
- Debian Volatile repository and also Debian Backports. These repos
contain newer versions of packages, even for Debian Stable. Good for packages which change all the time.
- Debian also has protection mechanism that says no major or no
major.minor version updates in apt (their yum equivalent).
- Red Hat already releases different lifecycle repositories (layered
products). Why not replicate what they do.
- What about implementing EPEL as a set of projects like Fedora
Playground?
- Would change the guarantees and expectations of EPEL *quite* a
bit.
- Maybe as the way to implement EPIC rather than EPEL.
I think we really do need to look at another repo for things that are so rapid. It's going to be very hard to change user expectations for EPEL now, since it's already entrenched.
I don't think these two options are exclusive.
A second repository would certainly help keep original expectations of epel in place, and is a good idea.
EPEL itself still provides a base for many projects, and could improve upon existing user expectations.
I guess for my uses the parallel installable stuff works fine. I know thats not the case for others though, so a more rapidly moving repo would be better for them. I wonder if it wouldn't be good to coordinate with CentOS folks and see where such a repo would make the most sense?
I certainly think we should do this. It would benefit both our SIG communities as well as other interested users.
The idea of having a release tree was an interesting one. However, it's going to be a lot more work. If we are saying "here's EPEL 7.0 release repo" we would need to make sure it's well tested and has no issues. ie, we would need to go through much of what Fedora does for a release. Do we have any QA folks at all? ;)
I believe there's an effort for common QA underway. This would be an area where we (CentOS and Fedora/EPEL) could coordinate.
I think more practical might be the idea of shipping multiple old versions in the existing repos.
- Formalize the governance of EPEL.
- Written policies of some sort for decision making
- Elections?
- Problems to solve:
- Don't want to just listen to the people who are loudest
- Don't want to just listen to the people who have been around the
longest
I'd prefer to avoid voting.... those that do the work should have the most input. Or perhaps those that show up. ;) There was some talk about trying to do meetings again. I gave up in dispair last time I tried, but would happily attend if someone else wants to organize them.
- Do we want automated testing of EPEL? yes. get it working on
whatever subset you can and we'll go on from there.
Yeah, I would love some automated testing.
This should be possible in the reasonably near future.
- Move to CentOS as build system? Gives us additional arches.
Yeah, if they do a 32bit x86 and 32bit arm, that would give us more arches.
32bit x86 is definitely happening. I'm not certain about 32bit arm, largely because of the largely varied hardware support requirements and uboot.
64bit arm(uefi backed) is certainly on our roadmap.
epel-devel@lists.fedoraproject.org