Hello,
Following a discussion [1] on the devel list, Till encouraged me to propose a "packager experience" (or possibly "packager quality of life") Community objective to the Council.
Basically, the goal of this objective would be to organize people to work on various annoyances (be they minor or major) that currently exist in the packager workflow. This probably involves creating a new SIG or WG focused on improving the packager workflow-- identifying problems and figuring out solutions (likely by contributing to various tools and other projects in the packaging pipeline like fedpkg, or Koji/Bodhi/Pagure/etc.).
My understanding of the Objective proposal process here is that I create a draft page on the wiki and then start a discussion on this list, so I thought I would do just that. Here is a very early draft I wrote on the wiki:
https://fedoraproject.org/wiki/Objectives/Packager_Experience
I only recorded three issues on the wiki draft-- all to do with the retiring of pkgdb2, as that's what started this discussion. But I'm sure I can come up with more, and I'm equally sure the community would as well once we started a discussion. (I think, for example, that improving the package review process might also be something in the scope of this objective).
These problems might seem a bit small, or minor, to be worthy of a Community Objective to fix them, but I think that's part of the problem-- something like having to manually request Pagure API keys to create new packages (this was what the linked discussion on the devel list was about) might only be a minor annoyance, but that means it doesn't get attention, while bigger, more urgent things do. If we could collect a list of such minor annoyances and set up a plan to fix them-- and also set up a group or process responsible for fixing similar issues as they crop up in the future-- I think it would do a lot to improve the overall experience of packaging software in Fedora.
Thanks for your consideration!
Sincerely, Ben Rosser
[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
Hi Ben,
Thanks for drafting this. I think it's a great step in the right direction. You're right that these are all minor annoyances that aggregate to a bigger problem, so it's good to take a holistic approach to addressing them. Right now it seems focused on replacing the features lost by the deprecation of pkgdb. I'd encourage you to think about what other changes could be made to improve the overall packaging experience. Before it comes up for a vote, I'd like to see more concrete goals and a rough idea of how they'd be achieved. Obviously some of the specifics will become clear and evolve as the work goes on.
On Fri, 2018-12-21 at 16:23 -0500, Ben Cotton wrote:
Hi Ben,
Thanks for drafting this. I think it's a great step in the right direction. You're right that these are all minor annoyances that aggregate to a bigger problem, so it's good to take a holistic approach to addressing them. Right now it seems focused on replacing the features lost by the deprecation of pkgdb. I'd encourage you to think about what other changes could be made to improve the overall packaging experience. Before it comes up for a vote, I'd like to see more concrete goals and a rough idea of how they'd be achieved. Obviously some of the specifics will become clear and evolve as the work goes on.
I would like to echo Ben here, I think that a package experience objective should look at how to automate the tedious manual repetitive bits in maintaining packages and allow people to only have to deal with things that truely need a person to look at them. I would like to see steps from fully automating package reviews and updates, and letting packagers know when they have something needing attention, to rethinking the workflows entirely. I think that there could be a lot of tie in with the CI objective.
Dennis
On Fri, Dec 21, 2018 at 6:16 PM Dennis Gilmore dennis@ausil.us wrote:
On Fri, 2018-12-21 at 16:23 -0500, Ben Cotton wrote:
Hi Ben,
Thanks for drafting this. I think it's a great step in the right direction. You're right that these are all minor annoyances that aggregate to a bigger problem, so it's good to take a holistic approach to addressing them. Right now it seems focused on replacing the features lost by the deprecation of pkgdb. I'd encourage you to think about what other changes could be made to improve the overall packaging experience. Before it comes up for a vote, I'd like to see more concrete goals and a rough idea of how they'd be achieved. Obviously some of the specifics will become clear and evolve as the work goes on.
I would like to echo Ben here, I think that a package experience objective should look at how to automate the tedious manual repetitive bits in maintaining packages and allow people to only have to deal with things that truely need a person to look at them. I would like to see steps from fully automating package reviews and updates, and letting packagers know when they have something needing attention, to rethinking the workflows entirely. I think that there could be a lot of tie in with the CI objective.
Dennis
Hi Dennis and Ben,
Sorry for the delay in responding here; I admit I wasn't thinking too much about Fedora over vacation...
Thanks a lot for the feedback! I think it definitely makes sense to broaden the proposal to include more than just cleaning up after the pagure migration-- though I still think that's a good place to start, since it's something that happened relatively recently.
I do have some thoughts about other areas of the workflow that could be improved (like the review process, which I just added a section on to the wiki page), but I think I'll first take this back to the devel list to solicit interest and ideas from others. Then, perhaps, we can try to set some more concrete goals and create a more specific plan for accomplishing them.
(Part of this might be identifying what can be done in the next 12-18 months and what's a longer term goal to keep working on in the future, too. For instance, I think a short-term goal for the review process might be to get fedora-review ran automatically on all review tickets, but a longer-term goal might be to replace Bugzilla entirely and write a new reviews app. The short-term goal might be something we'd want to get done explicitly as part of the Objective, while the long-term goal might be something for the "packager quality of life" SIG/WG to keep working on over time.)
Cheers, Ben Rosser
On Tue, Jan 08, 2019 at 11:55:32AM -0500, Ben Rosser wrote:
Sorry for the delay in responding here; I admit I wasn't thinking too much about Fedora over vacation...
Thanks a lot for the feedback! I think it definitely makes sense to broaden the proposal to include more than just cleaning up after the pagure migration-- though I still think that's a good place to start, since it's something that happened relatively recently.
Welcome back, and thanks for reminding me to reply to this. First of all, I really appreciate your proposal. I agree with the others that a larger scope is appropriate for an objective (while at the same time, I'd like there to be definite measurable expected results, too).
I'd like to see this include packager experience for our new packaging technologies -- modularity, container images, flatpaks. Having tried to modularize a traditional Fedora desktop application recently, I know for sure that the packager experience is sub-optimal. The current Modularity objective itself is at the end of its timeframe and it's possible that instead of renewing that, those further improvments should be part of this bigger thing.
Additionally, some people have been working on a "source git" idea (https://github.com/user-cont/source-git). Do you think that might be in scope? There's a talk coming up on this at DevConf.cz: https://devconfcz2019.sched.com/event/Jch1/auto-maintain-your-package
On Tue, Jan 8, 2019 at 2:00 PM Matthew Miller mattdm@fedoraproject.org wrote:
On Tue, Jan 08, 2019 at 11:55:32AM -0500, Ben Rosser wrote:
Sorry for the delay in responding here; I admit I wasn't thinking too much about Fedora over vacation...
Thanks a lot for the feedback! I think it definitely makes sense to broaden the proposal to include more than just cleaning up after the pagure migration-- though I still think that's a good place to start, since it's something that happened relatively recently.
Welcome back, and thanks for reminding me to reply to this. First of all, I really appreciate your proposal. I agree with the others that a larger scope is appropriate for an objective (while at the same time, I'd like there to be definite measurable expected results, too).
I'd like to see this include packager experience for our new packaging technologies -- modularity, container images, flatpaks. Having tried to modularize a traditional Fedora desktop application recently, I know for sure that the packager experience is sub-optimal. The current Modularity objective itself is at the end of its timeframe and it's possible that instead of renewing that, those further improvments should be part of this bigger thing.
Sure, I think it makes sense for improvements to container/flatpak/module packaging to fall under the scope of this objective as well!
I admit that Modularity is part of my motivation for filing this objective-- I'd like to make sure we keep the packager experience in mind as the modularity rollout continues. My worry with modularity (which I've expressed before on the devel list) has been that the added overhead of creating and maintaining a module will end up detracting from the other benefits of being able to modularize stuff... so it would be nice to help make sure that doesn't happen!
Additionally, some people have been working on a "source git" idea (https://github.com/user-cont/source-git). Do you think that might be in scope? There's a talk coming up on this at DevConf.cz: https://devconfcz2019.sched.com/event/Jch1/auto-maintain-your-package
This is an interesting idea! My initial thought was that the majority of packages probably don't *need* something like this, as they don't differ much, if at all, from upstream. But I think even for those packages, being able to browse a package's sources through a git repository via Pagure rather than having to fetch and unpack the tarballs would be a major improvement. So yes, I think trying out source-git would be something that could fall under the scope of this objective.
Cheers, Ben Rosser
On Tue, Jan 8, 2019 at 2:00 PM Matthew Miller <mattdm(a)fedoraproject.org> wrote:
Sure, I think it makes sense for improvements to container/flatpak/module packaging to fall under the scope of this objective as well!
I admit that Modularity is part of my motivation for filing this objective-- I'd like to make sure we keep the packager experience in mind as the modularity rollout continues. My worry with modularity (which I've expressed before on the devel list) has been that the added overhead of creating and maintaining a module will end up detracting from the other benefits of being able to modularize stuff... so it would be nice to help make sure that doesn't happen!
This is an interesting idea! My initial thought was that the majority of packages probably don't *need* something like this, as they don't differ much, if at all, from upstream. But I think even for those packages, being able to browse a package's sources through a git repository via Pagure rather than having to fetch and unpack the tarballs would be a major improvement. So yes, I think trying out source-git would be something that could fall under the scope of this objective.
Cheers, Ben Rosser
Ben, thank you for putting this proposal together! It's definitely the step to the right direction.
I'm one of the people behind source-git. At the moment we are in a phase where we are trying to figure out our goals and milestones. Our team will get together and make a plan around devconf. We'll then announce it to Fedora to get the discussion going (feedback is very important). The topics we are talking about are: * Packaging automation (as Dennis pointed out -- automate tedious tasks) * Rawhide gating and increasing its stability * Getting upstream projects closer to downstream * Easier packaging * On-demand side tags (what Richard mentioned) * More convenient packager experience * And plenty of others, this is just off top of my head
We'll likely won't address all. So it's important for all of us to be on the same page and work together so we don't step on each other's toes and there are clear expectations between each other.
Matt, thanks for the pointers, I'm not on this list actually so I'm glad you made me aware.
Tomas
On Fri, Jan 11, 2019 at 09:29:00AM -0000, Tomas Tomecek wrote:
On Tue, Jan 8, 2019 at 2:00 PM Matthew Miller <mattdm(a)fedoraproject.org> wrote:
Sure, I think it makes sense for improvements to container/flatpak/module packaging to fall under the scope of this objective as well!
I admit that Modularity is part of my motivation for filing this objective-- I'd like to make sure we keep the packager experience in mind as the modularity rollout continues. My worry with modularity (which I've expressed before on the devel list) has been that the added overhead of creating and maintaining a module will end up detracting from the other benefits of being able to modularize stuff... so it would be nice to help make sure that doesn't happen!
This is an interesting idea! My initial thought was that the majority of packages probably don't *need* something like this, as they don't differ much, if at all, from upstream. But I think even for those packages, being able to browse a package's sources through a git repository via Pagure rather than having to fetch and unpack the tarballs would be a major improvement. So yes, I think trying out source-git would be something that could fall under the scope of this objective.
Cheers, Ben Rosser
Ben, thank you for putting this proposal together! It's definitely the step to the right direction.
I'm one of the people behind source-git. At the moment we are in a phase where we are trying to figure out our goals and milestones. Our team will get together and make a plan around devconf. We'll then announce it to Fedora to get the discussion going (feedback is very important). The topics we are talking about are:
- Packaging automation (as Dennis pointed out -- automate tedious tasks)
- Rawhide gating and increasing its stability
- On-demand side tags (what Richard mentioned)
From my understanding on the current plan for rawhide gating, it involves on-demand side-tags for multi-package update in rawhide. So that's two items in one :)
Pierre
On Fri, Jan 11, 2019 at 7:54 AM Pierre-Yves Chibon pingou@pingoured.fr wrote:
- Rawhide gating and increasing its stability
- On-demand side tags (what Richard mentioned)
From my understanding on the current plan for rawhide gating, it involves on-demand side-tags for multi-package update in rawhide. So that's two items in one :)
Yes, that would take care of the most obvious use case but I still thing they would be useful for large packages with complex dependencies, both from a single packager workflow but also for collaboration.
I don't want to update a package like FreeCAD just using buildroot overrides, one because it's cumbersome, but also I don't want others accidentally building against a package I might need to update or change, especially if it's someone else's.
To me the side tag lets you experiment in a somewhat contained environment until you get everything "right". Once I get it working I can approach the maintainer of the dependent packages and ask "Are you OK with me making these changes?"
Hopefully I explained that well enough... <goes to get more coffee>
Thanks, Richard
I don't know if we need a voting system but how do we decide what makes it to the wiki?
One thing I would like to see is... Ok, wait... Make that two :)
1. Seamless arbitrary branches
I want to be able create an arbitrary branch so I can experiment and test without commiting to a "real" branch instead of having to resort to other local/remote git repositories or git stash/apply. The system should be smart enough to remember what I branched from so if I create a branch blah-test from master that if I issue a "fedpkg scratch-build" it should be smart enough to build for rawhide without me having to specify a buildroot.
"real" branches should be protected from deletion.
2. Easy side tags
I have used my personal machine (I'm sure we all do) when working on ABI breaking changes to make sure everything rebuilds, but then I only end up building for x86_64 before going "live". I know we don't have a lot of arch specific issues these days but it still makes me nervous.
This could also tie into #1 so you can test changes in a side tag and them merge them back together in a real branch, do the builds, and if all goes well, push them as candidates and maybe automagically into a bodhi update.
Thanks, Richard
On Tue, Dec 18, 2018 at 05:43:57PM +0000, Ben Rosser wrote:
Following a discussion [1] on the devel list, Till encouraged me to propose a "packager experience" (or possibly "packager quality of life") Community objective to the Council.
Hey Ben, what do you think about having an in-person hackfest to hash out and launch this objective? Is there a time in the next few months that might work for you?
council-discuss@lists.fedoraproject.org
-
Ben Cotton -
Ben Rosser -
Dennis Gilmore -
Matthew Miller -
Pierre-Yves Chibon -
Richard Shaw -
Tomas Tomecek