On Mon, May 04, 2020 at 05:05:02PM +0200, Tomas Tomecek wrote:
>Let’s talk about dist-git, as a place where we work. For us,
>packagers, it’s a well-known place. Yet for newcomers, it may take a
>while to learn all the details. Even though we operate with projects
>in a dist-git repository, the layout doesn’t resemble the respective
>There is a multitude of tasks we tend to perform in a dist-git repo:
>* Bumping a release field for sake of a rebuild.
>* Updating to the latest upstream release.
>* Resolving CVEs.
>* Fixing bugs by…
> * Changing a spec file.
> * Pulling a commit from upstream.
> * Or even backporting a commit.
>* And more...
>For some tasks, the workflow is just fine and pretty straightforward.
>But for the other, it’s very gruesome - the moment you need to touch
>patch files, the horror comes in. The fact that we operate with patch
>files, in a git repository, is just mind-boggling to me.
>Luckily, we have tooling which supports the repository layout -
>`fedpkg prep`, `srpm` or `mockbuild` are such handy commands - you can
>easily inspect the source tree or make sure your local change builds.
>Where am I getting with this?
>Over the years there have been multiple tools created to improve the
>rdopkg [r], rpkg-util [ru], tito [t] and probably much much more (e.g.
>the way Fedora kernel developers work on kernel [k]).
>In the packit project, we work in source-git repositories. These are
>pretty much upstream repositories combined with Fedora downstream
>packaging files. An example: I recently added a project called nyancat
>[n] to Fedora. I have worked [w] on packaging the project in the
>GitHub repo and then just pushed the changes to dist-git using packit
>tooling. These source-git repositories can live anywhere: we have
>support for GitHub right now and are working on supporting pagure.
>Would there be an interest within the community, as opt-in, to have
>such source-git repositories created for respective dist-git
>repositories? The idea is that you would work in the source-git repo
>and then let packit handle synchronization with a respective dist-git
>repo. Our aim is to provide the contribution experience you have in
>GitHub when working on your packages. Dist-git would still be the
>authoritative source and a place where official builds are done - the
>source-git repo would work as a way to collaborate. We also don’t have
>plans right now to integrate packit into fedpkg.
>The main reason I am sending this is to gather feedback from all of
>you whether there is an interest in such a workflow. We don’t have
>concrete plans for Fedora right now but based on your feedback we
This is an interesting idea and it is a direction I would like to see dist-git
move. I do not think it's possible to find a one size fits all approach since
every package has and needs varying workflows. And we should be flexible to
let teams and developers do what they need to do. For me, moving spec files
upstream does not seem that appealing from a package maintenance standpoint.
I still like the clear distinction between the upstream project and the
'Fedora bits' that make it a package we ship. But that might not be the case
for every package.
I have read through this thread as of 3pm Boston time on 08-May and there's a
lot of great feedback. I wanted to offer my own thoughts on what I'd like to
see related to this topic:
WHAT I WANT TO BE ABLE TO DO:
* View Fedora's dist-git repos as authoritative for packages built for
Fedora. That is, I want to see a package on my Fedora system and be able to
visit its dist-git repo to see how it's packaged.
* Make the lookaside cache optional. For SourceX lines, I want to be able to
specify a git URL to a specific tag. fedpkg should use git archive to
include that in the SRPM. e.g.:
* If we offer the above, honor signed git tags for verification at build time.
* Make PatchX lines optional. In dist-git, I should be able to set a remote
pointing to the upstream repo. Then do the Fedora work on the appropriate
Fedora branch. SourceX should still become a tarball using git archive and
the tag. Patches should be automatically generated for SRPM construction
using git format-patch or something comparing the Fedora dist-git branch
with the remote branch. Multiple remotes should be possible should new and
old versions of the upstream project need to be supported. Fedora dist-git
branches should know their remote.
* I still want to be able to do 'fedpkg srpm' and get a standalone
ready-to-build SRPM file that I can carry around.
* Possibly extend fedpkg to helper package maintainers submit patches from the
package to the upstream project.
I very much like the ideas that you described although I have a
slightly different view on implementation.
- I would be very happy if the original spec files in dist-git
described the building process. I.e. that I can open any spec file
anywhere on src.fp.o and I will know what happens when that spec file
gets built. I think this is a useful feature to quickly navigate
ourselves and don't rely on context and implicit knowledge too much.
As a concrete example, if something is going to dynamically generate
patch files from git commits and also the respective "Patch:" lines, I
think there should be something in the spec file telling me that this
will happen. The advantage is explicitness and I could imagine that
something could also have parameters that would e.g. enable to omit
certain commit from patch generation, i.e. it has imho better
flexibility when compared to some tool in our build pipeline that
would just blindly paste Patch lines at a certain place in spec file.
I think such behavior would be unexpected and unpleasant because the
spec file itself doesn't give you the full picture of the build
process and you need to have some extra knowledge that newcomers will
not have. You can say: "Not a big deal" but these things add up. Also
order of declarations sometimes plays a role in rpm spec file so
pasting some lines somewhere might not always be a valid operation.
- I think we should have an "importer" service that would download
sources directly from upstream and placed them into dist-git's
lookaside on demand, based on their checksum. It could also
automatically check signatures when importing. When import is done,
"sources" file would be updated. In future, it could also import git
tags instead of just tarballs (i.e. it would basically just clone git
repo and reset it to specific tag) and these would be the "source-git"
repos (of course their more manual population by git push should be
- I think we could try to have workflows that require no force pushes.
They are imho not really good for cooperation among more than 1 people
> PRs in dist-git would be more meaningful to me if we were able to have the
> upstream repo as a remote in dist-git and our branches just an extension of
> David Cantrell <dcantrell(a)redhat.com>
> Red Hat, Inc. | Boston, MA | EST5EDT
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: