Good morning, I'd like to announce the creation of Fedora Source-git SIG:
https://fedoraproject.org/wiki/SIGs/Source-git
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Please head to the SIG wiki page to learn more about our proposed MVP. We are looking for maintainers of Fedora Linux packages who'd be interested in being early adopters and give us feedback during the development process. You don't need to do any coding unless you want to :)
With this email, I'd like to invite you to our first SIG meeting hosted on freenode IRC next week. The details will be specified by this Friday, please fill in "when is good" so we can pick a suitable time:
https://whenisgood.net/bsyhsqq
Regards, Tomas
On 14.04.2021 10:45, Tomas Tomecek wrote:
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Debian style with full local copies of repositories?
I don't think this is a good idea, because I don't want to clone upstream repositories and store my SPECs in them.
On Wed, Apr 14, 2021 at 2:52 PM Vitaly Zaitsev via devel devel@lists.fedoraproject.org wrote:
On 14.04.2021 10:45, Tomas Tomecek wrote:
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Debian style with full local copies of repositories?
I don't think this is a good idea, because I don't want to clone upstream repositories and store my SPECs in them.
Could you, please, be more constructive and say what the actual problems are for you with such repositories?
I understand that upstream repositories can have a long history - we could optimize and have shallow copies or only fetch recent upstream history if needed. Also, one would ideally only clone once and kept fetching new changes.
Thank you,
Tomas
-- Sincerely, Vitaly Zaitsev (vitaly@easycoding.org) _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 14.04.2021 16:27, Tomas Tomecek wrote:
Could you, please, be more constructive and say what the actual problems are for you with such repositories?
1. Some upstream repositories (Qt, Chromium, Linux kernel) are very huge (more than 100 GiB). I don't want to download them from upstream and then upload to Fedora.
2. Keeping such huge repositories will take up a lot of disk space on the maintainer's computers.
3. Rebase problem. Maintainers will need do a manual rebase on every upstream release/commit. Rebasing to the next major version will be a serious problem for the projects with a lot of of downstream patches.
4. Some project have a weird git workflow between minor releases. I know at least one project that uses Git tags with cherry-picked and manually backported commits. Such detached tags cannot be merged into master branch without resolving merge conflicts.
5. Force pushes must be enabled, which is too dangerous IMO.
I understand that upstream repositories can have a long history - we could optimize and have shallow copies or only fetch recent upstream history if needed. Also, one would ideally only clone once and kept fetching new changes.
Do you want to force switch all Fedora packages to a new workflow?
On Wed, Apr 14, 2021 at 04:53:06PM +0200, Vitaly Zaitsev via devel wrote:
On 14.04.2021 16:27, Tomas Tomecek wrote:
Could you, please, be more constructive and say what the actual problems are for you with such repositories?
- Some upstream repositories (Qt, Chromium, Linux kernel) are very huge
(more than 100 GiB). I don't want to download them from upstream and then upload to Fedora.
- Keeping such huge repositories will take up a lot of disk space on the
maintainer's computers.
Bear in mind that many maintainers already use a source-git workflow with Fedora, and are also involved in the upstream project. IOW, they already have these upstream repos present locally, and also be hosting it in some remote git server outside normal Fedora dist-git.
source-git isn't proposed to be forced on all packages, and drive-by contributors can also take a shallow clone instead of fetching all history. Any possible "fedpkg" integration could potentially default to a shallow clone.
- Rebase problem. Maintainers will need do a manual rebase on every
upstream release/commit. Rebasing to the next major version will be a serious problem for the projects with a lot of of downstream patches.
That's not my experiance. The cases where I know of maintainers are using a source-git model with Fedora / RHEL already, are doing so precisely because it makes ongoing maint and rebasing waaaaay easier than with dist-git, especially when there are alot of downstream patches (100's or even 1000's).
- Some project have a weird git workflow between minor releases. I know at
least one project that uses Git tags with cherry-picked and manually backported commits. Such detached tags cannot be merged into master branch without resolving merge conflicts.
I woudn't expect Fedora to track the git-master in most cases. You generally still want Fedora to be base off releases, so you'd want to track starting fron a release tag or branch.
- Force pushes must be enabled, which is too dangerous IMO.
There are several ways you can do source git and they don't all imply force pushes, so I think this is probably inventing a problem where none yet exists.
One example approach to source-git I've used...
Rather than having source-git branch names matching dist-git, use a different naming convention that is based off the upstream version primarily.
eg if upstream has v1.0 and v1.2 tags, I might have a 'v1.0-f33' branch, and if I rebase Fedora to v1.2, then I'd just switch to using a v1.2-f33 branch instead. The v1.0-f33 history remains intact forever, no force push required to rebase to new version.
If upstream has stable branches v1.0-maint and v1.2-maint, then v1.0-f33 branch might be initialized with v1.0-maint content. If upstream adds more commits to v1.0-maint branch, then you wouldn't force push v1.0-f33, you'd just do a git merge to pull them in.
I understand that upstream repositories can have a long history - we could optimize and have shallow copies or only fetch recent upstream history if needed. Also, one would ideally only clone once and kept fetching new changes.
Do you want to force switch all Fedora packages to a new workflow?
There's no mention of forcing everyone to switch to source-git. Some upstreams don't even use git.
Regards, Daniel
On Wed, Apr 14, 2021 at 04:19:29PM +0100, Daniel P. Berrangé wrote:
One example approach to source-git I've used...
Rather than having source-git branch names matching dist-git, use a different naming convention that is based off the upstream version primarily.
eg if upstream has v1.0 and v1.2 tags, I might have a 'v1.0-f33' branch, and if I rebase Fedora to v1.2, then I'd just switch to using a v1.2-f33 branch instead. The v1.0-f33 history remains intact forever, no force push required to rebase to new version.
As a concrete example, this is how the Fedora OCaml repo works (with a different naming convention):
https://pagure.io/fedora-ocaml/branches?branchname=master
Rich.
On 14.04.2021 17:19, Daniel P. Berrangé wrote:
That's not my experiance. The cases where I know of maintainers are using a source-git model with Fedora / RHEL already, are doing so precisely because it makes ongoing maint and rebasing waaaaay easier than with dist-git, especially when there are alot of downstream patches (100's or even 1000's).
In some cases, yes.
I woudn't expect Fedora to track the git-master in most cases. You generally still want Fedora to be base off releases, so you'd want to track starting fron a release tag or branch.
One more question. If the upstream ignores tags, can I create tags myself in source-git?
There are several ways you can do source git and they don't all imply force pushes, so I think this is probably inventing a problem where none yet exists.
You need force pushes support in order to perform git rebases.
eg if upstream has v1.0 and v1.2 tags, I might have a 'v1.0-f33' branch, and if I rebase Fedora to v1.2, then I'd just switch to using a v1.2-f33 branch instead. The v1.0-f33 history remains intact forever, no force push required to rebase to new version.
Looks like a dirty hack. Another pain for the maintainers.
On Thu, Apr 15, 2021 at 11:52:40AM +0200, Vitaly Zaitsev via devel wrote:
On 14.04.2021 17:19, Daniel P. Berrangé wrote:
That's not my experiance. The cases where I know of maintainers are using a source-git model with Fedora / RHEL already, are doing so precisely because it makes ongoing maint and rebasing waaaaay easier than with dist-git, especially when there are alot of downstream patches (100's or even 1000's).
In some cases, yes.
I woudn't expect Fedora to track the git-master in most cases. You generally still want Fedora to be base off releases, so you'd want to track starting fron a release tag or branch.
One more question. If the upstream ignores tags, can I create tags myself in source-git?
Of course. Downstream source-git can have any additional git tags it wants. It would be wise to pick a naming scheme for the downstream tags that is unlikely to clash with potential future upstream tags.
There are several ways you can do source git and they don't all imply force pushes, so I think this is probably inventing a problem where none yet exists.
You need force pushes support in order to perform git rebases.
Err, I illustrated an example showing this is not the case, that you've quoted right here:
eg if upstream has v1.0 and v1.2 tags, I might have a 'v1.0-f33' branch, and if I rebase Fedora to v1.2, then I'd just switch to using a v1.2-f33 branch instead. The v1.0-f33 history remains intact forever, no force push required to rebase to new version.
Looks like a dirty hack. Another pain for the maintainers.
It isn't a dirty hack. This is a normal model for maintaining maint branches based off releases.
Regards, Daniel
Daniel, thank you for sharing your experience!
More comments inline.
On Wed, Apr 14, 2021 at 5:26 PM Daniel P. Berrangé berrange@redhat.com wrote:
On Wed, Apr 14, 2021 at 04:53:06PM +0200, Vitaly Zaitsev via devel wrote:
On 14.04.2021 16:27, Tomas Tomecek wrote:
Could you, please, be more constructive and say what the actual problems are for you with such repositories?
- Some upstream repositories (Qt, Chromium, Linux kernel) are very huge
(more than 100 GiB). I don't want to download them from upstream and then upload to Fedora.
Fedora kernel is already being maintained like this:
https://gitlab.com/cki-project/kernel-ark
- Keeping such huge repositories will take up a lot of disk space on the
maintainer's computers.
Bear in mind that many maintainers already use a source-git workflow with Fedora, and are also involved in the upstream project. IOW, they already have these upstream repos present locally, and also be hosting it in some remote git server outside normal Fedora dist-git.
source-git isn't proposed to be forced on all packages, and drive-by contributors can also take a shallow clone instead of fetching all history. Any possible "fedpkg" integration could potentially default to a shallow clone.
Agreed, dist-git isn't going anywhere and with the source-git workflow, we still want to preserve the rel-eng/proven-packager workflow in dist-git. The source-git repos should make it way more convenient for maintainers to track downstream patches, as you describe.
- Rebase problem. Maintainers will need do a manual rebase on every
upstream release/commit. Rebasing to the next major version will be a serious problem for the projects with a lot of of downstream patches.
That's not my experiance. The cases where I know of maintainers are using a source-git model with Fedora / RHEL already, are doing so precisely because it makes ongoing maint and rebasing waaaaay easier than with dist-git, especially when there are alot of downstream patches (100's or even 1000's).
+1, `git pull --rebase upstream $ref` is more convenient than trying to fix downstream patches in whatever obscure way.
Shout out to rebase-helper which is trying to automate the patch files rebase problem via a CLI utility: https://github.com/rebase-helper/rebase-helper
- Some project have a weird git workflow between minor releases. I know at
least one project that uses Git tags with cherry-picked and manually backported commits. Such detached tags cannot be merged into master branch without resolving merge conflicts.
I woudn't expect Fedora to track the git-master in most cases. You generally still want Fedora to be base off releases, so you'd want to track starting fron a release tag or branch.
Correct, the branches are meant to use upstream release git-tags as a base.
- Force pushes must be enabled, which is too dangerous IMO.
There are several ways you can do source git and they don't all imply force pushes, so I think this is probably inventing a problem where none yet exists.
One example approach to source-git I've used...
Rather than having source-git branch names matching dist-git, use a different naming convention that is based off the upstream version primarily.
eg if upstream has v1.0 and v1.2 tags, I might have a 'v1.0-f33' branch, and if I rebase Fedora to v1.2, then I'd just switch to using a v1.2-f33 branch instead. The v1.0-f33 history remains intact forever, no force push required to rebase to new version.
If upstream has stable branches v1.0-maint and v1.2-maint, then v1.0-f33 branch might be initialized with v1.0-maint content. If upstream adds more commits to v1.0-maint branch, then you wouldn't force push v1.0-f33, you'd just do a git merge to pull them in.
That's interesting: merge commits make it hard to work with the git-history - how do you solve that? Daniel, does your tooling account for a merge commit and is it able to still pick up commits and turn them into patch files correctly?
Our current solution to rebases is to create new branches for every upstream release.
I understand that upstream repositories can have a long history - we could optimize and have shallow copies or only fetch recent upstream history if needed. Also, one would ideally only clone once and kept fetching new changes.
Do you want to force switch all Fedora packages to a new workflow?
There's no mention of forcing everyone to switch to source-git. Some upstreams don't even use git.
100%, the workflow is optional, dist-git is staying and both workflows should be compatible (to some extent, synchronizing downstream patches b/w the two would be tricky to figure out)
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Thu, Apr 15, 2021 at 04:30:26PM +0200, Tomas Tomecek wrote:
Daniel, thank you for sharing your experience!
More comments inline.
On Wed, Apr 14, 2021 at 5:26 PM Daniel P. Berrangé berrange@redhat.com wrote> > > 4. Some project have a weird git workflow between minor releases. I know at
least one project that uses Git tags with cherry-picked and manually backported commits. Such detached tags cannot be merged into master branch without resolving merge conflicts.
I woudn't expect Fedora to track the git-master in most cases. You generally still want Fedora to be base off releases, so you'd want to track starting fron a release tag or branch.
Correct, the branches are meant to use upstream release git-tags as a base.
- Force pushes must be enabled, which is too dangerous IMO.
There are several ways you can do source git and they don't all imply force pushes, so I think this is probably inventing a problem where none yet exists.
One example approach to source-git I've used...
Rather than having source-git branch names matching dist-git, use a different naming convention that is based off the upstream version primarily.
eg if upstream has v1.0 and v1.2 tags, I might have a 'v1.0-f33' branch, and if I rebase Fedora to v1.2, then I'd just switch to using a v1.2-f33 branch instead. The v1.0-f33 history remains intact forever, no force push required to rebase to new version.
If upstream has stable branches v1.0-maint and v1.2-maint, then v1.0-f33 branch might be initialized with v1.0-maint content. If upstream adds more commits to v1.0-maint branch, then you wouldn't force push v1.0-f33, you'd just do a git merge to pull them in.
That's interesting: merge commits make it hard to work with the git-history - how do you solve that? Daniel, does your tooling account for a merge commit and is it able to still pick up commits and turn them into patch files correctly?
I've not actually hit situations where I've needed merge commits in general, except by my own mistake. My tooling merely consists of running "git format-patch vN.M..." where "vN.M" is the tag associated with the release the RPM is based off.
Our current solution to rebases is to create new branches for every upstream release.
Yes, that's the good way IMHO. I was only mentioning merge commits in the context of tracking changes within a release stream. eg some projects may have stable branches in their upstream git, but don't do releases off them very frequently. So you'd conceivably want to periodically do a merge against upstream stable branche.
Regards, Daniel
On Wed, Apr 14, 2021 at 04:53:06PM +0200, Vitaly Zaitsev via devel wrote:
On 14.04.2021 16:27, Tomas Tomecek wrote:
Could you, please, be more constructive and say what the actual problems are for you with such repositories?
- Some upstream repositories (Qt, Chromium, Linux kernel) are very huge
(more than 100 GiB). I don't want to download them from upstream and then upload to Fedora.
Which one exactly is more than 100 GiB? The Linux kernel certainly isn't (more like 2.5 GiB, last I checked.)
--b.
On Thu, Apr 15, 2021 at 5:05 PM J. Bruce Fields bfields@redhat.com wrote:
On Wed, Apr 14, 2021 at 04:53:06PM +0200, Vitaly Zaitsev via devel wrote:
On 14.04.2021 16:27, Tomas Tomecek wrote:
Could you, please, be more constructive and say what the actual problems are for you with such repositories?
- Some upstream repositories (Qt, Chromium, Linux kernel) are very huge
(more than 100 GiB). I don't want to download them from upstream and then upload to Fedora.
Which one exactly is more than 100 GiB? The Linux kernel certainly isn't (more like 2.5 GiB, last I checked.)
Chromium is almost 100GB on disk. It's the only repository where I use Btrfs subvolumes and snapshots to roll back and forth through changes instead of using Git commands, because it takes *so* long and can time out and fail.
-- 真実はいつも一つ！/ Always, there's only one truth!
On Wed, Apr 14, 2021 at 10:45:23AM +0200, Tomas Tomecek wrote:
Please head to the SIG wiki page to learn more about our proposed MVP. We are looking for maintainers of Fedora Linux packages who'd be interested in being early adopters and give us feedback during the development process. You don't need to do any coding unless you want to :)
Yeah, I'm interested. I have a few packages I maintain that I think would benefit from this workflow.
On Wed, Apr 14, 2021 at 10:45:23AM +0200, Tomas Tomecek wrote:
Good morning, I'd like to announce the creation of Fedora Source-git SIG:
https://fedoraproject.org/wiki/SIGs/Source-git
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Please head to the SIG wiki page to learn more about our proposed MVP. We are looking for maintainers of Fedora Linux packages who'd be interested in being early adopters and give us feedback during the development process. You don't need to do any coding unless you want to :)
We might be interested in trialling it with some of the virt packages.
I'm wondering about the scope of this statement from the wiki page above:
"Whatever we produce here, it MUST NOT violate Fedora Packaging Guidelines (we should strive to change them if needed)."
I can certainly understand the intent behind this when dealing with legally restricted content. eg don't allow impls of patented algorithms that are blocked from dist-git.
In terms of scope I can reasonably audit the source for current git master or a specific git tag to ensure legal compliance.
I can't reasonably audit the entire source-git history of the project back to day 1 though, to make sure the git repo has never had legally restricted content at any point in the past 20 years of its life.
IOW, I'd hope that in terms of FPG compliance, we only need to consider the specific tag/branch that's being used to populate dist-git and can ignore history.
This could still potentially mean that source-git is a complication for the packages where we have to re-create the tarballs after removing patented crypto. Will legal allow it to remain in source git, but require it to be purged when src-git syncs to dist-git or something like that ?
Overall this does seem to imply though that if Fedora hosts src-git repos with upstream history itself, then it is potentially opening a new liability that it hasn't had before. If we're going to host src-git on a Fedora namespace on gitlab.com though, then its someone else's problem to worry about, and Fedora only needs to worry about what's synced to dist-git for the actual RPMs builds.
Aside from legal, I also wonder about things like binary blobs or bundled libraries. These are relatively common to see in upstream git repos, even if they don't make it into the release tarballs that Fedora traditionally consumes.
Hopefully the requirement to comply with Fedora Pakaging Guidelines will only apply to files in src-git that actually get used for Fedora builds, and not stuff that exists but is skipped/ignored ?
Regards, Daniel
comments inline
On Wed, Apr 14, 2021 at 4:09 PM Daniel P. Berrangé berrange@redhat.com wrote:
On Wed, Apr 14, 2021 at 10:45:23AM +0200, Tomas Tomecek wrote:
Good morning, I'd like to announce the creation of Fedora Source-git SIG:
https://fedoraproject.org/wiki/SIGs/Source-git
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Please head to the SIG wiki page to learn more about our proposed MVP. We are looking for maintainers of Fedora Linux packages who'd be interested in being early adopters and give us feedback during the development process. You don't need to do any coding unless you want to :)
We might be interested in trialling it with some of the virt packages.
I'm wondering about the scope of this statement from the wiki page above:
"Whatever we produce here, it MUST NOT violate Fedora Packaging Guidelines (we should strive to change them if needed)."
I can certainly understand the intent behind this when dealing with legally restricted content. eg don't allow impls of patented algorithms that are blocked from dist-git.
In terms of scope I can reasonably audit the source for current git master or a specific git tag to ensure legal compliance.
I can't reasonably audit the entire source-git history of the project back to day 1 though, to make sure the git repo has never had legally restricted content at any point in the past 20 years of its life.
IOW, I'd hope that in terms of FPG compliance, we only need to consider the specific tag/branch that's being used to populate dist-git and can ignore history.
This could still potentially mean that source-git is a complication for the packages where we have to re-create the tarballs after removing patented crypto. Will legal allow it to remain in source git, but require it to be purged when src-git syncs to dist-git or something like that ?
Overall this does seem to imply though that if Fedora hosts src-git repos with upstream history itself, then it is potentially opening a new liability that it hasn't had before. If we're going to host src-git on a Fedora namespace on gitlab.com though, then its someone else's problem to worry about, and Fedora only needs to worry about what's synced to dist-git for the actual RPMs builds.
Thank you, Daniel, very good points! I'll add this to the agenda of the first meeting to bring this to Fedora Legal. I also hope to make the legal aspect of this someone else's problem - on the other hand, if this workflow ever gets approved by FESCO and become official (aside from the dist-git workflow) the repos would need to be hosted on Fedora Infra.
Aside from legal, I also wonder about things like binary blobs or bundled libraries. These are relatively common to see in upstream git repos, even if they don't make it into the release tarballs that Fedora traditionally consumes.
Hopefully the requirement to comply with Fedora Pakaging Guidelines will only apply to files in src-git that actually get used for Fedora builds, and not stuff that exists but is skipped/ignored ?
Yes, I'd say to files which are being put to dist-git and used for official builds. Short term, I don't think it's feasible to create production koji builds out of these source-git repositories. That would be the long term plan :)
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 14.04.21 10:45, Tomas Tomecek wrote:
Good morning, I'd like to announce the creation of Fedora Source-git SIG: https://fedoraproject.org/wiki/SIGs/Source-git
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Just wondering: will there be some coordination with the Fedora kernel developers that are relying on a git based workflow since a few weeks now (for rawhide even longer)?
To those who don't know: all the stuff in dist-git kernel is generated from https://gitlab.com/cki-project/kernel-ark these days afaik, so it might be wise to make sure the solution you work out works for them as well. Especially as I find find some parts of how they do it a bit questionable. The main tarball they use as Source0 for example doesn't match the tarball downloadable from kernel.org(¹); all the patches are stashed into one(²); patches for fedora specific stuff (like the configs needed for building the kernel) are in the same branch as the patches(³). I think that makes things quite confusing, especially for outsiders. Sometimes I wonder if some of what the kernel people do violates the Fedora Packaging Guidelines(⁴), but the kernel-ark people ensured me it's fine.
CU, knurd
(¹) https://src.fedoraproject.org/rpms/kernel/blob/f33/f/sources for example states:
SHA512 (linux-5.11.14.tar.xz) = ccf0eaf6df0dacd2984c361621f67a3d16cf7a7174155ebdf646f1acfec43e19ff942e6c17e5bc3b5dc7a300c32bdc6ee37877162c099f5bd9924244f9445467
But: $ wget --quiet https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.11.14.tar.xz $ sha512sum linux-5.11.14.tar.xz 8dfc7ff184e5cb33fff74686071f1605f3a834669e201d272f3047aa00657339ec1a3cfd605d8761b8a0f335b8488c02c701e72ed30031856e9c154aa1ff2d88 linux-5.11.14.tar.xz
(²) https://src.fedoraproject.org/rpms/kernel/blob/f33/f/patch-5.11-redhat.patch FWIW, links to the individual patches can be found here: https://src.fedoraproject.org/rpms/kernel/blob/f33/f/Patchlist.changelog
(³) for example https://gitlab.com/cki-project/kernel-ark/-/commits/fedora-5.11
(⁴) like this part "sources used to build a package should be the vanilla sources available from upstream. To help reviewers and QA scripts verify this, the packager needs to indicate where a reviewer can find the source that was used to make the rpm". The quote is from here: https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/
On Wed, Apr 14, 2021 at 8:52 PM Thorsten Leemhuis fedora@leemhuis.info wrote:
On 14.04.21 10:45, Tomas Tomecek wrote:
Good morning, I'd like to announce the creation of Fedora Source-git SIG: https://fedoraproject.org/wiki/SIGs/Source-git
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Just wondering: will there be some coordination with the Fedora kernel developers that are relying on a git based workflow since a few weeks now (for rawhide even longer)?
To those who don't know: all the stuff in dist-git kernel is generated from https://gitlab.com/cki-project/kernel-ark these days afaik, so it might be wise to make sure the solution you work out works for them as well. Especially as I find find some parts of how they do it a bit questionable. The main tarball they use as Source0 for example doesn't match the tarball downloadable from kernel.org(¹); all the patches are stashed into one(²); patches for fedora specific stuff (like the configs needed for building the kernel) are in the same branch as the patches(³). I think that makes things quite confusing, especially for outsiders. Sometimes I wonder if some of what the kernel people do violates the Fedora Packaging Guidelines(⁴), but the kernel-ark people ensured me it's fine.
CU, knurd
(¹) https://src.fedoraproject.org/rpms/kernel/blob/f33/f/sources for example states:
SHA512 (linux-5.11.14.tar.xz) = ccf0eaf6df0dacd2984c361621f67a3d16cf7a7174155ebdf646f1acfec43e19ff942e6c17e5bc3b5dc7a300c32bdc6ee37877162c099f5bd9924244f9445467
But: $ wget --quiet https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.11.14.tar.xz $ sha512sum linux-5.11.14.tar.xz 8dfc7ff184e5cb33fff74686071f1605f3a834669e201d272f3047aa00657339ec1a3cfd605d8761b8a0f335b8488c02c701e72ed30031856e9c154aa1ff2d88 linux-5.11.14.tar.xz
(²) https://src.fedoraproject.org/rpms/kernel/blob/f33/f/patch-5.11-redhat.patch FWIW, links to the individual patches can be found here: https://src.fedoraproject.org/rpms/kernel/blob/f33/f/Patchlist.changelog
(³) for example https://gitlab.com/cki-project/kernel-ark/-/commits/fedora-5.11
(⁴) like this part "sources used to build a package should be the vanilla sources available from upstream. To help reviewers and QA scripts verify this, the packager needs to indicate where a reviewer can find the source that was used to make the rpm". The quote is from here: https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/
Thorsten, thank you for pointing those out. I spoke with Don Zickus, one of the leads behind the kernel-ark project (or probably the lead :) and he confirmed to join the SIG. My hope is to create a platform that would unify all the different source-git implementations being used right now into one, which would have clear documentation, tooling and infra support. Dreaming big :)
Tomas
On Wed, Apr 14, 2021 at 1:47 PM Thorsten Leemhuis fedora@leemhuis.info wrote:
On 14.04.21 10:45, Tomas Tomecek wrote:
Good morning, I'd like to announce the creation of Fedora Source-git SIG: https://fedoraproject.org/wiki/SIGs/Source-git
Our main goal in the SIG right now is to establish a development workflow for Fedora Linux packages using repositories with sources and upstream history (this is what we call source-git), instead of just distribution files with links to tarballs (dist-git).
Just wondering: will there be some coordination with the Fedora kernel developers that are relying on a git based workflow since a few weeks now (for rawhide even longer)?
To those who don't know: all the stuff in dist-git kernel is generated from https://gitlab.com/cki-project/kernel-ark these days afaik, so it might be wise to make sure the solution you work out works for them as well. Especially as I find find some parts of how they do it a bit questionable. The main tarball they use as Source0 for example doesn't match the tarball downloadable from kernel.org(¹); all the patches are stashed into one(²); patches for fedora specific stuff (like the configs needed for building the kernel) are in the same branch as the patches(³). I think that makes things quite confusing, especially for outsiders. Sometimes I wonder if some of what the kernel people do violates the Fedora Packaging Guidelines(⁴), but the kernel-ark people ensured me it's fine.
CU, knurd
(¹) https://src.fedoraproject.org/rpms/kernel/blob/f33/f/sources for example states:
SHA512 (linux-5.11.14.tar.xz) = ccf0eaf6df0dacd2984c361621f67a3d16cf7a7174155ebdf646f1acfec43e19ff942e6c17e5bc3b5dc7a300c32bdc6ee37877162c099f5bd9924244f9445467
But: $ wget --quiet https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.11.14.tar.xz $ sha512sum linux-5.11.14.tar.xz 8dfc7ff184e5cb33fff74686071f1605f3a834669e201d272f3047aa00657339ec1a3cfd605d8761b8a0f335b8488c02c701e72ed30031856e9c154aa1ff2d88 linux-5.11.14.tar.xz
likely different compression options used by default, the upstream tarball is just a bit smaller. We create the tarballourselves, but based on greg's tag. If you explode and compare them, they are identical:
fedora64:[~/srcchk]>diff -ruNp stable-5.11.14/ ark-5.11.14/ fedora64:[~/srcchk]>ls -l *5.11.14.tar.xz -rw-rw-r--. 1 jforbes jforbes 123313416 Apr 15 12:38 ark-5.11.14.tar.xz -rw-r--r--. 1 jforbes jforbes 117651976 Apr 15 12:42 linux-5.11.14.tar.xz
For what it's worth, until they changed compression, we used to recompress the upstream tarball and end up with a similar result. As to your (*) they are tagged in the git tree, you can verify that tag in our clone of the upstream tree, or directly in the upstream tree if you wish, both audit sources are available.
As to the single patch vs the broken out, it is a matter of git merge vs git rebase. If we did a rebase with every release, they could be broken out cleanly, but the trade off is any update becomes a forced update, and any pending MR is invalidated. We do rebase the patches on the creation of stable branches, so fedora-5.11 was created by rebasing the rawhide branch over Greg's linux-5.11.y branch. It is merged forward from there, but none of the patches should be too different to quickly figure out.
(²) https://src.fedoraproject.org/rpms/kernel/blob/f33/f/patch-5.11-redhat.patch FWIW, links to the individual patches can be found here: https://src.fedoraproject.org/rpms/kernel/blob/f33/f/Patchlist.changelog
(³) for example https://gitlab.com/cki-project/kernel-ark/-/commits/fedora-5.11
(⁴) like this part "sources used to build a package should be the vanilla sources available from upstream. To help reviewers and QA scripts verify this, the packager needs to indicate where a reviewer can find the source that was used to make the rpm". The quote is from here: https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/ _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Wed, 2021-04-14 at 10:45 +0200, Tomas Tomecek wrote: ...
We are looking for maintainers of Fedora Linux packages who'd be interested in being early adopters and give us feedback during the development process. You don't need to do any coding unless you want to :)
I'd like to register interest in this topic on behalf of myself and Facebook. I'm the maintainer for one (very recently added) package. The source-git workflow has appeal, especially in context of other SIGs - namely CentOS Hyperscale which I'm part of. Looking forward to the meeting!
Matthew.
-
Daniel P. Berrangé -
J. Bruce Fields -
Justin Forbes -
Matthew Almond -
Matthew Miller -
Neal Gompa -
Richard W.M. Jones -
Thorsten Leemhuis -
Tomas Tomecek -
Vitaly Zaitsev