Fedora kernel git tree and package maintenance

[Josh Boyer]

On Thu, Jul 9, 2015 at 10:17 AM, Don Zickus <dzickus at redhat.com> wrote:
> On Thu, Jul 09, 2015 at 07:42:08AM -0400, Josh Boyer wrote:
>> On Thu, Jun 25, 2015 at 12:02 PM, Josh Boyer <jwboyer at fedoraproject.org> wrote:
>> >> Will the exploded tree go away if we don't change to this model?  Also
>> >> unclear.  I'd hesitantly say it would stick around, but it might
>> >> change in the way it is created and published.
>> >>
>> >> I'm sure there will be more questions, so please ask away.  Looking
>> >> forward to hearing other's thoughts.
>> >
>> > So.  Given that it has been a week and we've heard nothing back, I'm
>> > going to assume that either nobody cares how we maintain our tree
>> > (which tbh is a fair point of view), or all of this sounds great to
>> > everyone.
>> >
>> > So here is our plan going forward.  We need a place to try this out to
>> > make sure it is actually a net positive for us.  We intend to drive
>> > the master branch in pkg-git (rawhide) using the tree and scripts for
>> > the 4.2 merge window.  What that means for other people is basically
>> > this:
>>
>> The team and I were discussing this topic yesterday, and it really
>> isn't working as we'd hoped.  The exploded tree is great for rebasing
>> patches.  It's trivial and integrated into the workflow just fine.
>> However, taking that source code and producing an RPM is pretty
>> cumbersome now.  The scripts work, but they can be fragile in certain
>> places.  It's also still double commits in the long run, one in
>> pkg-git and another in the exploded tree for all the junk that is
>> needed for tracking and updating pkg-git.  Over time, a 'git rebase'
>> call will take forever as those commits pile up because they're never
>> going upstream.  Thorsten and Peter also pointed out that changes
>> "sneak" into pkg-git and only are mentioned in the RPM changelog, not
>> as separate commits.
>
> Hi Josh,
>
> Thanks for the effort!  Sad to hear it isn't working out, but I understand
> your reasons.
>
> While it is still fresh in your mind, can I ask a few questions for
> curiousity.
>
> Not that submodules is a good solution (from what I hear) but would a
> solution like that (if made easier) work for those fedora scripts?  The idea
> would be a rebase and a single commit on top to add the submodule-like
> directory/area that pulls in a separate tree full of those changes?
>
> This would sidestep the rebase growing larger over time, sort of
> (one initial commit, but a growing set of commits brought in later??).

I thought about this when I originally started, but avoided it for a
few different reasons.  The main one was my unfamiliarity with
submodules.  Another was now you're going from 2 "trees" to manage
(pkg-git, linux.git) to 3 (pkg-git, linux.git, submodule).  It would
help the rebase issue, but that is about all it would help.

> It might address the 'sneaky' changes problem too.

I'm not sure about that one.  The issue there stems from multiple
things happening in linux.git, and then when we sync to pkg-git it all
just gets sucked into one large commit.  Unless you held to a strict
1:1 linux.git -> pkg-git rule, you're always going to face that
problem.  And if you go to 1:1, you aren't really treating linux.git
as the canonical source at that point.  It is just an exploded tree
mirror essentially.

In some scenarios, this isn't a big deal.  Some distros don't expose
their package SCM and their users/customers are used to the SRPM as
being the canonical source, with %changelog being the equivalent of
the SCM commit logs.  However, since the Core+Extras merge Fedora has
always used the package SCM as the transparent location to see "what
changed and why".  Deviating from that, while it makes things easier
for my team, is somewhat against Fedora's transparency.

(It's also the reason why I bothered to spit out separate patches and
tarballs.  The easiest solution is to just suck up the output of
git-archive of the linux.git tree and use that as a tarball and build
it, but not having patches would probably cause people to lose their
minds in the Fedora world.)

> The only thing it doesn't handle is the extra Fedora kernel patches
> themselves that is carried (but I think you have that minimized and under
> control anyway).
>
> I don't expect you to hold your breathe for that solution to arise, but if
> it did, would it help do you think?

I think the only thing that would really make working from an exploded
tree viable is if koji could be pointed at it directly.  Even then,
the mechanics of source -> spec -> srpm -> rpm(s) isn't
straightforward.

josh