RFR: GIT Package VCS

[Jeremy Katz]

On Wed, 2007-06-06 at 22:46 -0400, Christopher Blizzard wrote:
> On Wed, 2007-06-06 at 17:31 -0400, Jeremy Katz wrote:
> > At the same time, I think we still need to be able to very clearly
> > separate out our changes from what upstream has.  Just a git repo of the
> > kernel very quickly gets out of control and you end up with bazillions
> > of things that you never push back upstream because it's easier to just
> > keep sitting on them.  So I don't think that just a VCS repo of the
> > source is what we want... we're going to end up wanting some integration
> > with something quilt-like to get patches out; so like stgit or mq or ...
> 
> Like I said, I think it depends on what the package is and most
> importantly what the maintainer and developers want to do.  I know that
> Dave Jones didn't want to use git to build the kernel, and that's fine
> He doesn't have to use it.
> 
> One thing I'm trying to do here is break the maintainer model that we
> have today.  I want developers to come and work directly in Fedora.  And
> that means taking out that extra step if we can - going directly from a
> VCS to a package.  Using HAL as an example - want to pull in the latest
> one?  Just point your repo at the upstream one and catch up to that
> release.  Then click to build.  Or, even better, the developer can do it
> himself and push it into our release.  Remember, in a lot of causes for
> us the maintainer _is_ the main developer!  So why not draw that line as
> close as possible?

Because in many more cases, the maintainer _isn't_ the main upstream
developer.  And in a lot of cases, the main upstream developer (or
developers) are very happy to have someone else take care of things that
aren't directly "write new feature, fix bugs in my code, make my program
kick ass".  eg, they're really happy when someone volunteers to take
over maintaining their project's website for them.  Similarly with
package maintenance.  Especially because we're not going to get to a
world where there's only one Linux distribution[1] and they're certainly
not going to want to maintain packages for many distros.

So I guess that's the point where our basic disagreement lies...  I have
no problems if upstream developers _want_ to maintain things.  But I
don't know that it's the case to optimize for.

> > > This might sound crazy (SUPPORT > 1 SYSTEM, ARE YOU CRAZY?)  Well, yes,
> > > until you realize what you need to do here.  To start with you only have
> > > to teach the rpm build side how pull a specific tag from a specific
> > > repo.  On the query side we need a browser for each kind, which is a bit
> > > of work, but something I think we need to do anyway.  (i.e. "What would
> > > git do?")
> > 
> > So if I am the owner of the rpm package which has an upstream of hg and
> > want to fix, test and commit a change to say (for the sake of argument)
> > neon which is in git, I now have to know two different systems?  You're
> > crazy ;-)  
> 
> No.  If you happen to be a maintainer _and_ a developer and you have
> _chosen_ to use hg or git or whatever, then we make it easy for you.
> This is about adding options to bring us closer to the upstream
> developers and make their lives easier.

But we're making the lives of the upstream developers (who, more often
than not, are more experienced and going to be better able to adapt to a
different VCS and some workflow on top of it) easier at the cost of our
contributors who maintain a lot of packages and _aren't_ the upstream
developer.

> > To add to the craziness of this path, think about actually maintaining
> > these packages across different distro releases...  every VCS has its
> > own unique and specially crack-ridden way of handling branching.  
> 
> Yes, but you only need to teach our systems about that once.

And add every time a new upstream cares about the new VCS of the week.
Or when $VCS changes it's branching.  And maintain and test across all
of the permutations.  

> > Or when you star to think about the "I'm a downstream of Fedora and need
> > to change X, Y, and Z" and you are then having them set up potentially 3
> > different VCS systems to do so.  
> 
> Depends on what they want to do.

I want to enable them to make any changes that they want.  That would be
one of the big points of a DVCS and also is one of the big points in
making it easy for anyone to set up a build farm with koji.  I want them
to be able to track what goes on, adjust their changes accordingly and
go on their way.  I don't want the setup to involve doing the server set
up of each of git, hg, bzr, svk, arch and monotone.  

> > Or the "it's time for a mass-rebuild; let's go and commit a version bump
> > to all the packages so we can rebuild.  Uhhm.  Uh-oh.
> 
> This is actually easier for those VCS packages than it is today.  Right
> now we have to go in and edit every single spec file, edit, and commit.
> If we back away from having spec files like that and instead generate
> that info before compile time (so a "pristine source" is just a set of
> metadata, rules and a source tag) doing mass rebuilds is _easy_.

So where's the metadata/rules kept?  Do I have to use two different
systems -- one for the source and one for the metadata/rules?  Because
I'm always going to want to be able to deal with both.  If they're not
in different systems, then the rebuild case still has to deal with many
systems  

And fundamentally, I think that the "pristine source + patches" bit is
just as important today as it was 10 years ago.  Because otherwise, you
get into a situation where you basically encourage forking and not
contributing things back upstream.  Your response is going to be "but
it's upstream doing it" -- at the same time, there are _always_ going to
be distro specific changes that won't necessarily make sense for
upstream.  Being able to contain and track those and making it easier
for changes to be pushed upstream as opposed to carried along in a
distro-specific world forever is a good thing.

> > > Plus, to be honest, it completely avoids the whole "which damn system do
> > > we use."  And I like focusing on the end user features instead of
> > > getting stuck in VCS dicussion hell.  We're not going to get everyone
> > > else to agree or even use the same system.  So let's build something
> > > that supports both.
> > 
> > So instead of picking _one_ answer, we now have to make sure that we
> > implement all of the end user features for N systems?  Seriously, this
> > is losing.
>
> So let me ask you this question: where do you want Fedora to go?  Just
> keep adding packages?  Situation as usual?  We got things outside of the
> firewall.  That's nice.  We will never rest on our laurels.  What's
> next?

Continuing to add packages is going to be important forever.  Because
the great thing is there's always more software :)  

> I have a pretty specific vision for where we should be two to three
> years down the road, and it involves innovating in these spaces
> including looking at having source repos to fix real problems with
> developer productivity.  How can we make developers (not just
> maintainers) lives easier?  How can we shorten the distance between
> them?  

I want to make it easier for users who are interested in something to
become involved in it within the context of Fedora.  I see a neat
application on gnomefiles and it's not in Fedora?  How do we make it
easier for that user to go from being just a user to being a
contributor?  And then, once that user is a maintainer it becomes a much
easier path for them to get involved as a contributor in upstream
projects.  Because they've had a chance to sort of hone their skills and
start off with something that's a little easier.

For all of the maintainers we have today, how do we make their lives
easier?  And one part of that is how do we make it easier for them to
interact with the upstream of the software they maintain for Fedora.
But another part is how do we make it easier for them to interact with
other parts of Fedora that their packages depend on.  Sure, some bugs
are easily traced and just in the piece of software that you're the
maintainer of.  But plenty of bugs are caused by a problem in another
library or something else -- how do we make it easier for them to be
able to track that down and fix it themselves and then quite possibly
get involved with another project.

But I think things are devolving a bit from the actual infrastructure
discussion at this point... reply-to set appropriately

Jeremy

[1] The point at which there's one Linux distro is the point at which I
start Jinux ;)  The variety and competition within the Linux distro
space is one of the things that continues to drive innovation and
progress and I think that it's an incredibly important force