One of the things on my longer-term todo list has been to get our canonical git repos out of bitbucket and on to something running free software (preferably something that we or infra controls). Now that we're getting closer to migrating off of our current phabricator instance, I think it's time to consider making this move for at least the repos that are currently hosted on bitbucket.
As I see it, we have two obvious choices: using phabricator or using pagure [1]. They both have advantages and disadvantages (that I've listed at the end of this email) but both are 100% open source and hosted by Fedora infra.
Thoughts?
Tim
Hosting git repos in phabricator --------------------------------
Phabricator has been capable of hosting git repos for a while now, the holdup has been me getting everything ready to migrate on to a new setup that's capable of doing it
* we have more control over the repo hosting * we have more responsibility for making sure things stay up, are recovered in the event of a system failure etc. * would likely be easier to use some of phab's automation features if it controls the repos * ACLs for repos would be controlled in one place * would require users to upload ssh keys - I don't think we can (not even sure about should, if it's possible) get ssh pubkey data from fas @ account creation time * any extra load from clients cloning task repos would be on our infra and not causing issues for others
A note that could be worth making is the permissions/policy structure on phabricator. It can be configured to allow users to create repos but in order to make that work well enough to justify doing, we'd need to allow all users to edit all repos as well - I don't see a more granular way of making this happen ATM. It's not something that isn't fixable long term if we're not happy with the "request repo for XXX from admins" process but figured I would mention it.
pagure ------
pagure is a project started in infra. pingou is the main dev and it's gained quite a few new features lately. Despite the non-fp.o domain, it is infra hosted.
* We don't have to worry about hosting our own repos * doing similar things to other groups - I don't think there's any interest by any other fedora groups in using phabricator unless something goes horribly wrong with pagure (I also suspect the odds of that happening are pretty low). * would require users to upload ssh keys but that's once for all the things that are (and eventually will be) hosted in pagure * we'd have to figure out how to handle the eventual pull requests that are out of process for us - the same thing kinda exists with bitbucket but I suspect it'll get more common as things progress.
One of the things on my longer-term todo list has been to get our canonical git repos out of bitbucket and on to something running free software (preferably something that we or infra controls). Now that we're getting closer to migrating off of our current phabricator instance, I think it's time to consider making this move for at least the repos that are currently hosted on bitbucket.
Do we have some repos elsewhere?
As I see it, we have two obvious choices: using phabricator or using pagure [1]. They both have advantages and disadvantages (that I've listed at the end of this email) but both are 100% open source and hosted by Fedora infra.
Thanks for compiling the lists. Some comments below.
Thoughts?
Tim
Hosting git repos in phabricator
Phabricator has been capable of hosting git repos for a while now, the holdup has been me getting everything ready to migrate on to a new setup that's capable of doing it
- we have more control over the repo hosting
- we have more responsibility for making sure things stay up, are recovered in the event of a system failure etc.
- would likely be easier to use some of phab's automation features if it controls the repos
I'm having troubles imagining what this could include. Do you have some concrete ideas of such automation feature?
- ACLs for repos would be controlled in one place
I imagine that Pagure controls access with FAS groups (or doesn't it?). With Phabricator, would it work similarly how projects are organized - the repo owner would define in repo properties the list of phab logins that can commit to that repo? Would it be possible to rely on FAS instead?
- would require users to upload ssh keys - I don't think we can (not even sure about should, if it's possible) get ssh pubkey data from fas @ account creation time
- any extra load from clients cloning task repos would be on our infra and not causing issues for others
I thought clients pull tasks from the buildmaster (grok mirror)?
A note that could be worth making is the permissions/policy structure on phabricator. It can be configured to allow users to create repos but in order to make that work well enough to justify doing, we'd need to allow all users to edit all repos as well - I don't see a more granular way of making this happen ATM. It's not something that isn't fixable long term if we're not happy with the "request repo for XXX from admins" process but figured I would mention it.
I think this is somewhat related to our perceived direction of Phabricator. If we consider our Phab instance to be just related to QA Devel tasks, "please create me a repo" approach is not such a big deal, because we're covered in most timezones and enough of us have admin access. And we don't need new repos that often anyway. But if we want to allow anybody to create any repo in Phab, that would of course be a bigger administration pain - more requests, documentation, etc. But if it should come to that, I'd rather see Phab hosted by Fedora Infra as an official service, than us doing this work for the whole Fedora project. Our main job is to create helpful QA tools, not to administer public services. (That of course doesn't mean we can't help with it.)
pagure
pagure is a project started in infra. pingou is the main dev and it's gained quite a few new features lately. Despite the non-fp.o domain, it is infra hosted.
- We don't have to worry about hosting our own repos
This is what I consider the major benefit of Pagure - decreasing the maintenance work for us.
- doing similar things to other groups - I don't think there's any interest by any other fedora groups in using phabricator unless something goes horribly wrong with pagure (I also suspect the odds of that happening are pretty low).
- would require users to upload ssh keys but that's once for all the things that are (and eventually will be) hosted in pagure
- we'd have to figure out how to handle the eventual pull requests that are out of process for us - the same thing kinda exists with bitbucket but I suspect it'll get more common as things progress.
I imagine we could replace that Pull Request tab with documentation how to send a pull request into Phab. It is possible to create the diff without installing arcanist and other tools, it's just a matter of showing people how. And all you need is a FAS account, which is also needed in Pagure. So I think if we provide good description of the steps, people should not be discouraged by it. What do you think?
On Mon, 2015-06-29 at 08:24 -0400, Kamil Paral wrote:
One of the things on my longer-term todo list has been to get our canonical git repos out of bitbucket and on to something running free software (preferably something that we or infra controls). Now that we're getting closer to migrating off of our current phabricator instance, I think it's time to consider making this move for at least the repos that are currently hosted on bitbucket.
Do we have some repos elsewhere?
The projects I started still live on happyassassin.net . I'm not opposed to moving them into phab or pagure.
On Mon, 29 Jun 2015 08:24:02 -0400 (EDT) Kamil Paral kparal@redhat.com wrote:
<snip>
Hosting git repos in phabricator
Phabricator has been capable of hosting git repos for a while now, the holdup has been me getting everything ready to migrate on to a new setup that's capable of doing it
- we have more control over the repo hosting
- we have more responsibility for making sure things stay up, are recovered in the event of a system failure etc.
- would likely be easier to use some of phab's automation features
if it controls the repos
I'm having troubles imagining what this could include. Do you have some concrete ideas of such automation feature?
These bits are still marked as a beta application and I'm not sure I understand what they're going to do eventually so this is probably not worth considering.
- ACLs for repos would be controlled in one place
I imagine that Pagure controls access with FAS groups (or doesn't it?). With Phabricator, would it work similarly how projects are organized - the repo owner would define in repo properties the list of phab logins that can commit to that repo? Would it be possible to rely on FAS instead?
Not practically, no. If someone wrote the code to interface phabricator with FAS it might be able to happen but AFAIK, phabricator relies on it's own groups/acls for object access.
- would require users to upload ssh keys - I don't think we can
(not even sure about should, if it's possible) get ssh pubkey data from fas @ account creation time
- any extra load from clients cloning task repos would be on our
infra and not causing issues for others
I thought clients pull tasks from the buildmaster (grok mirror)?
They do. I was wondering if we would need that extra layer of complexity if we were hosting our own repos, though.
A note that could be worth making is the permissions/policy structure on phabricator. It can be configured to allow users to create repos but in order to make that work well enough to justify doing, we'd need to allow all users to edit all repos as well - I don't see a more granular way of making this happen ATM. It's not something that isn't fixable long term if we're not happy with the "request repo for XXX from admins" process but figured I would mention it.
I think this is somewhat related to our perceived direction of Phabricator. If we consider our Phab instance to be just related to QA Devel tasks, "please create me a repo" approach is not such a big deal, because we're covered in most timezones and enough of us have admin access. And we don't need new repos that often anyway. But if we want to allow anybody to create any repo in Phab, that would of course be a bigger administration pain - more requests, documentation, etc. But if it should come to that, I'd rather see Phab hosted by Fedora Infra as an official service, than us doing this work for the whole Fedora project. Our main job is to create helpful QA tools, not to administer public services. (That of course doesn't mean we can't help with it.)
The only counter to this is if we wanted to host more task repos in phabricator, especially if they aren't bits that we're responsible for.
I'm of the mind that stuff that we're not directly maintaining should be hosted elsewhere so it may not really matter if only a few of us is able to create new hosted repos.
pagure
pagure is a project started in infra. pingou is the main dev and it's gained quite a few new features lately. Despite the non-fp.o domain, it is infra hosted.
- We don't have to worry about hosting our own repos
This is what I consider the major benefit of Pagure - decreasing the maintenance work for us.
I'm not sure how much it'd reduce the maintenance, to be honest. If we continue to use phabricator for everything but repo hosting, we still have to triage issues, build packages etc.
- doing similar things to other groups - I don't think there's any interest by any other fedora groups in using phabricator unless something goes horribly wrong with pagure (I also suspect the
odds of that happening are pretty low).
- would require users to upload ssh keys but that's once for all
the things that are (and eventually will be) hosted in pagure
- we'd have to figure out how to handle the eventual pull requests that are out of process for us - the same thing kinda exists with bitbucket but I suspect it'll get more common as things progress.
I imagine we could replace that Pull Request tab with documentation how to send a pull request into Phab. It is possible to create the diff without installing arcanist and other tools, it's just a matter of showing people how. And all you need is a FAS account, which is also needed in Pagure. So I think if we provide good description of the steps, people should not be discouraged by it. What do you think?
If you mean replace the pull request tab for the projects that we would have in Pagure, I'm not sure it would be so simple. I haven't looked at the code, so I could be wrong but if we hosted our code in pagure, we'd have to deal with forking and pull requests.
Not that this is a bad thing by definition. We've gotten some fixes via pull request (some of which are languishing in "accepted and forgotten land") and from what I can tell, folks outside of the core people aren't all that keen on learning how to submit stuff using phab. This gets us back to the question of whether or not we want to worry about the drive-by patching that we'd get with a more github-ish workflow.
I'm kinda leaning towards moving the repo hosting to phabricator and if we want to enable other workflows, we can set up a mirror on github/bitbucket/pagure.
Tim
qa-devel@lists.fedoraproject.org
-
Adam Williamson -
Kamil Paral -
Tim Flink