I created
https://fedoraproject.org/wiki/Taskotron
with some basic info and links, especially the quick start guide (kudos to Tim, this will surely be useful for many of us now).
I also updated
https://fedoraproject.org/wiki/QA/Tools
with the list of our current projects. If you see something missing, please add it. Thanks.
I also updated
https://fedoraproject.org/wiki/QA/Tools
with the list of our current projects. If you see something missing, please add it. Thanks.
Josef, I find this quite confusing:
https://bitbucket.org/rajcze/resultsdb https://bitbucket.org/rajcze/resultsdb_api https://bitbucket.org/rajcze/resultsdb_frontend https://fedorahosted.org/ResultsDB/ https://git.fedorahosted.org/git/ResultsDB.git
What is the canonical source? Where should people report issues? Please, pick one location to keep the project in (it seems we're going bitbucket, or bitbucket+phab way) and kill the other site. Also make sure issues can't be reported on two different places, and forward people to the single one. And make sure your wiki page points to a correct location: https://fedoraproject.org/wiki/ResultsDB (I updated it before learning there are other sources of ResultsDB. You might have some more under your sleeve.)
Thanks.
As a general note, I'm not fully happy when the source code lives somewhere else than the issues do. It confuses people. But if we want to keep easy-to-browse-and-fork functionality (bitbucket) and full-featured-review functionality (phab), it seems we don't have much choice. At least we should always disable the issue support on bitbucket for every project.
----- Original Message -----
From: "Kamil Paral" kparal@redhat.com To: "Fedora QA Development" qa-devel@lists.fedoraproject.org, "Josef Skladanka" jskladan@redhat.com Sent: Tuesday, January 21, 2014 4:00:33 PM Subject: Re: Taskotron wiki page
I also updated
https://fedoraproject.org/wiki/QA/Tools
with the list of our current projects. If you see something missing, please add it. Thanks.
Josef, I find this quite confusing:
https://bitbucket.org/rajcze/resultsdb https://bitbucket.org/rajcze/resultsdb_api https://bitbucket.org/rajcze/resultsdb_frontend https://fedorahosted.org/ResultsDB/ https://git.fedorahosted.org/git/ResultsDB.git
What is the canonical source? Where should people report issues? Please, pick one location to keep the project in (it seems we're going bitbucket, or bitbucket+phab way) and kill the other site. Also make sure issues can't be reported on two different places, and forward people to the single one. And make sure your wiki page points to a correct location: https://fedoraproject.org/wiki/ResultsDB (I updated it before learning there are other sources of ResultsDB. You might have some more under your sleeve.)
Thanks.
As a general note, I'm not fully happy when the source code lives somewhere else than the issues do. It confuses people. But if we want to keep easy-to-browse-and-fork functionality (bitbucket) and full-featured-review functionality (phab), it seems we don't have much choice. At least we should always disable the issue support on bitbucket for every project.
Weren't we going to move repos to phab as well?
M.
On Tue, 21 Jan 2014 10:23:40 -0500 (EST) Martin Krizek mkrizek@redhat.com wrote:
<snip>
As a general note, I'm not fully happy when the source code lives somewhere else than the issues do. It confuses people. But if we want to keep easy-to-browse-and-fork functionality (bitbucket) and full-featured-review functionality (phab), it seems we don't have much choice. At least we should always disable the issue support on bitbucket for every project.
Weren't we going to move repos to phab as well?
It's a possibility but not something that we'd discussed much as of yet.
Phabricator is capable of hosting repositories but it would require some reconfiguration and testing. The feature is a newer addition and I'd want to test it a bit in staging before moving all of our code there.
Any thoughts on how soon we might want to explore this? If we go this route, folks will have to upload their ssh pubkeys to phabricator because I strongly suspect there's no clean way of getting that data from FAS (if it's even possible at all).
Tim
Phabricator is capable of hosting repositories but it would require some reconfiguration and testing. The feature is a newer addition and I'd want to test it a bit in staging before moving all of our code there.
Any thoughts on how soon we might want to explore this? If we go this route, folks will have to upload their ssh pubkeys to phabricator because I strongly suspect there's no clean way of getting that data from FAS (if it's even possible at all).
I'd like to have a single location for our projects. If we consider abandoning bitbucket, let's do it ASAP, while we're not followed there yet by many people.
I see you've set up an example repo (in mirror mode) already: https://phab.qadevel.cloud.fedoraproject.org/diffusion/LTRN/
I don't use git web interface much, apart for community fork-me/follow features we don't have there anyway, so I'm not very demanding. The interface looks usable. But what's up with those commit hashes - "rLTRNda6fd348cdf3". Why does it have the rLTRN prefix? What is the Callsign?
На 22.01.2014 10:42, Kamil Paral написа:
Phabricator is capable of hosting repositories but it would require some reconfiguration and testing. The feature is a newer addition and I'd want to test it a bit in staging before moving all of our code there.
Any thoughts on how soon we might want to explore this? If we go this route, folks will have to upload their ssh pubkeys to phabricator because I strongly suspect there's no clean way of getting that data from FAS (if it's even possible at all).
I'd like to have a single location for our projects. If we consider abandoning bitbucket, let's do it ASAP, while we're not followed there yet by many people.
I see you've set up an example repo (in mirror mode) already: https://phab.qadevel.cloud.fedoraproject.org/diffusion/LTRN/
I don't use git web interface much, apart for community fork-me/follow features we don't have there anyway, so I'm not very demanding. The interface looks usable. But what's up with those commit hashes - "rLTRNda6fd348cdf3". Why does it have the rLTRN prefix? What is the Callsign?
Hi guys, a side note on the subject of migrating git repos:
BitBucket and GitHub are nice systems because of their fork/pull interface. This makes it very easy for collaborators to join, even if not very well experienced with git. GitHub here IMO has a leading edge because more people use it and is simply more popular. I myself prefer it.
I've also seen some projects use Gerrit for code review which for me as an experienced contributor always seems difficult to get. I always have to go back to the README b/c I forget how to push my changes for review - which is more or less the same as a pull request.
For some projects we also have fedorahosted.org which AFAIK is only command line based.
The above mentioned Phabricator doesn't seem to offer much in terms of easy contributor onboarding. At least I didn't see it.
I know these systems are designed to fit different teams and use cases and can't be compared so easily. My point is, when migrating look for a system with a clean and easy to use interface (preferably web as well) which will enable more contributors and less experienced contributors.
Can we look for a GitHub similar open source interface which can be hosted on Fedora infrastructure if we don't want to have our code base hosted on external providers ?
-- Alex
On Wed, 22 Jan 2014 14:03:17 +0200 Alexander Todorov atodorov@redhat.com wrote:
На 22.01.2014 10:42, Kamil Paral написа:
Phabricator is capable of hosting repositories but it would require some reconfiguration and testing. The feature is a newer addition and I'd want to test it a bit in staging before moving all of our code there.
Any thoughts on how soon we might want to explore this? If we go this route, folks will have to upload their ssh pubkeys to phabricator because I strongly suspect there's no clean way of getting that data from FAS (if it's even possible at all).
I'd like to have a single location for our projects. If we consider abandoning bitbucket, let's do it ASAP, while we're not followed there yet by many people.
I see you've set up an example repo (in mirror mode) already: https://phab.qadevel.cloud.fedoraproject.org/diffusion/LTRN/
I don't use git web interface much, apart for community fork-me/follow features we don't have there anyway, so I'm not very demanding. The interface looks usable. But what's up with those commit hashes - "rLTRNda6fd348cdf3". Why does it have the rLTRN prefix? What is the Callsign?
Hi guys, a side note on the subject of migrating git repos:
BitBucket and GitHub are nice systems because of their fork/pull interface. This makes it very easy for collaborators to join, even if not very well experienced with git. GitHub here IMO has a leading edge because more people use it and is simply more popular. I myself prefer it.
Yeah, that seems to be the popular method for doing things lately. github/bitbucket is a preference, though and I personally prefer bitbucket but the two are pretty much a wash. Either josef or I started putting stuff on bitbucket and we didn't see a really compelling reason to change at the time, so that's persisted.
I've also seen some projects use Gerrit for code review which for me as an experienced contributor always seems difficult to get. I always have to go back to the README b/c I forget how to push my changes for review - which is more or less the same as a pull request.
I've not used gerrit, but I've heard similar things about it. That and it's hosting requirements weren't quite what I wanted to deal with since it doesn't allow as much flexibility in repo hosting.
For some projects we also have fedorahosted.org which AFAIK is only command line based.
This (all fedorahosted) was dismissed as an option pretty early on from our experiences with the blocker tracking app. Trac + reviewboard + cgit does work, but it's a bit on the painful side and not something that I'm keen on duplicating.
The above mentioned Phabricator doesn't seem to offer much in terms of easy contributor onboarding. At least I didn't see it.
Well, we are lacking on "getting started" docs at the moment. It's on my todo list but I'm balancing that with a bunch of other priorities like getting code more coherent so that existing participants aren't completely starting from scratch when writing new functionality.
I know these systems are designed to fit different teams and use cases and can't be compared so easily. My point is, when migrating look for a system with a clean and easy to use interface (preferably web as well) which will enable more contributors and less experienced contributors.
I'm not saying that phabricator is perfect - it certainly has it's imperfections and warts. However, I don't know of anything that fits our needs better and it is improving at a pretty rapid pace.
As a side note, I'm far more interested in creating and supporting a good development community around qa tools than I am in making driveby patches easier. If the prospect of emailing a patch is too much work for someone ... I'm not all that saddened by the loss.
Can we look for a GitHub similar open source interface which can be hosted on Fedora infrastructure if we don't want to have our code base hosted on external providers ?
I'm a pretty solid 'no' on this or at the very least, 'not now'. I did look into other options before going forward with phabricator and we did have discussion on list as that process went along. All other options that I know of (github and bitbucket included) have severe shortcomings that, in my opinion, make them worse choices for us than phabricator.
The issue I have with github (and bitbucket, but for brevity's sake I'll just say github here) is not so much the proprietary nature of the platform but the issue tracker. If we had just one repo, it wouldn't be a huge issue but there are no good ways to aggregate issues from multiple repos and you end up with a situation where you either have issues in one repo and no great way to differentiate between the issues or you have issues spread across multiple trackers and no good way to search for them.
The open source github clones all have issues as well. Gitorous is a beast to maintain from a sysadmin perspective, is designed for a _much_ larger scale than we need and AFAIK doesn't have any issue tracking. Gitlab didn't support public repos last I looked and has the same issue tracker problem that github does. Rhodecode had stability issues when I tried it and was crashing on a daily basis when it ran out of file pointers. It's also pretty difficult to find the sources now that it's being sold as a product and I'd rather avoid it.
Another option would be to integrate standalone tools like reviewboard/gerrit, roundup/bugzilla and cgit/someotherhost. However, in order to be any less painful to use than the cgit+trac+reviewboard chain I mentioned above, a lot of glue code and setup would be required and I suspect that it would all lead to a rather fragile system and a timesink for maintenance.
As I mentioned above (and had a conversation with another phabricator admin who said pretty much the same thing), it's not perfect but I really don't know of another solution that would be better overall unless I started writing my own. Any toolchain we use is going to have issues and cause us pain in one way or another - my goal is to minimize those issues and pain points.
That all being said, I'm not against a better overall solution if it exists. I just haven't heard a suggestion of what that better solution is and the person-hours for the sysadmin and devel work to make it happen.
At this point, we're on a pretty tight schedule and have a limited window in which to deliver a new automation system. I'm completely unwilling to put us in the situation where we're saying "we're not done because we spent too much time on our support tools - they're really awesome, though". If we do decide to change things up, it's going to have to be done this week or it'll have to wait until f21 at the earliest.
Tim
On Wed, 22 Jan 2014 03:42:51 -0500 (EST) Kamil Paral kparal@redhat.com wrote:
Phabricator is capable of hosting repositories but it would require some reconfiguration and testing. The feature is a newer addition and I'd want to test it a bit in staging before moving all of our code there.
Any thoughts on how soon we might want to explore this? If we go this route, folks will have to upload their ssh pubkeys to phabricator because I strongly suspect there's no clean way of getting that data from FAS (if it's even possible at all).
I'd like to have a single location for our projects. If we consider abandoning bitbucket, let's do it ASAP, while we're not followed there yet by many people.
As I've been thinking about it more, my primary concern for using phabricator for git hosting is that we'd be self-hosting with a relatively new feature.
I'll see if I can get it working in stg today, though. I'm not willing to just enable it on production and see what happens but we can explore the idea.
I see you've set up an example repo (in mirror mode) already: https://phab.qadevel.cloud.fedoraproject.org/diffusion/LTRN/
Yeah, that's to support code reviews. The repo needs to be in phabricator before we can do that.
I don't use git web interface much, apart for community fork-me/follow features we don't have there anyway, so I'm not very demanding. The interface looks usable. But what's up with those commit hashes - "rLTRNda6fd348cdf3". Why does it have the rLTRN prefix? What is the Callsign?
The callsign is a unique identifier for the repo. In this case, LTRN is for libtaskotron.
The commit hashes are prepended by rLTRN to identify them as commits to the libtaskotron. r (revision) LTRN (repo callsign) <hash>
Tim
On Wed, 22 Jan 2014 08:57:53 -0700 Tim Flink tflink@redhat.com wrote:
On Wed, 22 Jan 2014 03:42:51 -0500 (EST) Kamil Paral kparal@redhat.com wrote:
Phabricator is capable of hosting repositories but it would require some reconfiguration and testing. The feature is a newer addition and I'd want to test it a bit in staging before moving all of our code there.
Any thoughts on how soon we might want to explore this? If we go this route, folks will have to upload their ssh pubkeys to phabricator because I strongly suspect there's no clean way of getting that data from FAS (if it's even possible at all).
I'd like to have a single location for our projects. If we consider abandoning bitbucket, let's do it ASAP, while we're not followed there yet by many people.
As I've been thinking about it more, my primary concern for using phabricator for git hosting is that we'd be self-hosting with a relatively new feature.
I'll see if I can get it working in stg today, though. I'm not willing to just enable it on production and see what happens but we can explore the idea.
It took a bit longer than I wanted it to but I got git repo hosting working on qadevel-stg and re-populated it with a copy of the production database (changed the header color to make it obvious which instance it is, though).
https://phab.qadevel-stg.cloud.fedoraproject.org/
The hosting does work over ssh, but I'm noticing some quirks - the ssh urls are displayed incorrectly. This may be fixed in the latest upstream (the version we're using is several weeks old) but I haven't checked yet. For the dummy repo I created: * shown: ssh://phab.qadevel-stg.cloud.fedoraproject.org/diffusion/PON/ * actual thing to clone if you want it to succeed: git@phab.qadevel-stg.cloud.fedoraproject.org:diffusion/PON
- http hosting doesn't work yet. I have some more tweaking to do in order to get that functional but it's do-able
- The repo names are ... weird. I understand why they end up like they do, but I was hoping the uris would contain the repo name, not the callsign.
The setup is pretty straightforward and doesn't really mess with the git hosting itself - just injects phabricator into the ssh auth mechanism in a similar way to how gitolite works. If we did decide to go this route for code hosting and something did go horribly wrong, we have backups of the raw repos and it would be pretty easy to resurrect them outside of phabricator.
Another feature that I haven't looked at much is mirroring - you can configure repos to push commits to a remote repository. The advantage here is that we could have the canonical upstream under our control and have bitbucket/github mirrors that other folks could use to create diffs from.
Anyhow, feel free to poke at it and leave thoughts.
Tim
https://phab.qadevel-stg.cloud.fedoraproject.org/
The hosting does work over ssh, but I'm noticing some quirks
the ssh urls are displayed incorrectly. This may be fixed in the latest upstream (the version we're using is several weeks old) but I haven't checked yet. For the dummy repo I created:
- shown: ssh://phab.qadevel-stg.cloud.fedoraproject.org/diffusion/PON/
- actual thing to clone if you want it to succeed: git@phab.qadevel-stg.cloud.fedoraproject.org:diffusion/PON
http hosting doesn't work yet. I have some more tweaking to do in order to get that functional but it's do-able
Does this mean that phab repos can't be accessed publicly at this moment? What about public git:// access, is that supported/working?
Does the http hosting require fixing some bugs, or is it purely a configuration issue?
- The repo names are ... weird. I understand why they end up like they do, but I was hoping the uris would contain the repo name, not the callsign.
That's a bit unfortunate. Does it mean that we need to differentiate all our repos by 4-letter access codes?
That would also mean that people cloning the repos need either provide a reasonable name to the git clone command, or they'll end up with cryptic directory names for our projects.
The setup is pretty straightforward and doesn't really mess with the git hosting itself - just injects phabricator into the ssh auth mechanism in a similar way to how gitolite works. If we did decide to go this route for code hosting and something did go horribly wrong, we have backups of the raw repos and it would be pretty easy to resurrect them outside of phabricator.
Another feature that I haven't looked at much is mirroring - you can configure repos to push commits to a remote repository. The advantage here is that we could have the canonical upstream under our control and have bitbucket/github mirrors that other folks could use to create diffs from.
This might be a very good idea. We can use our system while the public can easily find and access our code, fork it, send a pull request.
Or, if we find git hosting in Phab unsatisfactory, we can do it the other way round - host code on Github/Bitbucket and clone to Phab (if it helps something, for example reviews).
When it comes to Github/Bitbucket choice, I played with them a bit and both seem pretty equal. They are closed-source, they support teams, and because we won't need issue tracking there's not many other differences. Only Github is more popular and more people have an account there, so I think that would be the only reason to pick Github over Bitbucket.
On Thu, 23 Jan 2014 05:48:00 -0500 (EST) Kamil Paral kparal@redhat.com wrote:
https://phab.qadevel-stg.cloud.fedoraproject.org/
The hosting does work over ssh, but I'm noticing some quirks
- the ssh urls are displayed incorrectly. This may be fixed in the latest upstream (the version we're using is several weeks old)
but I haven't checked yet. For the dummy repo I created:
- shown: ssh://phab.qadevel-stg.cloud.fedoraproject.org/diffusion/PON/
- actual thing to clone if you want it to succeed: git@phab.qadevel-stg.cloud.fedoraproject.org:diffusion/PON
- http hosting doesn't work yet. I have some more tweaking to do in order to get that functional but it's do-able
Does this mean that phab repos can't be accessed publicly at this moment? What about public git:// access, is that supported/working?
There is no public access at the moment - just ssh based cloning. There is no git:// access, just http(s) and ssh.
Does the http hosting require fixing some bugs, or is it purely a configuration issue?
It's a configuration issue - phabricator can't find the git tool used to html-ize repo interactions and I don't think it'll be a huge problem to fix.
It would need to be fixed before deploying anything to production, though. I'm not willing to lose anonymous access, even if we did have mirrors on gh/bb
- The repo names are ... weird. I understand why they end up like
they do, but I was hoping the uris would contain the repo name, not the callsign.
That's a bit unfortunate. Does it mean that we need to differentiate all our repos by 4-letter access codes?
Yep, that's what it means.
That would also mean that people cloning the repos need either provide a reasonable name to the git clone command, or they'll end up with cryptic directory names for our projects.
Yeah. I can see if that's what upstream's intention was - code hosting is a relatively new feature and something may have changed but I doubt it.
The setup is pretty straightforward and doesn't really mess with the git hosting itself - just injects phabricator into the ssh auth mechanism in a similar way to how gitolite works. If we did decide to go this route for code hosting and something did go horribly wrong, we have backups of the raw repos and it would be pretty easy to resurrect them outside of phabricator.
Another feature that I haven't looked at much is mirroring - you can configure repos to push commits to a remote repository. The advantage here is that we could have the canonical upstream under our control and have bitbucket/github mirrors that other folks could use to create diffs from.
This might be a very good idea. We can use our system while the public can easily find and access our code, fork it, send a pull request.
I'm not sure how pull requests would work with reviews in phabricator but that would be the general idea, yeah.
If we're thinking it's a good idea to host repos in phabricator, I'll try to get the last config stuff done in the next day or so. I still want to test everything thoroughly in stg first but assuming that we don't hit any big problems, we can look at migrating next weekend.
Or, if we find git hosting in Phab unsatisfactory, we can do it the other way round - host code on Github/Bitbucket and clone to Phab (if it helps something, for example reviews).
Yeah, the libtaskotron repo that's currently on production is a mirror of bitbucket. Either way will work well enough for code reviews.
When it comes to Github/Bitbucket choice, I played with them a bit and both seem pretty equal. They are closed-source, they support teams, and because we won't need issue tracking there's not many other differences. Only Github is more popular and more people have an account there, so I think that would be the only reason to pick Github over Bitbucket.
I agree that the two systems are pretty much equivalent for what we're using. The one bit that's relevant to how we'd use either one is team management. I really don't like how github does team/group management and bitbucket's interface is easier to use.
I don't see how the benefit of switching to github outweighs the cost of doing so. The last time I talked to infra, they hadn't really seen an increase in contributions since they moved to github and I sincerely doubt that we would either. Their workflow has improved with the switch but almost all new contributors were already interested and involved in fedora.
I'd really like to put this discussion to bed - it keeps coming up and to be quite frank, we have more important things to do than talking about which site we're using to host code when there's so little difference between the two. I'm a big believer of choosing your battles and I'm not going to fight a migration to github but I'm not going to do the migration work, either. If there's enough support here and you want to do the migration, update all the needed links etc. (including the repos in phab) and find everyone's usernames on github for a team/group, go for it.
Tim
I'd really like to put this discussion to bed
OK, so the simplest approach I see here is: * Leave all repos at Bitbucket. * Let the git support in Phab mature a bit (maybe they'll fix annoyances like using callsigns for repo names). Resolve public hosting. * Don't change anything at least as long as we don't have FAS support in Phab. Until then, it doesn't really matter whether we manage commit permission using Bitbucket accounts or Phab accounts. * Once there is FAS support and improved git support in Phab, discuss whether we want to move the repos, so that we have everything in a single location (and mirror back to Bitbucket, if we want).
If this sounds reasonable, I'll probably move https://git.fedorahosted.org/cgit/fedora-qa.git/ to bitbucket as well, so that we start to get rid of fedorahosted and move all code into a single place.
On Fri, 24 Jan 2014 06:56:52 -0500 (EST) Kamil Paral kparal@redhat.com wrote:
I'd really like to put this discussion to bed
OK, so the simplest approach I see here is:
- Leave all repos at Bitbucket.
- Let the git support in Phab mature a bit (maybe they'll fix
annoyances like using callsigns for repo names). Resolve public hosting.
We should probably ask if they intend to leave things that way or if it will change. I suspect that it's not going to change since git hosting support was done a month ago.
Https hosting is technically working, git just thows a fit because the SSL cert on qadevel-stg is self signed.
- Don't change anything at least as long as we don't have FAS support
in Phab. Until then, it doesn't really matter whether we manage commit permission using Bitbucket accounts or Phab accounts.
I'm not sure that FAS matters much here. Even after we get FAS integration, there's not going to be any support for FAS groups in phabricator. AFAIK, phabricator doesn't support externally defined groups and even if it did, we'd have no way of getting that data since Persona doesn't have a mechanism to do anything other than auth.
- Once there is FAS support and improved git support in Phab, discuss
whether we want to move the repos, so that we have everything in a single location (and mirror back to Bitbucket, if we want).
Sure, I'm not in a huge hurry to move everything over. We have bigger fish to fry at the moment :)
If this sounds reasonable, I'll probably move https://git.fedorahosted.org/cgit/fedora-qa.git/ to bitbucket as well, so that we start to get rid of fedorahosted and move all code into a single place.
If we're going to hold off on migration for everything elase, I'm inclined to wait a bit on that. I'd prefer to avoid moving stuff twice when commit privileges etc. are setup and functioning for the fedorahosted repos right now.
Tim
- Don't change anything at least as long as we don't have FAS support
in Phab. Until then, it doesn't really matter whether we manage commit permission using Bitbucket accounts or Phab accounts.
I'm not sure that FAS matters much here. Even after we get FAS integration, there's not going to be any support for FAS groups in phabricator. AFAIK, phabricator doesn't support externally defined groups and even if it did, we'd have no way of getting that data since Persona doesn't have a mechanism to do anything other than auth.
Oh, I didn't know. OK, in that case it's even simpler - I don't see any major reason now to move the repos to Phab. Let's leave it at Bitbucket. The only thing we need to do is to update the README files in our repositories and point to Phab if people want to report issues.
If this sounds reasonable, I'll probably move https://git.fedorahosted.org/cgit/fedora-qa.git/ to bitbucket as well, so that we start to get rid of fedorahosted and move all code into a single place.
If we're going to hold off on migration for everything elase, I'm inclined to wait a bit on that. I'd prefer to avoid moving stuff twice when commit privileges etc. are setup and functioning for the fedorahosted repos right now.
OK, sounds reasonable.
qa-devel@lists.fedoraproject.org