-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello:
Several weeks ago, somebody wrote to the list about the continually growing size of the Docs web.git repository. I took a look around, and confessed that most of the storage is taken up by multiple copies of the Musicians' Guide. It's time to fix this.
I uploaded the audio files to SoundCloud long ago, but I don't think we should use them as our primary place to hose this content. For the "Ardour" chapter, we cannot host the files for licensing reasons... aside from that, I feel it's better for us to be self-sufficient and host our own files.
They're actually not that large (less than 100 MiB). The problem is that Publican copies the audio files every time the Guide is published in any language. We really only need one copy of the audio files. They don't change between releases, and they're not translated.
POSSIBLE SOLUTION: Can I make a static directory in the web.git repository, just for the Musicians' Guide's audio files? Then delete all the current copies, and replace with a symlink?
I don't want to do first and ask questions later, just in case it would interfere with Publican... that would be bad news so close to a release!
Christopher.
It seems like a good candidate for being hosted outside of git. Perhaps we could get some space on fedorahosted.org for the files?
On 05/24/2012 09:27 AM, Ben Cotton wrote:
It seems like a good candidate for being hosted outside of git. Perhaps we could get some space on fedorahosted.org for the files?
That would (kinda) solve the problem for now; but the deeper underlying problem is that I think we're reaching the limit of what we can sensibly do with Git :)
This was only ever a stop-gap mechanism while we implemented proper packaging for the docs; unfortunately, like many systems that work "well enough", we never got to replacing it.
Personal admission: it was actually my personal goal during my year on the board to make this happen, but I never got to it ;)
So: what do we need?
1. I think the procedural obstacles to getting every writer and translator access to Koji are probably bigger than we want to handle. We therefore need to stand up a build system of our own. Ideally, this would be a Koji instance of our own: the benefit of Koji is that it insulates the build process very well from issues that might be specific to people's own machines. If that's too much work to pull off in the short term, we should simply create a build server with current publican and publican-fedora installed on it, and access granted to anyone in the docs-publishers group. Packages built on this machine would be placed in a repo.
2. We would need to modify the staging server that currently pulls our docs.git to instead install packages built on the build server.
I think that if we pull together, we should be able to get this up and running reasonably soon. Who's interested in collaborating on a project like this?
Cheers Rudi
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/23/2012 10:32 PM, Ruediger Landmann wrote:
On 05/24/2012 09:27 AM, Ben Cotton wrote:
It seems like a good candidate for being hosted outside of git. Perhaps we could get some space on fedorahosted.org for the files?
That would (kinda) solve the problem for now; but the deeper underlying problem is that I think we're reaching the limit of what we can sensibly do with Git :)
This was only ever a stop-gap mechanism while we implemented proper packaging for the docs; unfortunately, like many systems that work "well enough", we never got to replacing it.
Personal admission: it was actually my personal goal during my year on the board to make this happen, but I never got to it ;)
So: what do we need?
- I think the procedural obstacles to getting every writer and
translator access to Koji are probably bigger than we want to handle. We therefore need to stand up a build system of our own. Ideally, this would be a Koji instance of our own: the benefit of Koji is that it insulates the build process very well from issues that might be specific to people's own machines. If that's too much work to pull off in the short term, we should simply create a build server with current publican and publican-fedora installed on it, and access granted to anyone in the docs-publishers group. Packages built on this machine would be placed in a repo.
- We would need to modify the staging server that currently pulls
our docs.git to instead install packages built on the build server.
I think that if we pull together, we should be able to get this up and running reasonably soon. Who's interested in collaborating on a project like this?
Cheers Rudi
I am! I don't know if I can, time-wise and programming language-wise, but that's not what you asked.
Christopher.
I am in the same boat with Christopher.
On Wed, May 23, 2012 at 8:57 PM, Christopher R. Antila < crantila@fedoraproject.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/23/2012 10:32 PM, Ruediger Landmann wrote:
On 05/24/2012 09:27 AM, Ben Cotton wrote:
It seems like a good candidate for being hosted outside of git. Perhaps we could get some space on fedorahosted.org for the files?
That would (kinda) solve the problem for now; but the deeper underlying problem is that I think we're reaching the limit of what we can sensibly do with Git :)
This was only ever a stop-gap mechanism while we implemented proper packaging for the docs; unfortunately, like many systems that work "well enough", we never got to replacing it.
Personal admission: it was actually my personal goal during my year on the board to make this happen, but I never got to it ;)
So: what do we need?
- I think the procedural obstacles to getting every writer and
translator access to Koji are probably bigger than we want to handle. We therefore need to stand up a build system of our own. Ideally, this would be a Koji instance of our own: the benefit of Koji is that it insulates the build process very well from issues that might be specific to people's own machines. If that's too much work to pull off in the short term, we should simply create a build server with current publican and publican-fedora installed on it, and access granted to anyone in the docs-publishers group. Packages built on this machine would be placed in a repo.
- We would need to modify the staging server that currently pulls
our docs.git to instead install packages built on the build server.
I think that if we pull together, we should be able to get this up and running reasonably soon. Who's interested in collaborating on a project like this?
Cheers Rudi
I am! I don't know if I can, time-wise and programming language-wise, but that's not what you asked.
Christopher. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux)
iQEcBAEBAgAGBQJPvbGvAAoJEInCktGVqZ8VgQAH/1guB2AlqMYDYQO32tPka5ZW SLUxArMbu50hZQ+14VA7bKulTL3xP0sEp1jPyEDve4GaKtVhxDaW38pPYR6jMBQF THRPHsXPur0H5aw8bPmZZdnQ7gxFX3KF+zSFQK10MWUBADehjatk0ob7+BU5Eo42 xOKe59NwJC9hbqOobvNUAI/GwaLBepGRj7NzkCN0nOqfSDB+UgTC99zuh+1seCnw YYjoeSsOSebqXhpkGhbrYt4Uk+9H+IapUmG2suhPVn4SxHoQyEj+v8xXBXk2MPzf 2wYFIQTVWVXUvZyBcmSp8CZQ429bUKAVDbPd1TitxCVIxRqeXeSJOZ5JHmOyMA4= =S/fM
-----END PGP SIGNATURE-----
docs mailing list docs@lists.fedoraproject.org To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/docs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/23/2012 10:32 PM, Ruediger Landmann wrote:
On 05/24/2012 09:27 AM, Ben Cotton wrote:
It seems like a good candidate for being hosted outside of git. Perhaps we could get some space on fedorahosted.org for the files?
That would (kinda) solve the problem for now; but the deeper underlying problem is that I think we're reaching the limit of what we can sensibly do with Git :)
This was only ever a stop-gap mechanism while we implemented proper packaging for the docs; unfortunately, like many systems that work "well enough", we never got to replacing it.
Personal admission: it was actually my personal goal during my year on the board to make this happen, but I never got to it ;)
So: what do we need?
- I think the procedural obstacles to getting every writer and
translator access to Koji are probably bigger than we want to handle. We therefore need to stand up a build system of our own. Ideally, this would be a Koji instance of our own: the benefit of Koji is that it insulates the build process very well from issues that might be specific to people's own machines. If that's too much work to pull off in the short term, we should simply create a build server with current publican and publican-fedora installed on it, and access granted to anyone in the docs-publishers group. Packages built on this machine would be placed in a repo.
- We would need to modify the staging server that currently pulls
our docs.git to instead install packages built on the build server.
I think that if we pull together, we should be able to get this up and running reasonably soon. Who's interested in collaborating on a project like this?
Cheers Rudi
I spoke with Dennis Gilmore this morning about getting a separate koji instance for Docs. Looks like doing this will be a lot easier than what we were thinking:
<Sparks> What would be required to stand up a separate instance of koji for Docs? <dgilmore> Sparks: why would you want a seperate instance? <dgilmore> Sparks: i dont see any valid reason to do so <Sparks> dgilmore: For the Docs website. Publican has the ability publish documentation from packages (separate repo from the Fedora repo) for the website. We want to replace the git repo that operates it now. <dgilmore> Sparks: and why does that need a seperate koji? <Sparks> The git repo has gotten HUGE and is becoming an issue. <dgilmore> Sparks: so, why does that mean a seperate koji instance <dgilmore> Sparks: we could use a seperate tag and targets in koji <dgilmore> i dont see why it would need its own koji <Sparks> dgilmore: It's either that or try to get everyone setup as packagers. <dgilmore> Sparks: get everyone setup as packagers <Sparks> dgilmore: Except that they really won't be packagers. <dgilmore> Sparks: though you dont need to be a a packager to get a koji cert <dgilmore> you just need fas <dgilmore> Sparks: what would be the workflow? <Sparks> dgilmore: Okay, and with that we can send packages through koji and tag them separately? <Sparks> dgilmore: Basically we just tell Publican to build the package and submit it to koji. The Publican software does all the work. <dgilmore> Sparks: I still really dont know what your trying to do. pretend im an idiot(not really that hard) and explain what it is and how it should work <Sparks> dgilmore: I'm not far off... <Sparks> dgilmore: So Publican will make an SRPM package, submit it to koji destined for a repo. Our Publican backend will install those packages and publish the data to the website. <Sparks> ...as I understand it <dgilmore> Sparks: so we would need to set up seperate tags and tagets for it, defining a koji policy allowing srpms to be built. likely we would add a new group to koji and add people allowed to build docs to it and limit access to the tags/targets to people in that group <dgilmore> Sparks: so its all doable <Sparks> dgilmore: Well that's a lot easier. :) <Sparks> dgilmore: We already have a group in FAS for people that should have access (docs-publishers). Should I open a ticket? <dgilmore> Sparks: koji doesnt know anything about fas <dgilmore> Sparks: please file a ticket. We wont be able to make changes until after f17 is done <Sparks> dgilmore: That's fine. We're not completely ready for the transition so after F17 would be good. <Sparks> dgilmore: Thanks!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/24/2012 11:56 AM, Eric Christensen wrote:
On 05/23/2012 10:32 PM, Ruediger Landmann wrote:
- I think the procedural obstacles to getting every writer and
translator access to Koji are probably bigger than we want to handle. We therefore need to stand up a build system of our own. Ideally, this would be a Koji instance of our own: the benefit of Koji is that it insulates the build process very well from issues that might be specific to people's own machines. If that's too much work to pull off in the short term, we should simply create a build server with current publican and publican-fedora installed on it, and access granted to anyone in the docs-publishers group. Packages built on this machine would be placed in a repo.
I spoke with Dennis Gilmore this morning about getting a separate koji instance for Docs. Looks like doing this will be a lot easier than what we were thinking:
...
But we were planning an adventure!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, May 24, 2012 at 11:56:13AM -0400, Eric Christensen wrote:
<dgilmore> Sparks: please file a ticket. We wont be able to make changes until after f17 is done <Sparks> dgilmore: That's fine. We're not completely ready for the transition so after F17 would be good. <Sparks> dgilmore: Thanks!
Sparks, Is a ticket filed for this then?
Zach
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/31/2012 08:53 AM, Zach Oglesby wrote:
On Thu, May 24, 2012 at 11:56:13AM -0400, Eric Christensen wrote:
<dgilmore> Sparks: please file a ticket. We wont be able to make changes until after f17 is done <Sparks> dgilmore: That's fine. We're not completely ready for the transition so after F17 would be good. <Sparks> dgilmore: Thanks!
Sparks, Is a ticket filed for this then?
Zach
Not yet. I think we were going to talk about it during a meeting first. I can do so, though.
- -Eric
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, May 31, 2012 at 08:54:58AM -0400, Eric Christensen wrote:
On 05/31/2012 08:53 AM, Zach Oglesby wrote:
On Thu, May 24, 2012 at 11:56:13AM -0400, Eric Christensen wrote:
<dgilmore> Sparks: please file a ticket. We wont be able to make changes until after f17 is done <Sparks> dgilmore: That's fine. We're not completely ready for the transition so after F17 would be good. <Sparks> dgilmore: Thanks!
Sparks, Is a ticket filed for this then?
Zach
Not yet. I think we were going to talk about it during a meeting first. I can do so, though.
Thanks, I did not look at the adgenda for the next meeting, I see it is there now.
Zach
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/24/2012 11:56 AM, Eric Christensen wrote:
On 05/23/2012 10:32 PM, Ruediger Landmann wrote:
On 05/24/2012 09:27 AM, Ben Cotton wrote:
It seems like a good candidate for being hosted outside of git. Perhaps we could get some space on fedorahosted.org for the files?
That would (kinda) solve the problem for now; but the deeper underlying problem is that I think we're reaching the limit of what we can sensibly do with Git :)
This was only ever a stop-gap mechanism while we implemented proper packaging for the docs; unfortunately, like many systems that work "well enough", we never got to replacing it.
Personal admission: it was actually my personal goal during my year on the board to make this happen, but I never got to it ;)
So: what do we need?
- I think the procedural obstacles to getting every writer and
translator access to Koji are probably bigger than we want to handle. We therefore need to stand up a build system of our own. Ideally, this would be a Koji instance of our own: the benefit of Koji is that it insulates the build process very well from issues that might be specific to people's own machines. If that's too much work to pull off in the short term, we should simply create a build server with current publican and publican-fedora installed on it, and access granted to anyone in the docs-publishers group. Packages built on this machine would be placed in a repo.
- We would need to modify the staging server that currently
pulls our docs.git to instead install packages built on the build server.
I think that if we pull together, we should be able to get this up and running reasonably soon. Who's interested in collaborating on a project like this?
Cheers Rudi
I spoke with Dennis Gilmore this morning about getting a separate koji instance for Docs. Looks like doing this will be a lot easier than what we were thinking:
<Sparks> What would be required to stand up a separate instance of koji for Docs? <dgilmore> Sparks: why would you want a seperate instance? <dgilmore> Sparks: i dont see any valid reason to do so <Sparks> dgilmore: For the Docs website. Publican has the ability publish documentation from packages (separate repo from the Fedora repo) for the website. We want to replace the git repo that operates it now. <dgilmore> Sparks: and why does that need a seperate koji? <Sparks> The git repo has gotten HUGE and is becoming an issue. <dgilmore> Sparks: so, why does that mean a seperate koji instance <dgilmore> Sparks: we could use a seperate tag and targets in koji <dgilmore> i dont see why it would need its own koji <Sparks> dgilmore: It's either that or try to get everyone setup as packagers. <dgilmore> Sparks: get everyone setup as packagers <Sparks> dgilmore: Except that they really won't be packagers. <dgilmore> Sparks: though you dont need to be a a packager to get a koji cert <dgilmore> you just need fas <dgilmore> Sparks: what would be the workflow? <Sparks> dgilmore: Okay, and with that we can send packages through koji and tag them separately? <Sparks> dgilmore: Basically we just tell Publican to build the package and submit it to koji. The Publican software does all the work. <dgilmore> Sparks: I still really dont know what your trying to do. pretend im an idiot(not really that hard) and explain what it is and how it should work <Sparks> dgilmore: I'm not far off... <Sparks> dgilmore: So Publican will make an SRPM package, submit it to koji destined for a repo. Our Publican backend will install those packages and publish the data to the website. <Sparks> ...as I understand it <dgilmore> Sparks: so we would need to set up seperate tags and tagets for it, defining a koji policy allowing srpms to be built. likely we would add a new group to koji and add people allowed to build docs to it and limit access to the tags/targets to people in that group <dgilmore> Sparks: so its all doable <Sparks> dgilmore: Well that's a lot easier. :) <Sparks> dgilmore: We already have a group in FAS for people that should have access (docs-publishers). Should I open a ticket? <dgilmore> Sparks: koji doesnt know anything about fas <dgilmore> Sparks: please file a ticket. We wont be able to make changes until after f17 is done <Sparks> dgilmore: That's fine. We're not completely ready for the transition so after F17 would be good. <Sparks> dgilmore: Thanks!
I've put in an Infra ticket (https://fedorahosted.org/fedora-infrastructure/ticket/3320) to request this.
- -Eric
-
Alexander Moore -
Ben Cotton -
Christopher Antila -
Eric Christensen -
Ruediger Landmann -
Zach Oglesby