Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
However we need to make a couple of decisions about naming conventions such that we can distinguish between rpms and container images as they are stored in Fedora DistGit. Also, alternatively we could setup a completely separate DistGit for container images that is disjoint from the one used for rpms. I would love feedback on everyone's preferences between those two options.
If we were to go with the former rather than the latter, we would need to find a way to "namespace" container images so they can be determined as different. I've thought about this a lot and I worry about defining a namespace by some alphanumberic sequence because I just know that at some point there will end up being a piece of software in the ecosystem that we want to package as a rpm that will share this pattern and result in problematic filtering. We could accept that risk and simply say "this sequence is a reserved word" or use a special character as the leading character in a DistGit repository name to signify that it is a container. A special character is the option I propose.
Proposal: In the event that container images (Dockerfiles for now, others in the future) and rpm spec files are to share the DistGit repositories, the container images leading character would be a special character. The decision upon which special character is open to the community at large to decide but certain ones impose obvious limitations for shell quoting and the like. For the sake of example I will use the special character '-' as my marker for a container image DistGit repository. Also in the example I'm going to use the cockpit-project[1] as it is already something that is packaged as an rpm[2] in Fedora and delivered as a container image[3] built from it's Fedora rpm set.
Example: Cockpit's rpm spec repo in DistGit would be stored as it is today: fedpkg clone cockpit
Cockpit's Dockerfile repo in DistGit would be stored with a leading special character: fedpkg clone -cockpit
Here, the leading '-' signifies that it is the container image DistGit repository which would make it an illegal name for an rpm by the N-V-R convention.
The technical details of this are all open to change so feel free to tell me I'm crazy and present a new idea and I'm happy to pursue it.
Looking forward to feedback.
Thank you, -AdamM
[0] - https://fedoraproject.org/wiki/Changes/Layered_Docker_Image_Build_Service [1] - http://cockpit-project.org/ [2] - https://apps.fedoraproject.org/packages/cockpit [3] - https://hub.docker.com/r/cockpit/ws/
On Tue, Nov 10, 2015 at 1:08 PM, Adam Miller maxamillion@fedoraproject.org wrote:
Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
However we need to make a couple of decisions about naming conventions such that we can distinguish between rpms and container images as they are stored in Fedora DistGit. Also, alternatively we could setup a completely separate DistGit for container images that is disjoint from the one used for rpms. I would love feedback on everyone's preferences between those two options.
If we were to go with the former rather than the latter, we would need to find a way to "namespace" container images so they can be determined as different. I've thought about this a lot and I worry about defining a namespace by some alphanumberic sequence because I just know that at some point there will end up being a piece of software in the ecosystem that we want to package as a rpm that will share this pattern and result in problematic filtering. We could accept that risk and simply say "this sequence is a reserved word" or use a special character as the leading character in a DistGit repository name to signify that it is a container. A special character is the option I propose.
Proposal: In the event that container images (Dockerfiles for now, others in the future) and rpm spec files are to share the DistGit repositories, the container images leading character would be a special character. The decision upon which special character is open to the community at large to decide but certain ones impose obvious limitations for shell quoting and the like. For the sake of example I will use the special character '-' as my marker for a container image DistGit repository. Also in the example I'm going to use the cockpit-project[1] as it is already something that is packaged as an rpm[2] in Fedora and delivered as a container image[3] built from it's Fedora rpm set.
Example: Cockpit's rpm spec repo in DistGit would be stored as it is today: fedpkg clone cockpit
Cockpit's Dockerfile repo in DistGit would be stored with aleading special character: fedpkg clone -cockpit
Here, the leading '-' signifies that it is the container imageDistGit repository which would make it an illegal name for an rpm by the N-V-R convention.
The technical details of this are all open to change so feel free to tell me I'm crazy and present a new idea and I'm happy to pursue it.
How much collaboration do we expect around the dockerfiles? I would think significantly more than an RPM spec file. With an RPM, there are only a handful of ways to build the software and adding patches isn't a high-frequency/multi-maintainer event (usually).
If we expect dockerfiles to be more dynamic, then I would prefer a separate git instance. Then we don't have to worry about collisions and hacks. We can also set up a separate set of ACLs around who is allow to commit to them, etc.
josh
On Tue, Nov 10, 2015 at 12:35 PM, Josh Boyer jwboyer@fedoraproject.org wrote:
On Tue, Nov 10, 2015 at 1:08 PM, Adam Miller maxamillion@fedoraproject.org wrote:
Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
However we need to make a couple of decisions about naming conventions such that we can distinguish between rpms and container images as they are stored in Fedora DistGit. Also, alternatively we could setup a completely separate DistGit for container images that is disjoint from the one used for rpms. I would love feedback on everyone's preferences between those two options.
If we were to go with the former rather than the latter, we would need to find a way to "namespace" container images so they can be determined as different. I've thought about this a lot and I worry about defining a namespace by some alphanumberic sequence because I just know that at some point there will end up being a piece of software in the ecosystem that we want to package as a rpm that will share this pattern and result in problematic filtering. We could accept that risk and simply say "this sequence is a reserved word" or use a special character as the leading character in a DistGit repository name to signify that it is a container. A special character is the option I propose.
Proposal: In the event that container images (Dockerfiles for now, others in the future) and rpm spec files are to share the DistGit repositories, the container images leading character would be a special character. The decision upon which special character is open to the community at large to decide but certain ones impose obvious limitations for shell quoting and the like. For the sake of example I will use the special character '-' as my marker for a container image DistGit repository. Also in the example I'm going to use the cockpit-project[1] as it is already something that is packaged as an rpm[2] in Fedora and delivered as a container image[3] built from it's Fedora rpm set.
Example: Cockpit's rpm spec repo in DistGit would be stored as it is today: fedpkg clone cockpit
Cockpit's Dockerfile repo in DistGit would be stored with aleading special character: fedpkg clone -cockpit
Here, the leading '-' signifies that it is the container imageDistGit repository which would make it an illegal name for an rpm by the N-V-R convention.
The technical details of this are all open to change so feel free to tell me I'm crazy and present a new idea and I'm happy to pursue it.
How much collaboration do we expect around the dockerfiles? I would think significantly more than an RPM spec file. With an RPM, there are only a handful of ways to build the software and adding patches isn't a high-frequency/multi-maintainer event (usually).
I'm honestly not entirely sure. I think that will just kind of organically happen. I would like to try and get a sense of what people would like to come out of it though so that we can cater to the preferred workload.
If we expect dockerfiles to be more dynamic, then I would prefer a separate git instance. Then we don't have to worry about collisions and hacks. We can also set up a separate set of ACLs around who is allow to commit to them, etc.
That's definitely an option. I'm open to either but we can do that now with what's surrounding DistGit.
-AdamM
josh
devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
On Tue, 2015-11-10 at 12:08 -0600, Adam Miller wrote:
Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
However we need to make a couple of decisions about naming conventions such that we can distinguish between rpms and container images as they are stored in Fedora DistGit. Also, alternatively we could setup a completely separate DistGit for container images that is disjoint from the one used for rpms. I would love feedback on everyone's preferences between those two options.
You could set it up in /srv/git/containers, in parallel to the current /srv/git/rpms ?
That nicely avoids naming collisions.
It also should be quite asy now that dist-git is in ansible, you could probably just factor out the path in the roles as a variable?
Cockpit's Dockerfile repo in DistGit would be stored with a leading special character: fedpkg clone -cockpit
With the "-" as a leading character, you open yourself to nice errors with the fact that it could be recognized as options. So you'd want to type: fedpkg clone -- -cockpit
Alternatively, if you separate the two distgits, you could have a fedcontainer tool, which would just be fedpkg but with its configuration pointing to the container distgit, the container build service, etc...
Or maybe just a: fedpkg --container clone cockpit ?
Most of the code (at least the one handling dist-git) could probably be reused without any difference.
All in all, separating them seems much simpler to me.
You'd have a bit of work up front (to adapt the Ansible playbooks and fedpkg slightly), but you pretty much avoid a whole class of issues with naming collisions, cli options, and future surprises.
-- Mathieu
On Tue, Nov 10, 2015 at 12:57 PM, Mathieu Bridon bochecha@daitauha.fr wrote:
On Tue, 2015-11-10 at 12:08 -0600, Adam Miller wrote:
Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
However we need to make a couple of decisions about naming conventions such that we can distinguish between rpms and container images as they are stored in Fedora DistGit. Also, alternatively we could setup a completely separate DistGit for container images that is disjoint from the one used for rpms. I would love feedback on everyone's preferences between those two options.
You could set it up in /srv/git/containers, in parallel to the current /srv/git/rpms ?
That nicely avoids naming collisions.
It also should be quite asy now that dist-git is in ansible, you could probably just factor out the path in the roles as a variable?
Cockpit's Dockerfile repo in DistGit would be stored with aleading special character: fedpkg clone -cockpit
With the "-" as a leading character, you open yourself to nice errors with the fact that it could be recognized as options. So you'd want to type: fedpkg clone -- -cockpit
Yeah, all special characters have special oddities. Some of them we have to consider from many targets and not just the shell so it's probably not ideal.
Alternatively, if you separate the two distgits, you could have a fedcontainer tool, which would just be fedpkg but with its configuration pointing to the container distgit, the container build service, etc...
Or maybe just a: fedpkg --container clone cockpit ?
Definitely something along these lines is what we'd want to do. I like the idea. +1
Most of the code (at least the one handling dist-git) could probably be reused without any difference.
Yes, it's likely something we'd want to maintain in the same code base and have one just be a light wrapper of the other with the different code paths as necessary.
All in all, separating them seems much simpler to me.
You'd have a bit of work up front (to adapt the Ansible playbooks and fedpkg slightly), but you pretty much avoid a whole class of issues with naming collisions, cli options, and future surprises.
Agreed.
+1
-AdamM
-- Mathieu -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Quoting Adam Miller (2015-11-10 19:08:25)
Cockpit's Dockerfile repo in DistGit would be stored with aleading special character: fedpkg clone -cockpit
How about
$ fedpkg clone cockpit-docker
Pros
* no need to change infrastructure
* easy to use
* repo name suggests content
Cons
* not a different namespace (in fact same namespace as rpms)
* someone can do `fedpkg clone cockpit-docker && cd cockpit-docker && fedpkg build` and get scared that it doesn't work
* `build` could be aware of Dockerfile and run `container-build` actually
* error could suggest using different build method
What's the way of building (qcow, docker base) images? Could that approach be resued? (meaning, what's the way of storing kickstart files atm?)
~~ Tomáš Tomeček Software Engineer Developer Experience UTC+2 (CEST)
On Thu, Nov 12, 2015 at 4:57 AM, Tomas Tomecek ttomecek@redhat.com wrote:
Quoting Adam Miller (2015-11-10 19:08:25)
Cockpit's Dockerfile repo in DistGit would be stored with aleading special character: fedpkg clone -cockpit
How about
$ fedpkg clone cockpit-docker
Pros
no need to change infrastructure
easy to use
repo name suggests content
I do like this, but I almost think if we were going to go this route we should have a policy around naming a container in DistGit with a prefix. I'd then suggest something like "ctr" to mean "container":
$ fedpkg clone ctr-cockpit
We would however need to request some kind of gating mechanism in pkgdb to make whatever the prefix is be a "reserved word" that wouldn't become the name of package in the future or else it would cause havok.
Another thing that was mentioned to me was if we have separate DistGit instances containing packages and containers named the same thing that koji would be unhappy so that's definitely something we need to keep in mind.
Cons
not a different namespace (in fact same namespace as rpms)
someone can do `fedpkg clone cockpit-docker && cd cockpit-docker && fedpkg build` and get scared that it doesn't work
`build` could be aware of Dockerfile and run `container-build` actually
error could suggest using different build method
I'd vote for a detection of Dockerfile without a spec file error out and suggest an alternative build method.
What's the way of building (qcow, docker base) images? Could that approach be resued? (meaning, what's the way of storing kickstart files atm?)
Currently the kickstarts are flattened and passed to the koji image-build command as a parameter, they are all stored as part of the spin-kickstarts[0] fedorahosted space and aren't really managed at all like DistGit.
-AdamM
[0] - https://fedorahosted.org/spin-kickstarts/
~~ Tomáš Tomeček Software Engineer Developer Experience UTC+2 (CEST)
Hi,
On Tue, Nov 10, 2015 at 1:08 PM, Adam Miller maxamillion@fedoraproject.org wrote:
If we were to go with the former rather than the latter, we would need to find a way to "namespace" container images so they can be determined as different. I've thought about this a lot and I worry about defining a namespace by some alphanumberic sequence because I just know that at some point there will end up being a piece of software in the ecosystem that we want to package as a rpm that will share this pattern and result in problematic filtering. We could accept that risk and simply say "this sequence is a reserved word" or use a special character as the leading character in a DistGit repository name to signify that it is a container.
git repositories normally use '/' to separate namespaces, so i'd propose
$ fedpkg clone containers/cockpit
and maybe add support for
$ fedpkg clone srpms/cockpit
at the same time.
This has the added benefit that cgit will automatically filter docker reposistories when you visit, e.g,
http://pkgs.fedoraproject.org/cgit/containers/
--Ray
On 12.11.2015 21:01, Ray Strode wrote:
Hi,
On Tue, Nov 10, 2015 at 1:08 PM, Adam Miller maxamillion@fedoraproject.org wrote:
If we were to go with the former rather than the latter, we would need to find a way to "namespace" container images so they can be determined as different. I've thought about this a lot and I worry about defining a namespace by some alphanumberic sequence because I just know that at some point there will end up being a piece of software in the ecosystem that we want to package as a rpm that will share this pattern and result in problematic filtering. We could accept that risk and simply say "this sequence is a reserved word" or use a special character as the leading character in a DistGit repository name to signify that it is a container.
git repositories normally use '/' to separate namespaces, so i'd propose
$ fedpkg clone containers/cockpit
and maybe add support for
$ fedpkg clone srpms/cockpit
at the same time.
This has the added benefit that cgit will automatically filter docker reposistories when you visit, e.g,
+1, sounds reasonable.
On Thu, Nov 12, 2015 at 03:01:49PM -0500, Ray Strode wrote:
Hi,
On Tue, Nov 10, 2015 at 1:08 PM, Adam Miller maxamillion@fedoraproject.org wrote:
If we were to go with the former rather than the latter, we would need to find a way to "namespace" container images so they can be determined as different. I've thought about this a lot and I worry about defining a namespace by some alphanumberic sequence because I just know that at some point there will end up being a piece of software in the ecosystem that we want to package as a rpm that will share this pattern and result in problematic filtering. We could accept that risk and simply say "this sequence is a reserved word" or use a special character as the leading character in a DistGit repository name to signify that it is a container.
git repositories normally use '/' to separate namespaces, so i'd propose
$ fedpkg clone containers/cockpit
and maybe add support for
$ fedpkg clone srpms/cockpit
at the same time.
This has the added benefit that cgit will automatically filter docker reposistories when you visit, e.g,
I like this too. Here are three thoughts:
Perhaps, we use 'dockerfiles' for the prefix instead of 'containers', because presumably there will be some whole new way to build containers in 2017, and we'll need to keep our dockerfiles/ repos separate from our awesomefiles/ repos.
We could also use this opportunity to move the kickstarts (another input to koji builds) away from https://fedorahosted.org/spin-kickstarts and over to dist-git as well, with a namespace like 'kickstarts/kde' and 'kickstarts/lxde'.
The existing rpm content could be moved to a 'specfiles/' namespace (or maybe 'srpms/'?) but we could further add some apache httpd rules that respond with a redirect to the 'srpms/' namespace if the requests to the base namespaceless-namespace level are met with a 404. -- "when in doubt, default to srpms/". That might help keep some of our existing tools working as-is without too much catastrophe.
On Fri, Nov 13, 2015 at 3:47 PM, Ralph Bean rbean@redhat.com wrote:
On Thu, Nov 12, 2015 at 03:01:49PM -0500, Ray Strode wrote:
Hi,
On Tue, Nov 10, 2015 at 1:08 PM, Adam Miller maxamillion@fedoraproject.org wrote:
If we were to go with the former rather than the latter, we would need to find a way to "namespace" container images so they can be determined as different. I've thought about this a lot and I worry about defining a namespace by some alphanumberic sequence because I just know that at some point there will end up being a piece of software in the ecosystem that we want to package as a rpm that will share this pattern and result in problematic filtering. We could accept that risk and simply say "this sequence is a reserved word" or use a special character as the leading character in a DistGit repository name to signify that it is a container.
git repositories normally use '/' to separate namespaces, so i'd propose
$ fedpkg clone containers/cockpit
and maybe add support for
$ fedpkg clone srpms/cockpit
at the same time.
This has the added benefit that cgit will automatically filter docker reposistories when you visit, e.g,
I like this too. Here are three thoughts:
Perhaps, we use 'dockerfiles' for the prefix instead of 'containers', because presumably there will be some whole new way to build containers in 2017, and we'll need to keep our dockerfiles/ repos separate from our awesomefiles/ repos.
We could also use this opportunity to move the kickstarts (another input to koji builds) away from https://fedorahosted.org/spin-kickstarts and over to dist-git as well, with a namespace like 'kickstarts/kde' and 'kickstarts/lxde'.
The existing rpm content could be moved to a 'specfiles/' namespace (or maybe 'srpms/'?) but we could further add some apache httpd rules that respond with a redirect to the 'srpms/' namespace if the requests to the base namespaceless-namespace level are met with a 404. -- "when in doubt, default to srpms/". That might help keep some of our existing tools working as-is without too much catastrophe.
+1
I'm a big fan of where this is going.
Maybe we should draft a proposal around this (maybe some notes in gobby?) and then start a new thread under a new name to generate more general discussion around this in the context of being able to deliver many different kinds of artifact for Fedora.Next. My only fear of continuing this topic thread is that folks who aren't interested in containers or docker might not be tuned in because of the subject line.
Thoughts?
-AdamM
-- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
On Tue, Nov 10, 2015 at 12:08:25PM -0600, Adam Miller wrote:
Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
So you're talking about one docker file per package, right? Or are these going to pull from multiple packages and have the docker container be a different level of object? Or both?
If you're talking about a 1:1 package:docker mapping then the only thing that makes sense to me is to have them live in the current system next to the .spec files. Anything else will easily introduce divergence from the rpm packages.
I also see that 'container packager' and 'rpm packager' are used in the build service page. I don't think these should be separate roles, everyone should be a packager. If you have one role working on container building for a package and another who only does rpm packaging we're going to, again, end up with things diverging.
Some things I'd suggest (given I don't know the answers to the above questions):
1. All bits have to come from koji built rpms. No rebuilding of bits for docker, it just installs existing bits.
2. docker files for packages live in dist-git next to the package spec file.
3. Any meta-containers collecting using packages are also an rpm package, following whatever naming convention gets decided on. fedora-docker-foo would make sense to me.
4. package level docker files are maintained by the same people who maintain the package.
On Fri, Nov 13, 2015 at 5:14 PM, Brian C. Lane bcl@redhat.com wrote:
On Tue, Nov 10, 2015 at 12:08:25PM -0600, Adam Miller wrote:
Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
So you're talking about one docker file per package, right? Or are these going to pull from multiple packages and have the docker container be a different level of object? Or both?
Both, it's mostly at the discretion of the person attempting to deliver something in a container. Some may be simple "I added httpd on top of the base Fedora Docker image" and others might be "I put all of ownCloud into a container so you can just run the whole thing with this one command"
That being said, I'm not married to the concept. It was just the idea going into it. If the general consensus among the community is "one Dockerfile to one package" we could go that route.
If you're talking about a 1:1 package:docker mapping then the only thing that makes sense to me is to have them live in the current system next to the .spec files. Anything else will easily introduce divergence from the rpm packages.
Absolutely, so far there has been the expectation that there will be divergence because of the more "dynamic" nature of what containers bring to the table.
I also see that 'container packager' and 'rpm packager' are used in the build service page. I don't think these should be separate roles, everyone should be a packager. If you have one role working on container building for a package and another who only does rpm packaging we're going to, again, end up with things diverging.
I believe they will diverge naturally as time goes on. Also, I don' t think everyone who has rpm packaging expertise is well versed in the ways of Docker and vice versa.
Some things I'd suggest (given I don't know the answers to the above questions):
- All bits have to come from koji built rpms. No rebuilding of bits
for docker, it just installs existing bits.
That is the plan, but configuration can take place inside the context of the container image build process to allow for single-command execution.
- docker files for packages live in dist-git next to the package spec
file.
We can, pending the agreement that's all we as a community/project use Dockerfiles for. The original idea going in to this though is that we could extend this to provide a more full featured solution for services that would need more than one package. Even simple things like httpd modules (mod_wsgi, mod_php, etc) would technically ship more rpms than they produce in the docker image.
Also in the distant future (out of scope for the original offering), we'd like to look at delivering multi-container apps via Nulecule[0] specifications.
- Any meta-containers collecting using packages are also an rpm
package, following whatever naming convention gets decided on. fedora-docker-foo would make sense to me.
I'm not sure I follow what you mean here, why would something that only comes as a Dockerfile also be a rpm?
- package level docker files are maintained by the same people who maintain
the package.
This certainly makes sense if we end up going the package:container 1:1 route.
-AdamM
[0] - https://github.com/projectatomic/nulecule
-- Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT) -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
On Fri 13 Nov 2015 06:34:21 PM EST Adam Miller maxamillion@fedoraproject.org wrote:
- package level docker files are maintained by the same people who maintain
the package.
This certainly makes sense if we end up going the package:container 1:1 route.
This assumption is not correct IMO. One docker image generally provides some functionality that can be used in different ways. It might be OK for basic httpd/mysql/postgres setup but I think inevitably you'll have people who will want to provide more usable/complete docker image which will have all of them integrated without need of involving Kubernetes or similar technologies.
-- Stanislav Ochotnicky sochotnicky@redhat.com Business System Analyst, PnT DevOps PMO Team - Brno
PGP: 7B087241 Red Hat Inc. http://cz.redhat.com
On Fri, Nov 13, 2015 at 05:34:21PM -0600, Adam Miller wrote:
On Fri, Nov 13, 2015 at 5:14 PM, Brian C. Lane bcl@redhat.com wrote:
On Tue, Nov 10, 2015 at 12:08:25PM -0600, Adam Miller wrote:
Hello all, In the Fedora 24 timeframe the Fedora Release Engineering group is aiming to deliver the Layered Image Build Service[0] to allow Fedora contributors to build containers images. In the first iteration of this we're targeting Docker layered image support. Part of this will be to allow Fedora contributors to maintain Dockerfiles much like we maintain rpm spec files via DistGit.
So you're talking about one docker file per package, right? Or are these going to pull from multiple packages and have the docker container be a different level of object? Or both?
Both, it's mostly at the discretion of the person attempting to deliver something in a container. Some may be simple "I added httpd on top of the base Fedora Docker image" and others might be "I put all of ownCloud into a container so you can just run the whole thing with this one command"
That being said, I'm not married to the concept. It was just the idea going into it. If the general consensus among the community is "one Dockerfile to one package" we could go that route.
If you're talking about a 1:1 package:docker mapping then the only thing that makes sense to me is to have them live in the current system next to the .spec files. Anything else will easily introduce divergence from the rpm packages.
Absolutely, so far there has been the expectation that there will be divergence because of the more "dynamic" nature of what containers bring to the table.
Ok, that makes sense. So some packages may also have docker images but most of this will be more along the lines of a spin for a specific collections of packages built from our existing rpms.
I also see that 'container packager' and 'rpm packager' are used in the build service page. I don't think these should be separate roles, everyone should be a packager. If you have one role working on container building for a package and another who only does rpm packaging we're going to, again, end up with things diverging.
I believe they will diverge naturally as time goes on. Also, I don' t think everyone who has rpm packaging expertise is well versed in the ways of Docker and vice versa.
I think packagers should be familiar with both if we're going to be producing both. Current rpm maintainers have a body of knowledge surrounding their packages and upstreams that should not be lost or ignored when using them for Docker images.
Some things I'd suggest (given I don't know the answers to the above questions):
- All bits have to come from koji built rpms. No rebuilding of bits
for docker, it just installs existing bits.
That is the plan, but configuration can take place inside the context of the container image build process to allow for single-command execution.
Sure, that's the big benefit of Docker, reasonable configuration of existing packages makes sense. As long as that's where it stops and the binaries are the same as what we produce with the rpms. I don't want to see us rebuilding things for the Docker container and using different binaries or pulling in external dependencies that aren't already packaged as an rpm.
- docker files for packages live in dist-git next to the package spec
file.
We can, pending the agreement that's all we as a community/project use Dockerfiles for. The original idea going in to this though is that we could extend this to provide a more full featured solution for services that would need more than one package. Even simple things like httpd modules (mod_wsgi, mod_php, etc) would technically ship more rpms than they produce in the docker image.
Also in the distant future (out of scope for the original offering), we'd like to look at delivering multi-container apps via Nulecule[0] specifications.
- Any meta-containers collecting using packages are also an rpm
package, following whatever naming convention gets decided on. fedora-docker-foo would make sense to me.
I'm not sure I follow what you mean here, why would something that only comes as a Dockerfile also be a rpm?
What I'm thinking is that dist-git doesn't have to always produce an rpm. If the package has both, then a fedpkg build produces both, and we can add fedpkg build --rpm and fedpkg build --docker. If a package only has a docker file then it only produces a docker image. This reuses the tooling we are all familiar with, and keeps everything in the same place. Use a container specific naming convention and it can be expanded to support other containers as they become popular.
- package level docker files are maintained by the same people who maintain
the package.
This certainly makes sense if we end up going the package:container 1:1 route.
I think it makes sense either way. Much of the packaging guidelines should still apply to creating containers.
Hello all, After consideration and discussion of feedback from this thread with members of the Fedora Infrastructure and Fedora Apps team (special thanks to pingou and threebean), the following is the proposal we'd like to move forward with:
DistGit Restructure/Namespace - Move all current rpm packages that live under repositories/ to live under repositories/rpms/ - Create new namespaces for other things (i.e. - repositories/docker/)
Current DistGit layout: /srv/git/repositories/<pkg>.git future: -> /srv/git/repositories/rpms/<pkg>.git (symlinked to the old location for a migration time period) => At one point, will break `git pull` if the remote url isn't fixed -> /srv/git/repositories/docker/<pkg>.git => /srv/git/repositories/<future_new_artifact>/<pkg>.git
This will require updates to different aspects of the Fedora Apps/Infrastructure over time which will be addressed in phases. Everything will continue to function with business as usual during the transition periods with the symlinks in place for backwards compatibility.
In the future, users of fedpkg can do the following:
fedpkg clone <namespace>/<pkg_name>
Examples:
fedpkg clone rpms/foo fedpkg clone docker/bar
In the event the namespace is omitted, fedpkg will default to rpms/foo in order to maintain function-alike backwards compatibility.
If this proposal is agreed upon by general consensus, the teams involved will begin to scope out what systems need what modifications and begin the migration, with updates along the way (in case folks are interested).
I look forward to feedback, thank you.
-AdamM
-
Adam Miller -
Brian C. Lane -
Josh Boyer -
Mathieu Bridon -
Petr Spacek -
Ralph Bean -
Ray Strode -
Stanislav Ochotnicky -
Tomas Tomecek