Migrating from fedimg to ore

[Sayan Chowdhury]

Last few days, I spent doing a feasibility analysis on the migrating
from fedimg to ore, a subproject of mantle to release the Fedora cloud
images.

ore/mantle comes from the CoreOS community. mantle is comprised of
multiple utility projects to keep the CL bits together.

The workflow for fedimg right now:
- Downloads the raw.xz image
- Uses the ImportVolume AWS API[1] to create a volume via S3
- Creates the snapshot using the earlier created volume
- Registers the AMI of the snapshot provided
- Copies the AMI to other regions and makes the AMI and the snapshot public.

What if we migrate to Ore?
The process remain more or less the same except for a few changes:
- Downloads the .qcow2 image
   * Once we start using the .qcow2 image we don't need releng to
produce two formats i.e. qcow2 & .raw.xz
- Converts the .qcow2 image to .vmdk format image using qemu-img
- Uses the newer supported AWS API i.e. ImportSnapshot which directly
creates the Snapshot uploading the image via S3
- Registers the AMI with the snapshot.
- Copies the AMI to other regions.

- Ore also skips any of the steps already processed which makes it
easier for the AMIs maintainer.
- Ore has a more maintainers than fedimg.
- It's better to focus on a single project than maintaining two.

Two areas where it needs more research

- how do we send out the fedmsg messages when we move to ore
- making the images & the snapshots public.

Thoughts?

[Kevin Fenzi]

On 07/20/2018 08:15 AM, Sayan Chowdhury wrote:
> Last few days, I spent doing a feasibility analysis on the migrating
> from fedimg to ore, a subproject of mantle to release the Fedora cloud
> images.
>
> ore/mantle comes from the CoreOS community. mantle is comprised of
> multiple utility projects to keep the CL bits together.
>
> The workflow for fedimg right now:
> Downloads the raw.xz image
> Uses the ImportVolume AWS API[1] to create a volume via S3
> Creates the snapshot using the earlier created volume
> Registers the AMI of the snapshot provided
> Copies the AMI to other regions and makes the AMI and the snapshot public.
>
> What if we migrate to Ore?
> The process remain more or less the same except for a few changes:
> Downloads the .qcow2 image
> Once we start using the .qcow2 image we don't need releng to
>
> produce two formats i.e. qcow2 & .raw.xz

I thought we needed raw for some cloud provider? perhaps not.
> Converts the .qcow2 image to .vmdk format image using qemu-img
> Uses the newer supported AWS API i.e. ImportSnapshot which directly
>
> creates the Snapshot uploading the image via S3
>
> Registers the AMI with the snapshot.
>
> Copies the AMI to other regions.
>
> Ore also skips any of the steps already processed which makes it
>
> easier for the AMIs maintainer.
> Ore has a more maintainers than fedimg.
> It's better to focus on a single project than maintaining two.
>
> Two areas where it needs more research
> how do we send out the fedmsg messages when we move to ore
> making the images & the snapshots public.
>
> Thoughts?

All this is in go I assume and not packaged?

I agree it would be nice to have fewer things and more maintainers on
those things. :)

kevin

[Jim Perrin]

On 07/22/2018 11:00 AM, Kevin Fenzi wrote:
> On 07/20/2018 08:15 AM, Sayan Chowdhury wrote:
> > Last few days, I spent doing a feasibility analysis on the migrating
> > from fedimg to ore, a subproject of mantle to release the Fedora cloud
> > images.
> >
> > ore/mantle comes from the CoreOS community. mantle is comprised of
> > multiple utility projects to keep the CL bits together.
> >
> > The workflow for fedimg right now:
> > Downloads the raw.xz image
> > Uses the ImportVolume AWS API[1] to create a volume via S3
> > Creates the snapshot using the earlier created volume
> > Registers the AMI of the snapshot provided
> > Copies the AMI to other regions and makes the AMI and the snapshot public.
> >
> > What if we migrate to Ore?
> > The process remain more or less the same except for a few changes:
> > Downloads the .qcow2 image
> > Once we start using the .qcow2 image we don't need releng to
> >
> > produce two formats i.e. qcow2 & .raw.xz
>
> I thought we needed raw for some cloud provider? perhaps not.

For amazon we don't. For others like Packet, I think it's still
necessary.  Is the Ore repo public and something the Packet folks could
review?

I've added Carl from Packet on this email. Carl, you may want to
subscribe to this mailing list while we work through the tooling around
this. I'd like to make sure we consider your use case as well.
> > Converts the .qcow2 image to .vmdk format image using qemu-img
> > Uses the newer supported AWS API i.e. ImportSnapshot which directly
> >
> > creates the Snapshot uploading the image via S3
> >
> > Registers the AMI with the snapshot.
> >
> > Copies the AMI to other regions.
> >
> > Ore also skips any of the steps already processed which makes it
> >
> > easier for the AMIs maintainer.
> > Ore has a more maintainers than fedimg.
> > It's better to focus on a single project than maintaining two.
> >
> > Two areas where it needs more research
> > how do we send out the fedmsg messages when we move to ore
> > making the images & the snapshots public.
> >
> > Thoughts?
>
> All this is in go I assume and not packaged?
>
> I agree it would be nice to have fewer things and more maintainers on
> those things. :)
>
> kevin
>
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]...

[Dennis Gilmore]

El dom, 22-07-2018 a las 11:00 -0700, Kevin Fenzi escribió:
> On 07/20/2018 08:15 AM, Sayan Chowdhury wrote:
> > Last few days, I spent doing a feasibility analysis on the
> > migrating
> > from fedimg to ore, a subproject of mantle to release the Fedora
> > cloud
> > images.
> >
> > ore/mantle comes from the CoreOS community. mantle is comprised of
> > multiple utility projects to keep the CL bits together.
> >
> > The workflow for fedimg right now:
> > Downloads the raw.xz image
> > Uses the ImportVolume AWS API[1] to create a volume via S3
> > Creates the snapshot using the earlier created volume
> > Registers the AMI of the snapshot provided
> > Copies the AMI to other regions and makes the AMI and the
> >
> > snapshot public.
> >
> > What if we migrate to Ore?
> > The process remain more or less the same except for a few changes:
> > Downloads the .qcow2 image
> > Once we start using the .qcow2 image we don't need releng to
> >
> > produce two formats i.e. qcow2 & .raw.xz
>
> I thought we needed raw for some cloud provider? perhaps not.

virt-builder needs it, as do a few other things afaik.

Dennis
> > Converts the .qcow2 image to .vmdk format image using qemu-img
> > Uses the newer supported AWS API i.e. ImportSnapshot which
> >
> > directly
> > creates the Snapshot uploading the image via S3
> >
> > Registers the AMI with the snapshot.
> >
> > Copies the AMI to other regions.
> >
> > Ore also skips any of the steps already processed which makes it
> >
> > easier for the AMIs maintainer.
> > Ore has a more maintainers than fedimg.
> > It's better to focus on a single project than maintaining two.
> >
> > Two areas where it needs more research
> > how do we send out the fedmsg messages when we move to ore
> > making the images & the snapshots public.
> >
> > Thoughts?
>
> All this is in go I assume and not packaged?
>
> I agree it would be nice to have fewer things and more maintainers on
> those things. :)
>
> kevin
>
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> ect.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelin
> es
> List Archives: https://lists.fedoraproject.org/archives/list/infrastr
> [e-mail]/message/6BZKHINX3A2VELWQRTGF4XPB2BD5EH
> TM/

[CoreOS] Migrating from fedimg to ore

[Neal Gompa]

On Sat, Jul 21, 2018 at 10:50 PM Sayan Chowdhury  wrote:
> Last few days, I spent doing a feasibility analysis on the migrating
> from fedimg to ore, a subproject of mantle to release the Fedora cloud
> images.
>
> ore/mantle comes from the CoreOS community. mantle is comprised of
> multiple utility projects to keep the CL bits together.
>
> The workflow for fedimg right now:
> Downloads the raw.xz image
> Uses the ImportVolume AWS API[1] to create a volume via S3
> Creates the snapshot using the earlier created volume
> Registers the AMI of the snapshot provided
> Copies the AMI to other regions and makes the AMI and the snapshot public.
>
> What if we migrate to Ore?
> The process remain more or less the same except for a few changes:
> Downloads the .qcow2 image
> Once we start using the .qcow2 image we don't need releng to
>
> produce two formats i.e. qcow2 & .raw.xz
> Converts the .qcow2 image to .vmdk format image using qemu-img
> Uses the newer supported AWS API i.e. ImportSnapshot which directly
>
> creates the Snapshot uploading the image via S3
>
> Registers the AMI with the snapshot.
>
> Copies the AMI to other regions.
>
> Ore also skips any of the steps already processed which makes it
>
> easier for the AMIs maintainer.
> Ore has a more maintainers than fedimg.
> It's better to focus on a single project than maintaining two.
>
> Two areas where it needs more research
> how do we send out the fedmsg messages when we move to ore
> making the images & the snapshots public.
>
> Thoughts?

Is there a good reason to migrate from fedimg to ore instead of
implementing what we "like" from ore in fedimg?

fedimg is already wired into Fedora Infrastructure, leverages
first-party API modules for various provider uploads (esp. AWS), and
is actually relatively painless to develop and use. The fact it uses
Python is also a huge advantage, since it's easy for people to pick up
and develop/contribute.

Like most things in Go, the development story is pretty awful, and
maintenance of Go software is... not great. Between the lack of
versioning and the complete lack of sane dependency management, I am
incredibly reluctant to encourage development of tools in Go. As
someone who actually _does_ maintain a package in Go and has to deal
with its idiosyncrasies on a regular basis, I'd rather not inflict
that horror on more people...

I do agree that we shouldn't have split maintenance between two tools,
though I'd rather see Fedora CoreOS images pushed via fedimg rather
than ore.

[CoreOS] Re: Migrating from fedimg to ore

[Dusty Mabe]

On 07/22/2018 04:06 PM, Neal Gompa wrote:
> Is there a good reason to migrate from fedimg to ore instead of
> implementing what we "like" from ore in fedimg?

Mostly the fact that with the acquisition of CoreOS Inc we have more
people that know and maintain ore than fedimg (where sayan was really
the only person who knows fedimg). ore also has the ability to upload
to many more clouds than just AWS, which I don't think is something we
ever had in fedimg.
> fedimg is already wired into Fedora Infrastructure, leverages
> first-party API modules for various provider uploads (esp. AWS), and
> is actually relatively painless to develop and use. The fact it uses
> Python is also a huge advantage, since it's easy for people to pick up
> and develop/contribute.
>
> Like most things in Go, the development story is pretty awful, and
> maintenance of Go software is... not great. Between the lack of
> versioning and the complete lack of sane dependency management, I am
> incredibly reluctant to encourage development of tools in Go. As
> someone who actually _does_ maintain a package in Go and has to deal
> with its idiosyncrasies on a regular basis, I'd rather not inflict
> that horror on more people...

I agree Go can be a pain. Despite that I think this is a good starting
point for now because we get the added functionality ore currently has 
without doing a bunch of new development. For now I think we have other
"bigger fish" to go after.
> I do agree that we shouldn't have split maintenance between two tools,
> though I'd rather see Fedora CoreOS images pushed via fedimg rather
> than ore.

[CoreOS] Re: Migrating from fedimg to ore

[Neal Gompa]

On Tue, Jul 24, 2018 at 10:50 AM Dusty Mabe  wrote:
> On 07/22/2018 04:06 PM, Neal Gompa wrote:
> > Is there a good reason to migrate from fedimg to ore instead of
> > implementing what we "like" from ore in fedimg?
>
> Mostly the fact that with the acquisition of CoreOS Inc we have more
> people that know and maintain ore than fedimg (where sayan was really
> the only person who knows fedimg). ore also has the ability to upload
> to many more clouds than just AWS, which I don't think is something we
> ever had in fedimg.

Well, Fedora itself doesn't really support cloud providers other than
AWS. For example, we don't have the required tooling for GCE packaged
in Fedora, so even if we wanted to make GCE images, they won't work. I
found this out while actually trying to get Fedora working on GCE for
Snappy's CI. I did make rudimentary packages for their tools to make
things work, but it's not in good enough shape to be in Fedora proper.

If we want to support more cloud providers, that's a different story,
and I doubt it'd be that difficult to add to fedimg.  If we were
interested in making official GCE images, I'd be happy to help with
fedimg and other things to get that going.

The lack of people working on things is a function of the lack of
priority on the cloud for Fedora. IMO, we've been chasing all this
Atomic stuff that most of the Cloud stuff has rotted. Do we care now?
> > fedimg is already wired into Fedora Infrastructure, leverages
> > first-party API modules for various provider uploads (esp. AWS), and
> > is actually relatively painless to develop and use. The fact it uses
> > Python is also a huge advantage, since it's easy for people to pick up
> > and develop/contribute.
> >
> > Like most things in Go, the development story is pretty awful, and
> > maintenance of Go software is... not great. Between the lack of
> > versioning and the complete lack of sane dependency management, I am
> > incredibly reluctant to encourage development of tools in Go. As
> > someone who actually _does_ maintain a package in Go and has to deal
> > with its idiosyncrasies on a regular basis, I'd rather not inflict
> > that horror on more people...
>
> I agree Go can be a pain. Despite that I think this is a good starting
> point for now because we get the added functionality ore currently has
> without doing a bunch of new development. For now I think we have other
> "bigger fish" to go after.

If we have bigger fish to go after, why are we bothering with this
conversation at all?

[CoreOS] Re: Migrating from fedimg to ore

[Dusty Mabe]

On 07/24/2018 10:57 AM, Neal Gompa wrote:
> On Tue, Jul 24, 2018 at 10:50 AM Dusty Mabe  wrote:
> > On 07/22/2018 04:06 PM, Neal Gompa wrote:
> > > Is there a good reason to migrate from fedimg to ore instead of
> > > implementing what we "like" from ore in fedimg?
> >
> > Mostly the fact that with the acquisition of CoreOS Inc we have more
> > people that know and maintain ore than fedimg (where sayan was really
> > the only person who knows fedimg). ore also has the ability to upload
> > to many more clouds than just AWS, which I don't think is something we
> > ever had in fedimg.
>
> Well, Fedora itself doesn't really support cloud providers other than
> AWS. For example, we don't have the required tooling for GCE packaged
> in Fedora, so even if we wanted to make GCE images, they won't work. I
> found this out while actually trying to get Fedora working on GCE for
> Snappy's CI. I did make rudimentary packages for their tools to make
> things work, but it's not in good enough shape to be in Fedora proper.
>
> If we want to support more cloud providers, that's a different story,
> and I doubt it'd be that difficult to add to fedimg.  If we were
> interested in making official GCE images, I'd be happy to help with
> fedimg and other things to get that going.

That's awesome! I know jdoss was interested in helping make sure we
get more regular releases of the cloud base image as well. See

https://lists.fedoraproject.org/archives/list/[e-mail]/...
> The lack of people working on things is a function of the lack of
> priority on the cloud for Fedora. IMO, we've been chasing all this
> Atomic stuff that most of the Cloud stuff has rotted. Do we care now?

As stated in the original FAQ, Fedora CoreOS will target many cloud 
providers. I'm hoping this means, by association, issues we had with
getting the Fedora Cloud base image into various clouds will no longer
be a problem.

https://discussion.fedoraproject.org/t/launch-faq-which-platforms-does-fedor...
> > > fedimg is already wired into Fedora Infrastructure, leverages
> > > first-party API modules for various provider uploads (esp. AWS), and
> > > is actually relatively painless to develop and use. The fact it uses
> > > Python is also a huge advantage, since it's easy for people to pick up
> > > and develop/contribute.
> > >
> > > Like most things in Go, the development story is pretty awful, and
> > > maintenance of Go software is... not great. Between the lack of
> > > versioning and the complete lack of sane dependency management, I am
> > > incredibly reluctant to encourage development of tools in Go. As
> > > someone who actually _does_ maintain a package in Go and has to deal
> > > with its idiosyncrasies on a regular basis, I'd rather not inflict
> > > that horror on more people...
> >
> > I agree Go can be a pain. Despite that I think this is a good starting
> > point for now because we get the added functionality ore currently has
> > without doing a bunch of new development. For now I think we have other
> > "bigger fish" to go after.
>
> If we have bigger fish to go after, why are we bothering with this
> conversation at all?

That's my opinion :) - IOW I'm +1 for ore because it has all the functionality
we already need and allows us to concentrate on other issues we have to knock
out for an initial release of FCOS. Are we causing ourselves issues by taking
on technical debt ? Maybe. I personally prefer dynamic/shared lib/interpreted
languages as well because of dep issues with Go, but i'm trying to be practical.

/me wishes we had a team of people for solving the packaging issues around Go/Rust

Dusty

[CoreOS] Re: Migrating from fedimg to ore

[Jim Perrin]

On 07/24/2018 08:15 AM, Dusty Mabe wrote:
> On 07/24/2018 10:57 AM, Neal Gompa wrote:
> > On Tue, Jul 24, 2018 at 10:50 AM Dusty Mabe  wrote:
> > > On 07/22/2018 04:06 PM, Neal Gompa wrote:
> > > > Is there a good reason to migrate from fedimg to ore instead of
> > > > implementing what we "like" from ore in fedimg?
> > >
> > > Mostly the fact that with the acquisition of CoreOS Inc we have more
> > > people that know and maintain ore than fedimg (where sayan was really
> > > the only person who knows fedimg). ore also has the ability to upload
> > > to many more clouds than just AWS, which I don't think is something we
> > > ever had in fedimg.
> >
> > Well, Fedora itself doesn't really support cloud providers other than
> > AWS. For example, we don't have the required tooling for GCE packaged
> > in Fedora, so even if we wanted to make GCE images, they won't work. I
> > found this out while actually trying to get Fedora working on GCE for
> > Snappy's CI. I did make rudimentary packages for their tools to make
> > things work, but it's not in good enough shape to be in Fedora proper.
> >
> > If we want to support more cloud providers, that's a different story,
> > and I doubt it'd be that difficult to add to fedimg.  If we were
> > interested in making official GCE images, I'd be happy to help with
> > fedimg and other things to get that going.
>
> That's awesome! I know jdoss was interested in helping make sure we
> get more regular releases of the cloud base image as well. See
>
> https://lists.fedoraproject.org/archives/list/[e-mail]/...
> > The lack of people working on things is a function of the lack of
> > priority on the cloud for Fedora. IMO, we've been chasing all this
> > Atomic stuff that most of the Cloud stuff has rotted. Do we care now?
>
> As stated in the original FAQ, Fedora CoreOS will target many cloud 
> providers. I'm hoping this means, by association, issues we had with
> getting the Fedora Cloud base image into various clouds will no longer
> be a problem.
>
> https://discussion.fedoraproject.org/t/launch-faq-which-platforms-does-fedor...

+1. I want to try to get Fedora into more providers, and this should
help accomplish doing so.
> > > > fedimg is already wired into Fedora Infrastructure, leverages
> > > > first-party API modules for various provider uploads (esp. AWS), and
> > > > is actually relatively painless to develop and use. The fact it uses
> > > > Python is also a huge advantage, since it's easy for people to pick up
> > > > and develop/contribute.
> > > >
> > > > Like most things in Go, the development story is pretty awful, and
> > > > maintenance of Go software is... not great. Between the lack of
> > > > versioning and the complete lack of sane dependency management, I am
> > > > incredibly reluctant to encourage development of tools in Go. As
> > > > someone who actually _does_ maintain a package in Go and has to deal
> > > > with its idiosyncrasies on a regular basis, I'd rather not inflict
> > > > that horror on more people...
> > >
> > > I agree Go can be a pain. Despite that I think this is a good starting
> > > point for now because we get the added functionality ore currently has
> > > without doing a bunch of new development. For now I think we have other
> > > "bigger fish" to go after.
> >
> > If we have bigger fish to go after, why are we bothering with this
> > conversation at all?
>
> That's my opinion :) - IOW I'm +1 for ore because it has all the functionality
> we already need and allows us to concentrate on other issues we have to knock
> out for an initial release of FCOS. Are we causing ourselves issues by taking
> on technical debt ? Maybe. I personally prefer dynamic/shared lib/interpreted
> languages as well because of dep issues with Go, but i'm trying to be practical.

We already have tech debt with fedimg. While it serves its current
purpose, there are some issues with the current iteration.

It's primarily single-commiter. With Ore, we can share development load
across 2 teams with more eyes, and more importantly fingers-on-keyboards.

It's using legacy amazon api calls in some places that cause issues.
Additionally there are some places where it shells out and exec's
binaries on the system rather than using python libs or methods.

It already requires retooling to support python3, in addition to the
above issues. I'd rather share that load with other teams and developers
rather than have one dev constantly recreate this particular wheel.
> /me wishes we had a team of people for solving the packaging issues around Go/Rust

[CoreOS] Re: Migrating from fedimg to ore

[Sayan Chowdhury]

On 24/07/18, Neal Gompa wrote:
> The lack of people working on things is a function of the lack of
> priority on the cloud for Fedora. IMO, we've been chasing all this
> Atomic stuff that most of the Cloud stuff has rotted. Do we care now?

Fedimg was built with the intention to support multiple providers[1] and due to
the same reason it was built using libcloud. But, sadly we never supported more
than AWS.

[1] https://github.com/fedora-infra/fedimg/issues/10

[CoreOS] Migrating from fedimg to ore

[Dusty Mabe]

On 07/20/2018 11:15 AM, Sayan Chowdhury wrote:
> Last few days, I spent doing a feasibility analysis on the migrating
> from fedimg to ore, a subproject of mantle to release the Fedora cloud
> images.

<snip>

Note to everyone who is not using "reply-all". This was sent to two
mailing lists. I'll link to the respective threads from both so anyone
who is interested in the conversation on the other can click:

CoreOS: https://lists.fedoraproject.org/archives/list/[e-mail]...
Infra:  https://lists.fedoraproject.org/archives/list/[e-mail]...