Packaging the AWS SSM Agent

[Major Hayden]

Hey there,

I've been working on packaging the AWS SSM Agent[0] in Fedora on and off for a few
months now and I finally got a free moment to work on it again. However, there are a few
problems that have me scratching my head a little.

Just to level set, AWS Systems Manager[1] is an offering that allows AWS customers to
manage their instances outside the instance itself. We all know (and love) cloud-init, but
it really only handles the first boot provisioning of the instance. The SSM agent has
similarities to Azure's WALinuxAgent as it allows for management *after* the first
boot.

(For reference, WALinuxAgent is already packaged in Fedora and automatically enabled when
Fedora lands in an Azure instance.)

The main issue is around dependencies:

----------
# No commits since Mar 2021 💀
'golang(github.com/Workiva/go-datastructures)'

# Should be covered by the existing aws-sdk-go package 🤔
'golang(github.com/aws/aws-sdk-go/service/backupstorage)'
'golang(github.com/aws/aws-sdk-go/service/privatenetworks)'
'golang(github.com/aws/aws-sdk-go/service/ssm/ssmiface/mocks)'
'golang(github.com/aws/aws-sdk-go/service/ssmmds)'
'golang(github.com/aws/aws-sdk-go/service/ssmmds/ssmmdsiface)'
'golang(github.com/aws/aws-sdk-go/service/ssmmds/ssmmdsiface/mocks)'

# No commits since Jan 2017 💀
'golang(github.com/carlescere/scheduler)'

# No commits since Jan 2017 💀
'golang(github.com/cihub/seelog)'

# No commits since Sep 2018 💀
'golang(github.com/digitalocean/go-smbios/smbios)'

# No commits since Sep 2016 💀
'golang(github.com/pborman/ansi)'

# Last commit Feb 2023 👍
'golang(github.com/xtaci/smux)'
----------

Some of these have not been updated in quite some time and many of them have several
unresolved bugs that are years old. Although I'd like to get the SSM Agent packaged in
Fedora, I don't like the idea of packaging quite a few packages which aren't being
actively maintained.

One potential option is to work with upstream (AWS) to change these dependencies out for
actively maintained ones instead, but that likely requires significant development work.
🥵

Another option might be to vendor these dependencies in the main SSM package, but that
still means we're building against unmaintained code and it likely violates some
Fedora policies. 😱

How should I proceed? Thanks for reading this far?

[0] https://github.com/aws/amazon-ssm-agent
[1]
https://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-syst...

--
Major Hayden

[Mikel Olasagasti]

Hi Major,

Hau idatzi du Major Hayden (major(a)mhtx.net) erabiltzaileak (2023 mai.
8(a), al. (22:51)):
>  Hey there,
>
>  I've been working on packaging the AWS SSM Agent[0] in Fedora on and off for a few
> months now and I finally got a free moment to work on it again. However, there are a few
> problems that have me scratching my head a little.
>
>  Just to level set, AWS Systems Manager[1] is an offering that allows AWS customers to
> manage their instances outside the instance itself. We all know (and love) cloud-init, but
> it really only handles the first boot provisioning of the instance. The SSM agent has
> similarities to Azure's WALinuxAgent as it allows for management *after* the first
> boot.
>
>  (For reference, WALinuxAgent is already packaged in Fedora and automatically enabled when
> Fedora lands in an Azure instance.)
>
>  The main issue is around dependencies:
>
>  ----------
>  # No commits since Mar 2021 💀
>  'golang(github.com/Workiva/go-datastructures)'
>
>  # Should be covered by the existing aws-sdk-go package 🤔
>  'golang(github.com/aws/aws-sdk-go/service/backupstorage)'
>  'golang(github.com/aws/aws-sdk-go/service/privatenetworks)'
>  'golang(github.com/aws/aws-sdk-go/service/ssm/ssmiface/mocks)'
>  'golang(github.com/aws/aws-sdk-go/service/ssmmds)'
>  'golang(github.com/aws/aws-sdk-go/service/ssmmds/ssmmdsiface)'
>  'golang(github.com/aws/aws-sdk-go/service/ssmmds/ssmmdsiface/mocks)'
>
>  # No commits since Jan 2017 💀
>  'golang(github.com/carlescere/scheduler)'
>
>  # No commits since Jan 2017 💀
>  'golang(github.com/cihub/seelog)'
>
>  # No commits since Sep 2018 💀
>  'golang(github.com/digitalocean/go-smbios/smbios)'
>
>  # No commits since Sep 2016 💀
>  'golang(github.com/pborman/ansi)'
>
>  # Last commit Feb 2023 👍
>  'golang(github.com/xtaci/smux)'
>  ----------
>
>  Some of these have not been updated in quite some time and many of them have several
> unresolved bugs that are years old. Although I'd like to get the SSM Agent packaged in
> Fedora, I don't like the idea of packaging quite a few packages which aren't being
> actively maintained.
>
>  One potential option is to work with upstream (AWS) to change these dependencies out for
> actively maintained ones instead, but that likely requires significant development work.
> 🥵
>
>  Another option might be to vendor these dependencies in the main SSM package, but that
> still means we're building against unmaintained code and it likely violates some
> Fedora policies. 😱
>
>  How should I proceed? 
My option would be to package those "dead" deps.

The impact for Fedora is very limited, as only SSM Agent require those
packages and, hopefully, no other packages will require them. If in
the future builds start to fail, upstream should be also affected
sooner or later, so upstream should be interested on fixing.

IMO the major (no pun intended) problem would be for you, as it can be
challenging to keep the dep stack up to date.

Kind regards,
Mikel

[Smith, Stewart]

>  On May 9, 2023, at 1:53 PM, Mikel Olasagasti
> <mikel(a)olasagasti.info&gt; wrote:
>  
>  Hi Major,
>  
>  Hau idatzi du Major Hayden (major(a)mhtx.net) erabiltzaileak (2023 mai.
>  8(a), al. (22:51)):
> > 
> > One potential option is to work with upstream (AWS) to change these dependencies out
> for actively maintained ones instead, but that likely requires significant development
> work. 🥵
>  
I’m poking the SSM team internally on this to see if there’s any plans of getting away
from any of these (and to force the evaluation of if there *should* be any plans or
not).

> > Another option might be to vendor these dependencies in the main
> SSM package, but that still means we're building against unmaintained code and it
> likely violates some Fedora policies. 😱
>  
We vendor the dependencies for SSM in the SSM package we ship in Amazon Linux for two
reasons:
1. it was always done that way (not a good reason)
2. avoids exposing any of these deps to a broader audience than “just the scope in which
the SSM agent uses them” and thus alleviates a pile of issues regarding moving them
forward major versions or not.

Seeing as it’s net new to Fedora, (1) doesn’t apply, and (2) is less of an issue because
of the shorter life cycle of Fedora.

[Smith, Stewart]

On May 11, 2023, at 7:14 AM, Smith, Stewart <trawets(a)amazon.com&gt; wrote:
>  
> > On May 9, 2023, at 1:53 PM, Mikel Olasagasti <mikel(a)olasagasti.info&gt; wrote:
> > 
> > Hi Major,
> > 
> > Hau idatzi du Major Hayden (major(a)mhtx.net) erabiltzaileak (2023 mai.
> > 8(a), al. (22:51)):
> >> 
> >> One potential option is to work with upstream (AWS) to change these dependencies
> out for actively maintained ones instead, but that likely requires significant development
> work. 🥵
>  
>  I’m poking the SSM team internally on this to see if there’s any plans of getting away
> from any of these (and to force the evaluation of if there *should* be any plans or not).
>  

Just updating on this effort: I’m poking again, possibly harder this time.

[Smith, Stewart]

On May 9, 2023, at 1:53 PM, Mikel Olasagasti <mikel(a)olasagasti.info&gt; wrote:
>  Hau idatzi du Major Hayden (major(a)mhtx.net) erabiltzaileak (2023
> mai.
>  8(a), al. (22:51)):
> > 
> > Hey there,
> > 
> > I've been working on packaging the AWS SSM Agent[0] in Fedora on and off for a
> few months now and I finally got a free moment to work on it again. However, there are a
> few problems that have me scratching my head a little.
> > 
> > Just to level set, AWS Systems Manager[1] is an offering that allows AWS customers to
> manage their instances outside the instance itself. We all know (and love) cloud-init, but
> it really only handles the first boot provisioning of the instance. The SSM agent has
> similarities to Azure's WALinuxAgent as it allows for management *after* the first
> boot.
> > 
> > (For reference, WALinuxAgent is already packaged in Fedora and automatically enabled
> when Fedora lands in an Azure instance.)
> > 
> > The main issue is around dependencies:
> > 
> > ----------
> > # No commits since Mar 2021 💀
> > 'golang(github.com/Workiva/go-datastructures)'
>  
This appears to have had some commits in mid-May 2023!

Perhaps it’s getting better and doesn’t want to get on the cart?

[Smith, Stewart]

On May 9, 2023, at 1:53 PM, Mikel Olasagasti <mikel(a)olasagasti.info&gt; wrote:

Hau idatzi du Major Hayden (major(a)mhtx.net) erabiltzaileak (2023 mai.
8(a), al. (22:51)):

# No commits since Sep 2018 💀
'golang(github.com/digitalocean/go-smbios/smbios)'

URL correction: https://github.com/digitalocean/go-smbios

but no more commits there.