Disabling ssh hostkey generation/caching

[Dominik Vogt]

I have a problem with ssh host keys being automatically
generated(?).  Our installation process looks like this:

1) Install IdM server and clients.
 2) Generate ssh host keys for all machines in a special way
    because of a requirement about available entropy that is not
 3) Distribute the new keys to all machines but do not install
    them.  (Any required known_hosts files are also generated
    and distributed). and known_hosts files
 4) Reboot all machines at the same time.
 5) During reboot, a script installs the new keys and deletes the
    old ones (rsa and ecdsa format).

However, host key files in rsa and ecdsa format keep reappearing.
I'm not exactly sure when this happens.  Does it have something to
do with sssd?

--

So, we need to disable

* automatic generation of ssh keys,
 * restoring them from backups (in case some component does that).

Caching the keys in sssd would be in order if we can make sure
that sssd does not cache the old keys at any time.  Running
"sss_cache -H" does not seem to affect the cached known_hosts file
in /var/lib though.

I understand that IdM can manage host keys, but we don't want to
use that feature because of the underlying system requirements,
and we'll never add new machines anyway.

Ciao

Dominik ^_^  ^_^

--

Dominik Vogt

[Rob Crittenden]

Dominik Vogt via FreeIPA-users wrote:
> I have a problem with ssh host keys being automatically
> generated(?).  Our installation process looks like this:
> Install IdM server and clients.
> Generate ssh host keys for all machines in a special way
> because of a requirement about available entropy that is not
> Distribute the new keys to all machines but do not install
> them.  (Any required known_hosts files are also generated
> and distributed). and known_hosts files
> Reboot all machines at the same time.
> During reboot, a script installs the new keys and deletes the
> old ones (rsa and ecdsa format).
>
> However, host key files in rsa and ecdsa format keep reappearing.
> I'm not exactly sure when this happens.  Does it have something to
> do with sssd?

I believe sshd generates keys on startup if they do not exist.

You probably want to include the --no-dns-sshfp option for
ipa-client-install to prevent any existing SSH keys from appearing in DNS.

ipa-client-install will add any keys to the host entry at the time of
enrollment.
> --
>
> So, we need to disable
> automatic generation of ssh keys,
> restoring them from backups (in case some component does that).
>
> Caching the keys in sssd would be in order if we can make sure
> that sssd does not cache the old keys at any time.  Running
> "sss_cache -H" does not seem to affect the cached known_hosts file
> in /var/lib though.
>
> I understand that IdM can manage host keys, but we don't want to
> use that feature because of the underlying system requirements,
> and we'll never add new machines anyway.

I don't believe there is an easy way to prevent ssh keys from getting
into IPA. There is an ACI that grants a host write access to their own
ssh pub keys but there is no equivalent permission for it. You could,
for example, try tweaking the write to read and see if that helps. It'll
prevent keys from getting into IPA so that sssd can never find them.

My spidey sense isn't tingling with this suggestion but that doesn't
mean there isn't the possibility of a side-effort or future upgrade issues.

rob

[Dominik Vogt]

> Dominik Vogt via FreeIPA-users wrote:
> > However, host key files in rsa and ecdsa format keep reappearing.
> > I'm not exactly sure when this happens.  Does it have something to
> > do with sssd?
>
> I believe sshd generates keys on startup if they do not exist.

I'll check that.  There was some "AUTOGENERATE..." setting for ssh
in RHEL 5 and 6 in the system defaults, but I couldn't find
anything like that on the installed machines.
> You probably want to include the --no-dns-sshfp option for
> ipa-client-install to prevent any existing SSH keys from appearing in DNS.
>
> ipa-client-install will add any keys to the host entry at the time of
> enrollment.
> > I understand that IdM can manage host keys, but we don't want to
> > use that feature because of the underlying system requirements,
> > and we'll never add new machines anyway.
>
> I don't believe there is an easy way to prevent ssh keys from getting
> into IPA. There is an ACI that grants a host write access to their own
> ssh pub keys but there is no equivalent permission for it. You could,
> for example, try tweaking the write to read and see if that helps. It'll
> prevent keys from getting into IPA so that sssd can never find them.

It would be good enough to make sure these keys are not used and
don't interfere with the keys that were installed manually.

At the moment there's a real problem though because sssd keeps
caching keys in the wrong format in /var/lib/.../known_hosts, and
ssh then complains that this copy is out of date.
> My spidey sense isn't tingling with this suggestion but that doesn't
> mean there isn't the possibility of a side-effort or future upgrade issues.

Ciao

Dominik ^_^  ^_^

--

Dominik Vogt

[Dominik Vogt]

On Wed, Sep 15, 2021 at 10:57:55AM -0400, Rob Crittenden via FreeIPA-users wrote:
> Dominik Vogt via FreeIPA-users wrote:
> > However, host key files in rsa and ecdsa format keep reappearing.
> > I'm not exactly sure when this happens.  Does it have something to
> > do with sssd?
>
> I believe sshd generates keys on startup if they do not exist.

For the records, I've fixed the problem with

$ systemctl mask [e-mail]
  $ systemctl mask [e-mail]
  $ systemctl mask [e-mail]
> You probably want to include the --no-dns-sshfp option for
> ipa-client-install to prevent any existing SSH keys from appearing in DNS.

Yes.
> > Caching the keys in sssd would be in order if we can make sure
> > that sssd does not cache the old keys at any time.  Running
> > "sss_cache -H" does not seem to affect the cached known_hosts file
> > in /var/lib though.

We now remove /var/lib/sss/.../known_hosts at startup.

Our ssh connection problems because of old keys in the sss cache
are gone, and no keys are being generated when sshd starts up.

Thanks for the help!

Dominik ^_^  ^_^

--

Dominik Vogt