8:32 a.m.
Hey,
So I am trying to implement TOTP+password for SSH on a server. In the
past its been as simple as using google authenticatior but seeing as how
we have a shiny FreeIPA server...
Created a user, then gave them a TOTP token (synched and tested that it
works by logging into the web ui). But I'm stuck at the correct way to
implement this on the SSH server.
Found the earlier thread[1] and got some pointers.
sshd config:
ChallengeResponseAuthentication yes
AuthenticationMethods keyboard-interactive
If I do not define password/otp for the host via the IPA web interface,
login works fine with password. If I set it to password/otp only it fails.
Looking at journalctl -xeu ssh.service there clearly is some issue.
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): received for user kjell: 7 (Authentication failure)
error: PAM: Authentication failure for kjell from 192.168.31.102
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): received for user kjell: 4 (System error)
error: PAM: Authentication failure for kjell from 192.168.31.102
Postponed keyboard-interactive for kjell from 192.168.31.102 port 38832
ssh2 [preauth]
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): received for user kjell: 4 (System error)
error: PAM: Authentication failure for kjell from 192.168.31.102
Failed keyboard-interactive/pam for kjell from 192.168.31.102 port 38832
ssh2
Connection closed by authenticating user kjell 192.168.31.102 port 38832
[preauth]
Tried giving my password, and my password+otp (without the '+'). But
nothing works.
Anyone got any pointers or see any obvious mistakes ?
1:
https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorah...
--
Mvh,
Kjell C. Nicolaysen
PGP Public key available on request.
Current key (at time of this email) fingerprint:
3F59 7410 AFD5 FC22 F2F1 EEC9 980A 8C9E C126 6716
1:28 p.m.
On pe, 23 joulu 2022, Kjell Cornelius Nicolaysen via FreeIPA-users wrote:
Hey,
So I am trying to implement TOTP+password for SSH on a server. In the
past its been as simple as using google authenticatior but seeing as
how we have a shiny FreeIPA server...
Created a user, then gave them a TOTP token (synched and tested that
it works by logging into the web ui). But I'm stuck at the correct way
to implement this on the SSH server.
Found the earlier thread[1] and got some pointers.
sshd config:
ChallengeResponseAuthentication yes
AuthenticationMethods keyboard-interactive
If I do not define password/otp for the host via the IPA web
interface, login works fine with password. If I set it to password/otp
only it fails.
Looking at journalctl -xeu ssh.service there clearly is some issue.
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): received for user kjell: 7 (Authentication failure)
error: PAM: Authentication failure for kjell from 192.168.31.102
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): received for user kjell: 4 (System error)
error: PAM: Authentication failure for kjell from 192.168.31.102
Postponed keyboard-interactive for kjell from 192.168.31.102 port
38832 ssh2 [preauth]
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=192.168.31.102 user=kjell
pam_sss(sshd:auth): received for user kjell: 4 (System error)
error: PAM: Authentication failure for kjell from 192.168.31.102
Failed keyboard-interactive/pam for kjell from 192.168.31.102 port
38832 ssh2
Connection closed by authenticating user kjell 192.168.31.102 port
38832 [preauth]
Tried giving my password, and my password+otp (without the '+'). But
nothing works.
Anyone got any pointers or see any obvious mistakes ?
You get system error from pam_sss. You need to enable debug logging in
SSSD and collect logs. Please see
https://sssd.io/troubleshooting/basics.html for more details.
--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland
7:22 a.m.
So here is a twist on getting SSH/OTP to work. If I use the web UI and:
1) set user authentication types to "Two factor authentication (password
+ OTP), then set the host I am trying to login to to "password+OTP", it
works.
2) if the user has no authentication types hooked, or has Hardened
Password or Password in addition to the 2FA option hooked, login fails.
Had a look at SSSD logs (with debug level 7) but afraid I cannot spot
any clear issues save "pre authentication failed" if I have any of the
settings mentioned in point 2 above (have tried tracing that but cannot
for the life of me find the reason why).
All I am trying to do is require password+otp for the SSH portion. Sudo
should only require password, not password and otp...
Sorry, but very fresh to FreeIPA so I am certain there is some concept
at play here which I am just not seeing.
On 23/12/2022 20:28, Alexander Bokovoy wrote:
On pe, 23 joulu 2022, Kjell Cornelius Nicolaysen via FreeIPA-users
wrote:
> Hey,
>
>
> So I am trying to implement TOTP+password for SSH on a server. In
> the past its been as simple as using google authenticatior but seeing
> as how we have a shiny FreeIPA server...
>
>
> Created a user, then gave them a TOTP token (synched and tested that
> it works by logging into the web ui). But I'm stuck at the correct
> way to implement this on the SSH server.
> Found the earlier thread[1] and got some pointers.
> sshd config:
>
> ChallengeResponseAuthentication yes
> AuthenticationMethods keyboard-interactive
>
>
> If I do not define password/otp for the host via the IPA web
> interface, login works fine with password. If I set it to
> password/otp only it fails.
>
>
> Looking at journalctl -xeu ssh.service there clearly is some issue.
>
> pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
> tty=ssh ruser= rhost=192.168.31.102 user=kjell
> pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
> tty=ssh ruser= rhost=192.168.31.102 user=kjell
> pam_sss(sshd:auth): received for user kjell: 7 (Authentication failure)
> error: PAM: Authentication failure for kjell from 192.168.31.102
> pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
> tty=ssh ruser= rhost=192.168.31.102 user=kjell
> pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
> tty=ssh ruser= rhost=192.168.31.102 user=kjell
> pam_sss(sshd:auth): received for user kjell: 4 (System error)
> error: PAM: Authentication failure for kjell from 192.168.31.102
> Postponed keyboard-interactive for kjell from 192.168.31.102 port
> 38832 ssh2 [preauth]
> pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
> tty=ssh ruser= rhost=192.168.31.102 user=kjell
> pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
> tty=ssh ruser= rhost=192.168.31.102 user=kjell
> pam_sss(sshd:auth): received for user kjell: 4 (System error)
> error: PAM: Authentication failure for kjell from 192.168.31.102
> Failed keyboard-interactive/pam for kjell from 192.168.31.102 port
> 38832 ssh2
> Connection closed by authenticating user kjell 192.168.31.102 port
> 38832 [preauth]
>
>
> Tried giving my password, and my password+otp (without the '+'). But
> nothing works.
>
> Anyone got any pointers or see any obvious mistakes ?
You get system error from pam_sss. You need to enable debug logging in
SSSD and collect logs. Please see
https://sssd.io/troubleshooting/basics.html for more details.
--
Mvh,
Kjell C. Nicolaysen
Bitfrost AS
PGP Public key available on request.
Current key (at time of this email) fingerprint:
3F59 7410 AFD5 FC22 F2F1 EEC9 980A 8C9E C126 6716
8:45 a.m.
On ti, 27 joulu 2022, Kjell Cornelius Nicolaysen wrote:
So here is a twist on getting SSH/OTP to work. If I use the web UI
and:
1) set user authentication types to "Two factor authentication
(password + OTP), then set the host I am trying to login to to
"password+OTP", it works.
2) if the user has no authentication types hooked, or has Hardened
Password or Password in addition to the 2FA option hooked, login
fails.
This is as designed. If you have forced authentication indicator on the
host service to 'otp' then any Kerberos ticket presented to KDC when
asking for a ticket to that host/.. service must be obtained with 'otp'.
Please read
https://freeipa.readthedocs.io/en/latest/workshop/11-kerberos-ticket-poli...,
especially 'Enforcing authentication indicators' section but you need to
familiarize yourself with the other parts of the same unit as well.
Had a look at SSSD logs (with debug level 7) but afraid I cannot spot
any clear issues save "pre authentication failed" if I have any of the
settings mentioned in point 2 above (have tried tracing that but
cannot for the life of me find the reason why).
All I am trying to do is require password+otp for the SSH portion.
Sudo should only require password, not password and otp...
Sorry, but very fresh to FreeIPA so I am certain there is some concept
at play here which I am just not seeing.
This is not possible, in general. This is cannot be achieved without
maintaining an authentication session trail and that session trail is
not flexible enough to allow what you want. It is not a problem of
FreeIPA, it is a general complexity of application-specific solutions
that cannot really be solved here. We can do that in certain way with
Kerberos tickets but even there exists no way to enforce a specific
authentication process in case an otherwise valid Kerberos ticket is
unfit for a specific purpose.
You either have a ticket granting ticket (TGT) with certain properties
because you have obtained it somewhere using a particular
pre-authentication method or you don't have it and there is no logic
anywhere that would force you to loop back to initial authentication
step.
When using Kerberos, the process of authentication to a service really
is broken down into several independent authentication and authorization
steps:
- [a] obtain an initial ticket granting ticket, TGT
- [b] use TGT to request a service ticket to the specific Kerberos service
- [c] use application-specific protocol to present a service ticket to the
application's service to be authenticated
- [d] after authentication happened, application needs to authorize the
access
The important part here is that there are two independent Kerberos
operations where a client side communicates with KDC and KDC might apply
some access controls:
- request an initial TGT ([a])
- request a service ticket using that initial TGT ([b])
After these steps were done, an application client side does [c] and a
application server side does [d]. [d] is where another access control
step is typically applied.
An authentication indicator is a mark in the TGT that tells which method
was used to authenticate the issuance of the TGT (step [a]). If a
service ticket was issued by the KDC, that ticket will contain the same
authentication indicator as was in the TGT. Any new indicator cannot be
added at any step past [a].
A networking application which uses Kerberos typically does not deal
with step [a] directly. Most of them expect a TGT to exist in a
credentials cache on the client side. This ticket is used to obtain a
service ticket (step [b]) or, if such ticket exists in the credentials
cache, it will be reused. For all such applications (SSH is no
exception) if Kerberos ticket exists, they don't care about the
specifics of its properties other than whether the ticket is valid.
If you are not using Kerberos (GSSAPI) in the first place to login to
SSH server, a default SSH server configuration on IPA clients would
drive you through the SSH server's PAM stack to authenticate. The PAM
stack definition for SSH server typically includes pam_sss.so module.
You can think of it as a 'client app' in the description above now.
pam_sss.so module would ask SSSD backend to acquire a Kerberos ticket
for you. SSSD would ask for a password+otp if KDC advertises the 'otp'
pre-authentication method and Kerberos library on the system supports
'otp' method. pam_sss.so module would pass this request as a text
message to the application (in this case, internals of SSH server
process) and SSH would use its own protocol to tell to SSH client to
show that message to the user.
But if the Kerberos library does not support 'otp' method or KDC does
not advertise it, SSSD will happily use the password SSH has provided to
pam_sss.so as a part of that communication to obtain a 'normal' TGT,
e.g. the one that does not have 'otp' authentication indicator. If user
is allowed to use a plain password to authenticate, this request would
succeed.
The TGT obtained by SSSD will then be used to ask for a service ticket
to host/... principal to validate your ticket (regardless whether it
was obtained by SSSD itself or presented from SSH client). This is a
step [b]. If you set 'otp' on the host/... principal, KDC will reject
any request which provided an evidence of the TGT without 'otp'
authentication indicator. There is no way to fall back from here to
request a user to obtain a new TGT with other method. At least, I don't
know about any application that implements this using Kerberos.
On 23/12/2022 20:28, Alexander Bokovoy wrote:
>On pe, 23 joulu 2022, Kjell Cornelius Nicolaysen via FreeIPA-users wrote:
>>Hey,
>>
>>
>>So I am trying to implement TOTP+password for SSH on a server. In
>>the past its been as simple as using google authenticatior but
>>seeing as how we have a shiny FreeIPA server...
>>
>>
>>Created a user, then gave them a TOTP token (synched and tested
>>that it works by logging into the web ui). But I'm stuck at the
>>correct way to implement this on the SSH server.
>>Found the earlier thread[1] and got some pointers.
>>sshd config:
>>
>>ChallengeResponseAuthentication yes
>>AuthenticationMethods keyboard-interactive
>>
>>
>>If I do not define password/otp for the host via the IPA web
>>interface, login works fine with password. If I set it to
>>password/otp only it fails.
>>
>>
>>Looking at journalctl -xeu ssh.service there clearly is some issue.
>>
>>pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
>>tty=ssh ruser= rhost=192.168.31.102 user=kjell
>>pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
>>tty=ssh ruser= rhost=192.168.31.102 user=kjell
>>pam_sss(sshd:auth): received for user kjell: 7 (Authentication failure)
>>error: PAM: Authentication failure for kjell from 192.168.31.102
>>pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
>>tty=ssh ruser= rhost=192.168.31.102 user=kjell
>>pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
>>tty=ssh ruser= rhost=192.168.31.102 user=kjell
>>pam_sss(sshd:auth): received for user kjell: 4 (System error)
>>error: PAM: Authentication failure for kjell from 192.168.31.102
>>Postponed keyboard-interactive for kjell from 192.168.31.102 port
>>38832 ssh2 [preauth]
>>pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0
>>tty=ssh ruser= rhost=192.168.31.102 user=kjell
>>pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0
>>tty=ssh ruser= rhost=192.168.31.102 user=kjell
>>pam_sss(sshd:auth): received for user kjell: 4 (System error)
>>error: PAM: Authentication failure for kjell from 192.168.31.102
>>Failed keyboard-interactive/pam for kjell from 192.168.31.102 port
>>38832 ssh2
>>Connection closed by authenticating user kjell 192.168.31.102 port
>>38832 [preauth]
>>
>>
>>Tried giving my password, and my password+otp (without the '+').
>>But nothing works.
>>
>>Anyone got any pointers or see any obvious mistakes ?
>
>You get system error from pam_sss. You need to enable debug logging in
>SSSD and collect logs. Please see
>https://sssd.io/troubleshooting/basics.html for more details.
>
>
--
Mvh,
Kjell C. Nicolaysen
Bitfrost AS
PGP Public key available on request.
Current key (at time of this email) fingerprint:
3F59 7410 AFD5 FC22 F2F1 EEC9 980A 8C9E C126 6716
--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland
10:34 a.m.
Kjell,
I have a working solution using Kerberos TGTs to ssh between servers within the realm
using OTP. This solution requires that you log into one realm server with password and
otp. From there you can ssh to the rest of the realm via kerberos tickets.
I only got to this state by trial and error so some of my statements and assumptions could
be wrong. My servers are a mix of Rhel 7.9 and Rhel8.x My IDM servers are Rhel8 latest.
On the ssh server you need to enable GSSAPIAuthentication in sshd_config.
On all the clients you also need to enable GSSAPIAuthentication in ssh_config,
There are some gotchas.
You can't ssh to the ssh server by IP address. GSSAPIAuthentication passes the
request to IDM by hostname. So ssh user(a)192.168.1.101 will fail back to password via
pam.sss
On Rhel7 you can only ssh to the FQDN of the ssh server. So you should do: ssh
user(a)myserver.realm.com .
If the Client is Rhel8 you should be able to ssh to the short hostname such as ssh
user@myserver .
On Rhel7 clients getting the ssh to work with short hostnames is problematic. You can
resolve the problem with the following settings in ssh_config:
CanonicalizeHostname always
CanonicalDomains myrealm.com
CanonicalizeFallbackLocal yes
CanonicalizeMaxDots 0
GSSAPITrustDns yes
Host *.myrealm.com
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
Host ! *.myrealm.com
GSSAPIAuthentication no
Your experience with users that don't have an OTP being unable to log into a host that
requires OTP is expected.
There are plenty of edge cases around caching, so test before production.
It is also problematic to get a new ticket with kinit if you use OTP. There are steps to
get around it, but I am waiting for the solution to be a native Redhat process.
We regularly get this TGT ssh method to fail, when there are too may expired tickets on
the ssh server. Once logged into the ssh server by password OtP, you can run kdestroy -A
to clear cache and you can again ssh via TGTs.
Obviously ssh and sshd should be configured to failback to username/password+otp with:
ChallengeResponseAuthentication yes
468
days inactive
496
days old
freeipa-users@lists.fedorahosted.org
4 comments
3 participants
participants (3)
-
Alexander Bokovoy -
Bob Strachan -
Kjell Cornelius Nicolaysen