using SSH with password authentication when NIS is still running with FreeIPA

[Robert Kudyba]

We have freeipa-server-4.8.10-6.fc33 running on top of NIS and I'm trying
to determine why ssh -k from any client is hanging and not even connecting.
Does sssd need to be configured as in this 2013 training document?
https://www.freeipa.org/images/1/10/Freeipa30_SSSD_OpenSSH_integration.pdf

The goal is to eliminate NIS so perhaps the issue is running both
concurrently? The good news is, thanks to tips here last week, all the NIS
users migrated along with their passwords. And kinit on the Free IPA server
even prompts to change their password.

sssd is running:
sssd_be[2329]: GSSAPI client step 1
sssd_be[2329]: GSSAPI client step 2

/etc/krb.conf
includedir /etc/krb5.conf.d/
includedir /var/lib/sss/pubconf/krb5.include.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = ourserver.EDU
 dns_lookup_realm = true
 dns_lookup_kdc = true
 rdns = false
 ticket_lifetime = 24h
 forwardable = true
 udp_preference_limit = 0
 default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 ourserver.EDU = {
  kdc = ourserver.edu:88
  master_kdc = ourserver.edu:88
  admin_server = ourserver.edu:749
  default_domain = ourserver.edu
  pkinit_anchors = FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem
  pkinit_pool = FILE:/var/lib/ipa-client/pki/ca-bundle.pem
}

[domain_realm]
 .ourserver.edu = ourserver.EDU
 ourserver.edu = ourserver.EDU
 ourserver.edu = ourserver.EDU

[dbmodules]
  ourserver.EDU = {
    db_library = ipadb.so
  }

[plugins]
 certauth = {
  module = ipakdb:kdb/ipadb.so
  enable_only = ipakdb
 }

 HBAC is wide open:

ipa hbacrule-find
--------------------
2 HBAC rules matched
--------------------
  Rule name: allow_all
  User category: all
  Host category: all
  Service category: all
  Description: Allow all users to access any host from any host
  Enabled: TRUE

  Rule name: allow_systemd-user
  User category: all
  Host category: all
  Description: Allow pam_systemd to run user@.service to create a system
user session
  Enabled: TRUE

Here are some debug ssh server logs:
Feb  8 16:23:27 ourserver sshd[381563]: debug1: Forked child 510395.
Feb  8 16:23:27 ourserver sshd[510395]: debug1: Set
/proc/self/oom_score_adj to 0
Feb  8 16:23:27 ourserver sshd[510395]: debug1: rexec start in 5 out 5
newsock 5 pipe 10 sock 11
Feb  8 16:23:27 ourserver sshd[510395]: debug1: inetd sockets after
dupping: 4, 4
Feb  8 16:23:27 ourserver sshd[510395]: Connection from 150.108.68.26 port
45806 on 150.108.64.156 port 22 rdomain ""
Feb  8 16:23:27 ourserver sshd[510395]: debug1: Local version string
SSH-2.0-OpenSSH_8.4
Feb  8 16:23:27 ourserver sshd[510395]: debug1: Remote protocol version
2.0, remote software version OpenSSH_8.4
Feb  8 16:23:27 ourserver sshd[510395]: debug1: match: OpenSSH_8.4 pat
OpenSSH* compat 0x04000000
Feb  8 16:23:27 ourserver sshd[510395]: debug1: SELinux support disabled
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: permanently_set_uid: 74/74
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: list_hostkey_types:
rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_KEXINIT sent
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_KEXINIT received
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: algorithm:
curve25519-sha256 [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: host key algorithm:
ecdsa-sha2-nistp256 [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: client->server cipher:
aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: server->client cipher:
aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: expecting
SSH2_MSG_KEX_ECDH_INIT [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: rekey out after 4294967296
blocks [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_NEWKEYS sent
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: Sending SSH2_MSG_EXT_INFO
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: expecting SSH2_MSG_NEWKEYS
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_NEWKEYS received
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: rekey in after 4294967296
blocks [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: KEX done [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: userauth-request for user
ouruser service ssh-connection method none [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: attempt 0 failures 0
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: PAM: initializing for
"ouruser"
Feb  8 16:23:27 ourserver sshd[510395]: debug1: PAM: setting PAM_RHOST to
"xx.xx.xx.xx"
Feb  8 16:23:27 ourserver sshd[510395]: debug1: PAM: setting PAM_TTY to
"ssh"
Feb  8 16:23:27 ourserver sshd[510395]: debug1: userauth-request for user
ouruser service ssh-connection method publickey [preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: attempt 1 failures 0
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: userauth_pubkey: test pkalg
rsa-sha2-256 pkblob RSA SHA256:++6z7fhR603SUI0fWp7k7noRz1/41+9/hM/rWjNQlXk
[preauth]
Feb  8 16:23:27 ourserver sshd[510395]: debug1: temporarily_use_uid:
5879/200 (e=0/0)
Feb  8 16:23:27 ourserver sshd[510395]: debug1: trying public key file
/home/ouruser/.ssh/authorized_keys

and ssh -k from a Fedora client, note the user I'm logged in as is NOT the
same user I'm trying to log in to:
ssh -vv -k ouruser@ourserver
OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS  8 Dec 2020
debug1: Reading configuration data /home/ouruser/.ssh/config
debug1: /home/ouruser/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host ourserver originally ourserver
debug2: match not found
debug1: Reading configuration data
/etc/crypto-policies/back-ends/openssh.config
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /home/ouruser/.ssh/config
debug1: /home/ouruser/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host ourserver originally ourserver
debug2: match found
debug1: Reading configuration data
/etc/crypto-policies/back-ends/openssh.config
debug1: auto-mux: Trying existing master
debug1: Control socket "/home/ouruser/.ssh/sockets/ouruser@ourserver-22"
does not exist
debug2: resolving "ourserver" port 22
debug2: ssh_connect_direct
debug1: Connecting to ourserver [150.108.64.156] port 22.
debug1: Connection established.
debug1: identity file /home/ouruser/.ssh/id_rsa type 0
debug1: identity file /home/ouruser/.ssh/id_rsa-cert type -1
debug1: identity file /home/ouruser/.ssh/id_dsa type -1
debug1: identity file /home/ouruser/.ssh/id_dsa-cert type -1
debug1: identity file /home/ouruser/.ssh/id_ecdsa type -1
debug1: identity file /home/ouruser/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/ouruser/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/ouruser/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/ouruser/.ssh/id_ed25519 type 3
debug1: identity file /home/ouruser/.ssh/id_ed25519-cert type -1
debug1: identity file /home/ouruser/.ssh/id_ed25519_sk type -1
debug1: identity file /home/ouruser/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/ouruser/.ssh/id_xmss type -1
debug1: identity file /home/ouruser/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4
debug1: match: OpenSSH_8.4 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to ourserver:22 as 'ouruser'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01(a)openssh.com,
ecdsa-sha2-nistp384-cert-v01(a)openssh.com,
ecdsa-sha2-nistp521-cert-v01(a)openssh.com,
[e-mail],ssh-ed25519-cert-v01(a)openssh.com
,[e-mail],rsa-sha2-512-cert-v01(a)openssh.com,
[e-mail],ssh-rsa-cert-v01(a)openssh.com
,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
[e-mail],ssh-ed25519,sk-ssh-ed25519(a)openssh.com
,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [e-mail],chacha20-poly1305(a)openssh.com
,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
debug2: ciphers stoc: [e-mail],chacha20-poly1305(a)openssh.com
,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
debug2: MACs ctos: [e-mail],hmac-sha1-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com
,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
debug2: MACs stoc: [e-mail],hmac-sha1-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com
,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib(a)openssh.com,zlib
debug2: compression stoc: none,zlib(a)openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
debug2: host key algorithms:
rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305(a)openssh.com
,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
aes256-gcm(a)openssh.com
debug2: ciphers stoc: chacha20-poly1305(a)openssh.com
,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
aes256-gcm(a)openssh.com
debug2: MACs ctos: [e-mail],umac-128-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com,
[e-mail],[e-mail],umac-128(a)openssh.com
,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [e-mail],umac-128-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com,
[e-mail],[e-mail],umac-128(a)openssh.com
,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib(a)openssh.com
debug2: compression stoc: none,zlib(a)openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm(a)openssh.com MAC: <implicit>
compression: none
debug1: kex: client->server cipher: aes256-gcm(a)openssh.com MAC: <implicit>
compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256
SHA256:XUXhRKNYwxAGhwVIMa3fuo8uNMay6q4/qVeSWlQAOpM
debug1: Host 'ourserver' is known and matches the ECDSA host key.
debug1: Found key in /home/ouruser/.ssh/known_hosts:46
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /home/ouruser/.ssh/id_rsa RSA
SHA256:++6z7fhR603SUI0fWp7k7noRz1/41+9/hM/rWjNQlXk
debug1: Will attempt key: /home/ouruser/.ssh/id_dsa
debug1: Will attempt key: /home/ouruser/.ssh/id_ecdsa
debug1: Will attempt key: /home/ouruser/.ssh/id_ecdsa_sk
debug1: Will attempt key: /home/ouruser/.ssh/id_ed25519 ED25519
SHA256:OoedE0VhmLFtl9nifW57Mca+GHDD0xKkJ2BCLGlV9xc
debug1: Will attempt key: /home/ouruser/.ssh/id_ed25519_sk
debug1: Will attempt key: /home/ouruser/.ssh/id_xmss
debug2: pubkey_prepare: done
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,
sk-ssh-ed25519(a)openssh.com
,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
sk-ecdsa-sha2-nistp256(a)openssh.com,
webauthn-sk-ecdsa-sha2-nistp256(a)openssh.com&gt;
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)
debug2: we did not send a packet, disable method
debug1: Next authentication method: publickey
debug1: Offering public key: /home/ouruser/.ssh/id_rsa RSA
SHA256:++6z7fhR603SUI0fWp7k7noRz1/41+9/hM/rWjNQlXk
debug2: we sent a publickey packet, wait for reply

What am I missing? I appreciate the help last week!

Rob

[Sumit Bose]

On Mon, Feb 08, 2021 at 04:42:31PM -0500, Robert Kudyba via FreeIPA-users wrote:
>  We have freeipa-server-4.8.10-6.fc33 running on top of NIS and
> I'm trying
>  to determine why ssh -k from any client is hanging and not even connecting.
>  Does sssd need to be configured as in this 2013 training document?
>  https://www.freeipa.org/images/1/10/Freeipa30_SSSD_OpenSSH_integration.pdf
>  
>  The goal is to eliminate NIS so perhaps the issue is running both
>  concurrently? The good news is, thanks to tips here last week, all the NIS
>  users migrated along with their passwords. And kinit on the Free IPA server
>  even prompts to change their password.
>  
>  sssd is running:
>  sssd_be[2329]: GSSAPI client step 1
>  sssd_be[2329]: GSSAPI client step 2
>  
>  /etc/krb.conf
>  includedir /etc/krb5.conf.d/
>  includedir /var/lib/sss/pubconf/krb5.include.d/
>  
>  [logging]
>   default = FILE:/var/log/krb5libs.log
>   kdc = FILE:/var/log/krb5kdc.log
>   admin_server = FILE:/var/log/kadmind.log
>  
>  [libdefaults]
>   default_realm = ourserver.EDU
>   dns_lookup_realm = true
>   dns_lookup_kdc = true
>   rdns = false
>   ticket_lifetime = 24h
>   forwardable = true
>   udp_preference_limit = 0
>   default_ccache_name = KEYRING:persistent:%{uid}
>  
>  [realms]
>   ourserver.EDU = {
>    kdc = ourserver.edu:88
>    master_kdc = ourserver.edu:88
>    admin_server = ourserver.edu:749
>    default_domain = ourserver.edu
>    pkinit_anchors = FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem
>    pkinit_pool = FILE:/var/lib/ipa-client/pki/ca-bundle.pem
>  }
>  
>  [domain_realm]
>   .ourserver.edu = ourserver.EDU
>   ourserver.edu = ourserver.EDU
>   ourserver.edu = ourserver.EDU
>  
>  [dbmodules]
>    ourserver.EDU = {
>      db_library = ipadb.so
>    }
>  
>  [plugins]
>   certauth = {
>    module = ipakdb:kdb/ipadb.so
>    enable_only = ipakdb
>   }
>  
>   HBAC is wide open:
>  
>  ipa hbacrule-find
>  --------------------
>  2 HBAC rules matched
>  --------------------
>    Rule name: allow_all
>    User category: all
>    Host category: all
>    Service category: all
>    Description: Allow all users to access any host from any host
>    Enabled: TRUE
>  
>    Rule name: allow_systemd-user
>    User category: all
>    Host category: all
>    Description: Allow pam_systemd to run user@.service to create a system
>  user session
>    Enabled: TRUE
>  
>  Here are some debug ssh server logs:
>  Feb  8 16:23:27 ourserver sshd[381563]: debug1: Forked child 510395.
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: Set
>  /proc/self/oom_score_adj to 0
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: rexec start in 5 out 5
>  newsock 5 pipe 10 sock 11
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: inetd sockets after
>  dupping: 4, 4
>  Feb  8 16:23:27 ourserver sshd[510395]: Connection from 150.108.68.26 port
>  45806 on 150.108.64.156 port 22 rdomain ""
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: Local version string
>  SSH-2.0-OpenSSH_8.4
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: Remote protocol version
>  2.0, remote software version OpenSSH_8.4
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: match: OpenSSH_8.4 pat
>  OpenSSH* compat 0x04000000
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: SELinux support disabled
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: permanently_set_uid: 74/74
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: list_hostkey_types:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_KEXINIT sent
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_KEXINIT received
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: algorithm:
>  curve25519-sha256 [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: host key algorithm:
>  ecdsa-sha2-nistp256 [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: client->server cipher:
>  aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: server->client cipher:
>  aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: expecting
>  SSH2_MSG_KEX_ECDH_INIT [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: rekey out after 4294967296
>  blocks [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_NEWKEYS sent
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: Sending SSH2_MSG_EXT_INFO
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: expecting SSH2_MSG_NEWKEYS
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: SSH2_MSG_NEWKEYS received
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: rekey in after 4294967296
>  blocks [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: KEX done [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: userauth-request for user
>  ouruser service ssh-connection method none [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: PAM: initializing for
>  "ouruser"
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: PAM: setting PAM_RHOST to
>  "xx.xx.xx.xx"
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: userauth-request for user
>  ouruser service ssh-connection method publickey [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: userauth_pubkey: test pkalg
>  rsa-sha2-256 pkblob RSA SHA256:++6z7fhR603SUI0fWp7k7noRz1/41+9/hM/rWjNQlXk
>  [preauth]
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: temporarily_use_uid:
>  5879/200 (e=0/0)
>  Feb  8 16:23:27 ourserver sshd[510395]: debug1: trying public key file
>  /home/ouruser/.ssh/authorized_keys 
Hi,

looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys and
is stuck. Can you read this file from the command line? Is it e.g. on
NFS which might not be properly mounted?

Does it work if you skip pubkey authentication

    ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver

bye,
Sumit
>  
>  and ssh -k from a Fedora client, note the user I'm logged in as is NOT the
>  same user I'm trying to log in to:
>  ssh -vv -k ouruser@ourserver
>  OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS  8 Dec 2020
>  debug1: Reading configuration data /home/ouruser/.ssh/config
>  debug1: /home/ouruser/.ssh/config line 1: Applying options for *
>  debug1: Reading configuration data /etc/ssh/ssh_config
>  debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
>  debug2: checking match for 'final all' host ourserver originally ourserver
>  debug2: match not found
>  debug1: Reading configuration data
>  /etc/crypto-policies/back-ends/openssh.config
>  debug1: configuration requests final Match pass
>  debug1: re-parsing configuration
>  debug1: Reading configuration data /home/ouruser/.ssh/config
>  debug1: /home/ouruser/.ssh/config line 1: Applying options for *
>  debug1: Reading configuration data /etc/ssh/ssh_config
>  debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
>  debug2: checking match for 'final all' host ourserver originally ourserver
>  debug2: match found
>  debug1: Reading configuration data
>  /etc/crypto-policies/back-ends/openssh.config
>  debug1: auto-mux: Trying existing master
>  debug1: Control socket "/home/ouruser/.ssh/sockets/ouruser@ourserver-22"
>  does not exist
>  debug2: resolving "ourserver" port 22
>  debug2: ssh_connect_direct
>  debug1: Connecting to ourserver [150.108.64.156] port 22.
>  debug1: Connection established.
>  debug1: identity file /home/ouruser/.ssh/id_rsa type 0
>  debug1: identity file /home/ouruser/.ssh/id_rsa-cert type -1
>  debug1: identity file /home/ouruser/.ssh/id_dsa type -1
>  debug1: identity file /home/ouruser/.ssh/id_dsa-cert type -1
>  debug1: identity file /home/ouruser/.ssh/id_ecdsa type -1
>  debug1: identity file /home/ouruser/.ssh/id_ecdsa-cert type -1
>  debug1: identity file /home/ouruser/.ssh/id_ecdsa_sk type -1
>  debug1: identity file /home/ouruser/.ssh/id_ecdsa_sk-cert type -1
>  debug1: identity file /home/ouruser/.ssh/id_ed25519 type 3
>  debug1: identity file /home/ouruser/.ssh/id_ed25519-cert type -1
>  debug1: identity file /home/ouruser/.ssh/id_ed25519_sk type -1
>  debug1: identity file /home/ouruser/.ssh/id_ed25519_sk-cert type -1
>  debug1: identity file /home/ouruser/.ssh/id_xmss type -1
>  debug1: identity file /home/ouruser/.ssh/id_xmss-cert type -1
>  debug1: Local version string SSH-2.0-OpenSSH_8.4
>  debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4
>  debug1: match: OpenSSH_8.4 pat OpenSSH* compat 0x04000000
>  debug2: fd 3 setting O_NONBLOCK
>  debug1: Authenticating to ourserver:22 as 'ouruser'
>  debug1: SSH2_MSG_KEXINIT sent
>  debug1: SSH2_MSG_KEXINIT received
>  debug2: local client KEXINIT proposal
>  debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
>
> ,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c
>  debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01(a)openssh.com,
>  ecdsa-sha2-nistp384-cert-v01(a)openssh.com,
>  ecdsa-sha2-nistp521-cert-v01(a)openssh.com,
>  [e-mail],ssh-ed25519-cert-v01(a)openssh.com
>  ,[e-mail],rsa-sha2-512-cert-v01(a)openssh.com,
>  [e-mail],ssh-rsa-cert-v01(a)openssh.com
>  ,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
>  [e-mail],ssh-ed25519,sk-ssh-ed25519(a)openssh.com
>  ,rsa-sha2-512,rsa-sha2-256,ssh-rsa
>  debug2: ciphers ctos: [e-mail],chacha20-poly1305(a)openssh.com
>  ,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
>  debug2: ciphers stoc: [e-mail],chacha20-poly1305(a)openssh.com
>  ,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
>  debug2: MACs ctos: [e-mail],hmac-sha1-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com
>  ,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
>  debug2: MACs stoc: [e-mail],hmac-sha1-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com
>  ,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
>  debug2: compression ctos: none,zlib(a)openssh.com,zlib
>  debug2: compression stoc: none,zlib(a)openssh.com,zlib
>  debug2: languages ctos:
>  debug2: languages stoc:
>  debug2: first_kex_follows 0
>  debug2: reserved 0
>  debug2: peer server KEXINIT proposal
>  debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
>
> ,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
>  debug2: host key algorithms:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  debug2: ciphers ctos: chacha20-poly1305(a)openssh.com
>  ,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
>  aes256-gcm(a)openssh.com
>  debug2: ciphers stoc: chacha20-poly1305(a)openssh.com
>  ,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
>  aes256-gcm(a)openssh.com
>  debug2: MACs ctos: [e-mail],umac-128-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com,
>  [e-mail],[e-mail],umac-128(a)openssh.com
>  ,hmac-sha2-256,hmac-sha2-512,hmac-sha1
>  debug2: MACs stoc: [e-mail],umac-128-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com,
>  [e-mail],[e-mail],umac-128(a)openssh.com
>  ,hmac-sha2-256,hmac-sha2-512,hmac-sha1
>  debug2: compression ctos: none,zlib(a)openssh.com
>  debug2: compression stoc: none,zlib(a)openssh.com
>  debug2: languages ctos:
>  debug2: languages stoc:
>  debug2: first_kex_follows 0
>  debug2: reserved 0
>  debug1: kex: algorithm: curve25519-sha256
>  debug1: kex: host key algorithm: ecdsa-sha2-nistp256
>  debug1: kex: server->client cipher: aes256-gcm(a)openssh.com MAC: <implicit>
>  compression: none
>  debug1: kex: client->server cipher: aes256-gcm(a)openssh.com MAC: <implicit>
>  compression: none
>  debug1: kex: curve25519-sha256 need=32 dh_need=32
>  debug1: kex: curve25519-sha256 need=32 dh_need=32
>  debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
>  debug1: Server host key: ecdsa-sha2-nistp256
>  SHA256:XUXhRKNYwxAGhwVIMa3fuo8uNMay6q4/qVeSWlQAOpM
>  debug1: Host 'ourserver' is known and matches the ECDSA host key.
>  debug1: Found key in /home/ouruser/.ssh/known_hosts:46
>  debug2: set_newkeys: mode 1
>  debug1: rekey out after 4294967296 blocks
>  debug1: SSH2_MSG_NEWKEYS sent
>  debug1: expecting SSH2_MSG_NEWKEYS
>  debug1: SSH2_MSG_NEWKEYS received
>  debug2: set_newkeys: mode 0
>  debug1: rekey in after 4294967296 blocks
>  debug1: Will attempt key: /home/ouruser/.ssh/id_rsa RSA
>  SHA256:++6z7fhR603SUI0fWp7k7noRz1/41+9/hM/rWjNQlXk
>  debug1: Will attempt key: /home/ouruser/.ssh/id_dsa
>  debug1: Will attempt key: /home/ouruser/.ssh/id_ecdsa
>  debug1: Will attempt key: /home/ouruser/.ssh/id_ecdsa_sk
>  debug1: Will attempt key: /home/ouruser/.ssh/id_ed25519 ED25519
>  SHA256:OoedE0VhmLFtl9nifW57Mca+GHDD0xKkJ2BCLGlV9xc
>  debug1: Will attempt key: /home/ouruser/.ssh/id_ed25519_sk
>  debug1: Will attempt key: /home/ouruser/.ssh/id_xmss
>  debug2: pubkey_prepare: done
>  debug1: SSH2_MSG_EXT_INFO received
>  debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,
>  sk-ssh-ed25519(a)openssh.com
>
> ,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
>  sk-ecdsa-sha2-nistp256(a)openssh.com,
>  webauthn-sk-ecdsa-sha2-nistp256(a)openssh.com&gt;
>  debug2: service_accept: ssh-userauth
>  debug1: SSH2_MSG_SERVICE_ACCEPT received
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug1: Next authentication method: gssapi-with-mic
>  debug1: Unspecified GSS failure.  Minor code may provide more information
>  No Kerberos credentials available (default cache: KCM:)
>  debug1: Unspecified GSS failure.  Minor code may provide more information
>  No Kerberos credentials available (default cache: KCM:)
>  debug2: we did not send a packet, disable method
>  debug1: Next authentication method: publickey
>  debug1: Offering public key: /home/ouruser/.ssh/id_rsa RSA
>  SHA256:++6z7fhR603SUI0fWp7k7noRz1/41+9/hM/rWjNQlXk
>  debug2: we sent a publickey packet, wait for reply
>  
>  What am I missing? I appreciate the help last week!
>  
>  Rob 
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...

[Robert Kudyba]

>  looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys and
>  is stuck. Can you read this file from the command line? Is it e.g. on
>  NFS which might not be properly mounted?
>
>  Does it work if you skip pubkey authentication
>
>      ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
>
>  bye,
>  Sumit
>  
Thanks for the suggestion. What happens is the NIS password works. The
FreeIPA password, which I update with:
ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the below
errors/logs

Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
/proc/self/oom_score_adj to 0
Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
newsock 5 pipe 7 sock 8
Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
dupping: 4, 4
Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332
on 150.108.64.156 port 22 rdomain ""
Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
SSH-2.0-OpenSSH_8.4
Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
2.0, remote software version OpenSSH_8.4
Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
OpenSSH* compat 0x04000000
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
curve25519-sha256 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
ecdsa-sha2-nistp256 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher:
aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher:
aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
SSH2_MSG_KEX_ECDH_INIT [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296
blocks [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296
blocks [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
ouruser service ssh-connection method none [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
"ouruser"
Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to
"x.x.x.x"
Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
"ssh"
Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs
 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 53332 ssh2 [preauth]
Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
user ouruser: 9 (Authentication service cannot retrieve authentication info)
Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure
for ouruser from x.x.x.x
Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for
ouruser from x.x.x.x port 53332 ssh2
Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
[preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs
 [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 53332 ssh2 [preauth]


Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
/proc/self/oom_score_adj to 0
Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
newsock 5 pipe 7 sock 8
Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
dupping: 4, 4
Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332
on 150.108.64.156 port 22 rdomain ""
Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
SSH-2.0-OpenSSH_8.4
Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
2.0, remote software version OpenSSH_8.4
Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
OpenSSH* compat 0x04000000
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
curve25519-sha256 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
ecdsa-sha2-nistp256 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher:
aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher:
aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
SSH2_MSG_KEX_ECDH_INIT [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296
blocks [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296
blocks [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
ouruser service ssh-connection method none [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
"ouruser"
Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to
"x.x.x.x"
Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
"ssh"
Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs
 [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 53332 ssh2 [preauth]
Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
user ouruser: 9 (Authentication service cannot retrieve authentication info)
Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure
for ouruser from x.x.x.x
Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for
ouruser from x.x.x.x port 53332 ssh2
Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
[preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs
 [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 53332 ssh2 [preauth]

With the NIS password the logs show this:
Feb  9 11:16:57 debug1: do_pam_account: called
Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env strings 2
Feb  9 11:16:57 ourserver sshd[536226]: Postponed keyboard-interactive/pam
for cai from 150.108.68.26 port 53646 ssh2 [preauth]
Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called
Feb  9 11:16:57 ourserver sshd[536226]: Accepted keyboard-interactive/pam
for cai from 150.108.68.26 port 53646 ssh2
Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth: cai
has been authenticated by privileged process
Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child log
fd closed
Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled
event 2
Feb  9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid:
5879/200 (e=0/0)
Feb  9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds: Not
a GSSAPI mechanism
Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled
Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
credentials
Feb  9 11:16:57 ourserver systemd[536237]: pam_unix(systemd-user:session):
session opened for user cai(uid=5879) by (uid=0)

What options should be set in /etc/ssh/sshd_config? Is sssd necessary for
this to work with the FreeIPA password?

[Sumit Bose]

On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via FreeIPA-users wrote:
>  >
>  > looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys and
>  > is stuck. Can you read this file from the command line? Is it e.g. on
>  > NFS which might not be properly mounted?
>  >
>  > Does it work if you skip pubkey authentication
>  >
>  >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
>  >
>  > bye,
>  > Sumit
>  >
>  
>  Thanks for the suggestion. What happens is the NIS password works. The
>  FreeIPA password, which I update with:
>  ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the below
>  errors/logs
>  
>  Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  /proc/self/oom_score_adj to 0
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
>  newsock 5 pipe 7 sock 8
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  dupping: 4, 4
>  Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332
>  on 150.108.64.156 port 22 rdomain ""
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  SSH-2.0-OpenSSH_8.4
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
>  2.0, remote software version OpenSSH_8.4
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
>  OpenSSH* compat 0x04000000
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  curve25519-sha256 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
>  ecdsa-sha2-nistp256 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher:
>  aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher:
>  aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  SSH2_MSG_KEX_ECDH_INIT [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296
>  blocks [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296
>  blocks [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  ouruser service ssh-connection method none [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
>  "ouruser"
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to
>  "x.x.x.x"
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
>  Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
>  user ouruser: 9 (Authentication service cannot retrieve authentication info)
>  Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure
>  for ouruser from x.x.x.x
>  Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for
>  ouruser from x.x.x.x port 53332 ssh2
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
>  [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  
>  
>  Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  /proc/self/oom_score_adj to 0
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
>  newsock 5 pipe 7 sock 8
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  dupping: 4, 4
>  Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332
>  on 150.108.64.156 port 22 rdomain ""
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  SSH-2.0-OpenSSH_8.4
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
>  2.0, remote software version OpenSSH_8.4
>  Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
>  OpenSSH* compat 0x04000000
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  curve25519-sha256 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
>  ecdsa-sha2-nistp256 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher:
>  aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher:
>  aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  SSH2_MSG_KEX_ECDH_INIT [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296
>  blocks [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296
>  blocks [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  ouruser service ssh-connection method none [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
>  "ouruser"
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to
>  "x.x.x.x"
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
>  [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
>  Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
>  user ouruser: 9 (Authentication service cannot retrieve authentication info)
>  Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure
>  for ouruser from x.x.x.x
>  Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for
>  ouruser from x.x.x.x port 53332 ssh2
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam'
>  [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  
>  With the NIS password the logs show this: 
Hi,

did you drop what happened before or is this the only debug output for
the NIS password?

>  Feb  9 11:16:57 debug1: do_pam_account: called
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env strings 2
>  Feb  9 11:16:57 ourserver sshd[536226]: Postponed keyboard-interactive/pam
>  for cai from 150.108.68.26 port 53646 ssh2 [preauth]
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called
>  Feb  9 11:16:57 ourserver sshd[536226]: Accepted keyboard-interactive/pam
>  for cai from 150.108.68.26 port 53646 ssh2
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth: cai
>  has been authenticated by privileged process
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child log
>  fd closed
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled
>  event 2
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid:
>  5879/200 (e=0/0)
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds: Not
>  a GSSAPI mechanism
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled
>  Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
>  credentials
>  Feb  9 11:16:57 ourserver systemd[536237]: pam_unix(systemd-user:session):
>  session opened for user cai(uid=5879) by (uid=0)
>  
>  What options should be set in /etc/ssh/sshd_config? Is sssd necessary for
>  this to work with the FreeIPA password? 
Yes, SSSD must be configured and runnnig.

bye,
Sumit

>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...

[Robert Kudyba]

On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org&gt; wrote:

>  On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via
> FreeIPA-users
>  wrote:
>  > >
>  > > looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys
>  and
>  > > is stuck. Can you read this file from the command line? Is it e.g. on
>  > > NFS which might not be properly mounted?
>  > >
>  > > Does it work if you skip pubkey authentication
>  > >
>  > >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
>  > >
>  > > bye,
>  > > Sumit
>  > >
>  >
>  > Thanks for the suggestion. What happens is the NIS password works. The
>  > FreeIPA password, which I update with:
>  > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the below
>  > errors/logs
>  >
>  > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  > /proc/self/oom_score_adj to 0
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
>  > newsock 5 pipe 7 sock 8
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  > dupping: 4, 4
>  > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
>  53332
>  > on 150.108.64.156 port 22 rdomain ""
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  > SSH-2.0-OpenSSH_8.4
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
>  > 2.0, remote software version OpenSSH_8.4
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
>  > OpenSSH* compat 0x04000000
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
>  74/74
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  > curve25519-sha256 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
>  > ecdsa-sha2-nistp256 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server
>  cipher:
>  > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client
>  cipher:
>  > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  > need=32 dh_need=32 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  > need=32 dh_need=32 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
>  4294967296
>  > blocks [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  SSH2_MSG_NEWKEYS
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296
>  > blocks [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  > ouruser service ssh-connection method none [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
>  > "ouruser"
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to
>  > "x.x.x.x"
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
>  > "ssh"
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs
>  >  [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  > user=ouruser devs= [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  'pam'
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  > trying authentication method 'pam' [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive
>  for
>  > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  authentication
>  > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
>  > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  authentication
>  > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
>  > user ouruser: 9 (Authentication service cannot retrieve authentication
>  info)
>  > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
>  failure
>  > for ouruser from x.x.x.x
>  > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam
>  for
>  > ouruser from x.x.x.x port 53332 ssh2
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user
>  > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  > [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs
>  >  [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  > user=ouruser devs= [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  'pam'
>  > [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  > trying authentication method 'pam' [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive
>  for
>  > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >
>  >
>  > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  > /proc/self/oom_score_adj to 0
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
>  > newsock 5 pipe 7 sock 8
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  > dupping: 4, 4
>  > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
>  53332
>  > on 150.108.64.156 port 22 rdomain ""
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  > SSH-2.0-OpenSSH_8.4
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
>  > 2.0, remote software version OpenSSH_8.4
>  > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
>  > OpenSSH* compat 0x04000000
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
>  74/74
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  > curve25519-sha256 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
>  > ecdsa-sha2-nistp256 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server
>  cipher:
>  > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client
>  cipher:
>  > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  > need=32 dh_need=32 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  > need=32 dh_need=32 [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
>  4294967296
>  > blocks [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  SSH2_MSG_NEWKEYS
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296
>  > blocks [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  > ouruser service ssh-connection method none [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
>  > "ouruser"
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to
>  > "x.x.x.x"
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
>  > "ssh"
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user
>  > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs
>  >  [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  > user=ouruser devs= [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  'pam'
>  > [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  > trying authentication method 'pam' [preauth]
>  > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive
>  for
>  > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  authentication
>  > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
>  > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  authentication
>  > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
>  > user ouruser: 9 (Authentication service cannot retrieve authentication
>  info)
>  > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
>  failure
>  > for ouruser from x.x.x.x
>  > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam
>  for
>  > ouruser from x.x.x.x port 53332 ssh2
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user
>  > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  > [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs
>  >  [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  > user=ouruser devs= [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  'pam'
>  > [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  > trying authentication method 'pam' [preauth]
>  > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive
>  for
>  > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >
>  > With the NIS password the logs show this:
>
>  Hi,
>
>  did you drop what happened before or is this the only debug output for
>  the NIS password?
>  
The below here are just logs from /var/log/secure for the user that
successfully logs in with his/her NIS password.

By "drop what happened before" do you mean the original log snip? Yes I
removed those in an attempt to shorten the message content.

>  Feb  9 11:16:57 debug1: do_pam_account: called
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env strings
>  2
>  > Feb  9 11:16:57 ourserver sshd[536226]: Postponed
>  keyboard-interactive/pam
>  > for cai from 150.108.68.26 port 53646 ssh2 [preauth]
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called
>  > Feb  9 11:16:57 ourserver sshd[536226]: Accepted keyboard-interactive/pam
>  > for cai from 150.108.68.26 port 53646 ssh2
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth:
>  cai
>  > has been authenticated by privileged process
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child
>  log
>  > fd closed
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled
>  > event 2
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid:
>  > 5879/200 (e=0/0)
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds:
>  Not
>  > a GSSAPI mechanism
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled
>  > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
>  > credentials
>  > Feb  9 11:16:57 ourserver systemd[536237]:
>  pam_unix(systemd-user:session):
>  > session opened for user cai(uid=5879) by (uid=0)
>  >
>  > What options should be set in /etc/ssh/sshd_config? Is sssd necessary for
>  > this to work with the FreeIPA password
>  

Yes, SSSD must be configured and runnnig. ssd does appear to be working
fine and in /etc/ipa/ca.crt and the service is running correctly:

[domain/ourdomain.edu]

id_provider = ipa
ipa_server_mode = True
ipa_server = ourdomain.edu
ipa_domain = ourdomain.edu
ipa_hostname = ourdomain.edu
auth_provider = ipa
chpass_provider = ipa
access_provider = ipa
cache_credentials = True
ldap_tls_cacert = /etc/ipa/ca.crt
krb5_store_password_if_offline = True
[sssd]
services = nss, pam, ifp, ssh, sudo

domains = ourdomain.edu
[nss]
homedir_substring = /home
memcache_timeout = 600

[ifp]
allowed_uids = ipaapi, root

systemctl status sssd
* sssd.service - System Security Services Daemon
     Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor
preset: enabled)
     Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1 weeks 3
days ago

[Robert Kudyba]

I tried this on another test server, and configured NIS for the users,
which are different. Same issue. All the verbose output adds a lot of log
noise but I'm hoping it provides a clue.

ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful

here is ssh -o PubkeyAuthentication=no -vvv -k ouruser@ourserver
OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS  8 Dec 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file
/etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host ourserver originally ourserver
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file
/etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data
/etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok:
[gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org
,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file
/etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host ourserver originally ourserver
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file
/etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data
/etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok:
[gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org
,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' ->
'/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' ->
'/root/.ssh/known_hosts2'
debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p
22 ourserver
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4
debug1: match: OpenSSH_8.4 pat OpenSSH* compat 0x04000000
debug2: fd 5 setting O_NONBLOCK
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to ourserver:22 as 'ouruser'
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file
/root/.ssh/known_hosts:31
debug3: load_hostkeys: loaded 1 keys from ourserver
debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
debug3: order_hostkeyalgs: have matching best-preference key type
ecdsa-sha2-nistp256-cert-v01(a)openssh.com, using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01(a)openssh.com,
ecdsa-sha2-nistp384-cert-v01(a)openssh.com,
ecdsa-sha2-nistp521-cert-v01(a)openssh.com,
[e-mail],ssh-ed25519-cert-v01(a)openssh.com
,[e-mail],rsa-sha2-512-cert-v01(a)openssh.com,
[e-mail],ssh-rsa-cert-v01(a)openssh.com
,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
[e-mail],ssh-ed25519,sk-ssh-ed25519(a)openssh.com
,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [e-mail],chacha20-poly1305(a)openssh.com
,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
debug2: ciphers stoc: [e-mail],chacha20-poly1305(a)openssh.com
,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
debug2: MACs ctos: [e-mail],hmac-sha1-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com
,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
debug2: MACs stoc: [e-mail],hmac-sha1-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com
,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib(a)openssh.com,zlib
debug2: compression stoc: none,zlib(a)openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
debug2: host key algorithms:
rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305(a)openssh.com
,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
aes256-gcm(a)openssh.com
debug2: ciphers stoc: chacha20-poly1305(a)openssh.com
,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
aes256-gcm(a)openssh.com
debug2: MACs ctos: [e-mail],umac-128-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com,
[e-mail],[e-mail],umac-128(a)openssh.com
,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [e-mail],umac-128-etm(a)openssh.com,
[e-mail],hmac-sha2-512-etm(a)openssh.com,
[e-mail],[e-mail],umac-128(a)openssh.com
,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib(a)openssh.com
debug2: compression stoc: none,zlib(a)openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm(a)openssh.com MAC: <implicit>
compression: none
debug1: kex: client->server cipher: aes256-gcm(a)openssh.com MAC: <implicit>
compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256
SHA256:yVm8V2ODZo0nAuvr9k2ydTJv0RtOgkl8Sp5Mkmp/F0M
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file
/root/.ssh/known_hosts:31
debug3: load_hostkeys: loaded 1 keys from ourserver
debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
debug1: Host 'ourserver' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:31
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /root/.ssh/id_rsa RSA
SHA256:Sena4XB1wVt7x+o55Y9EI5WnQIyZ/SLFk+t6tmBFMYA
debug1: Will attempt key: /root/.ssh/id_dsa
debug1: Will attempt key: /root/.ssh/id_ecdsa
debug1: Will attempt key: /root/.ssh/id_ecdsa_sk
debug1: Will attempt key: /root/.ssh/id_ed25519
debug1: Will attempt key: /root/.ssh/id_ed25519_sk
debug1: Will attempt key: /root/.ssh/id_xmss
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,
sk-ssh-ed25519(a)openssh.com
,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
sk-ecdsa-sha2-nistp256(a)openssh.com,
webauthn-sk-ecdsa-sha2-nistp256(a)openssh.com&gt;
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: start over, passed a different list
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: preferred gssapi-with-mic,keyboard-interactive,password
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Server host/ourserver.edu(a)ourserver.edu not found in Kerberos database
debug3: send packet: type 50
debug2: we sent a gssapi-with-mic packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug3: send packet: type 50
debug2: we sent a keyboard-interactive packet, wait for reply
debug3: receive packet: type 60
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1

And here are the server logs from /var/log secure and you can see sssd is
being used:
Feb 10 14:36:24 ourserver sshd[3024290]: debug1: Forked child 3084339.
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Set
/proc/self/oom_score_adj to 0
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rexec start in 4 out 4
newsock 4 pipe 6 sock 7
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: inetd sockets after
dupping: 4, 4
Feb 10 14:36:24 ourserver sshd[3084339]: Connection from x.x.x.x port 34160
on 150.108.68.128 port 22 rdomain ""
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Local version string
SSH-2.0-OpenSSH_8.4
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Remote protocol version
2.0, remote software version OpenSSH_8.4
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: match: OpenSSH_8.4 pat
OpenSSH* compat 0x04000000
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SELinux support disabled
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: permanently_set_uid: 74/74
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: list_hostkey_types:
rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT sent
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT received
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: algorithm:
curve25519-sha256 [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: host key algorithm:
ecdsa-sha2-nistp256 [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: client->server
cipher: aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: server->client
cipher: aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256
need=32 dh_need=32 [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting
SSH2_MSG_KEX_ECDH_INIT [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey out after 4294967296
blocks [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS sent
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Sending SSH2_MSG_EXT_INFO
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting SSH2_MSG_NEWKEYS
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS received
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey in after 4294967296
blocks [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: KEX done [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user
ouruser service ssh-connection method none [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 0 failures 0
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: connection from x.x.x.x
matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: initializing for
"ouruser"
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_RHOST to
"x.x.x.x"
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_TTY to
"ssh"
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user
ouruser service ssh-connection method gssapi-with-mic [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 1 failures 0
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 2 failures 0
[preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: keyboard-interactive devs
 [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
'pam' [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb 10 14:36:24 ourserver sshd[3084339]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 34160 ssh2 [preauth]
Feb 10 14:36:28 ourserver sshd[3084344]: pam_unix(sshd:auth):
authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
 user=ouruser
Feb 10 14:36:28 ourserver proxy_child: pam_unix(sssd-shadowutils:auth):
authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
 user=ouruser
Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received for
user ouruser: 7 (Authentication failure)
Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication failure
for ouruser from x.x.x.x
Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam
for ouruser from x.x.x.x port 34160 ssh2
Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1
[preauth]
Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive devs
 [preauth]
Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
'pam' [preauth]
Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 34160 ssh2 [preauth]

I verified the FreeIPA password in both the GUI and via ipa user-mod. The
only time the user is able to log in is using the NIS password. ldapsearch
-x -D and kinit username work successfully. klist displays the user details
correctly.

I can see that the installation script edits /etc/ssh/sshd_config with:
Include /etc/ssh/sshd_config.d/04-ipa.conf

which has:
PubkeyAuthentication yes
KerberosAuthentication no
GSSAPIAuthentication yes
UsePAM yes
ChallengeResponseAuthentication yes
AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
AuthorizedKeysCommandUser nobody

When the NIS password is used successfully here are the server logs:
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user
ouruser service ssh-connection method none [preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0
[preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x
matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for
"ouruser"
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST to
"x.x.x.x"
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to
"ssh"
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user
ouruser service ssh-connection method gssapi-with-mic [preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0
[preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0
[preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive devs
 [preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices
'pam' [preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 35046 ssh2 [preauth]
Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called
Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env strings 2
Feb 10 14:56:42 ourserver sshd[3085147]: Postponed keyboard-interactive/pam
for ouruser from x.x.x.x port 35046 ssh2 [preauth]
Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called

I do see the error that sticks out is " Server host/
ourserver.edu(a)ourserver.edu not found in Kerberos database" but we have
students that log in from all over the world so do all clients need to be
added? iptables, firewalld, and nftables are off and disabled. No hbac
rules:
ipa hbacrule-find
--------------------
2 HBAC rules matched
--------------------
  Rule name: allow_all
  User category: all
  Host category: all
  Service category: all
  Description: Allow all users to access any host from any host
  Enabled: TRUE

  Rule name: allow_systemd-user
  User category: all
  Host category: all
  Description: Allow pam_systemd to run user@.service to create a system
user session
  Enabled: TRUE
----------------------------
Number of entries returned 2

Am I missing something obvious to regulars?


On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:

>  On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users <
>  freeipa-users(a)lists.fedorahosted.org&gt; wrote:
>
> > On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via FreeIPA-users
> > wrote:
> > > >
> > > > looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys
> > and
> > > > is stuck. Can you read this file from the command line? Is it e.g. on
> > > > NFS which might not be properly mounted?
> > > >
> > > > Does it work if you skip pubkey authentication
> > > >
> > > >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
> > > >
> > > > bye,
> > > > Sumit
> > > >
> > >
> > > Thanks for the suggestion. What happens is the NIS password works. The
> > > FreeIPA password, which I update with:
> > > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the
> below
> > > errors/logs
> > >
> > > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
> > > /proc/self/oom_score_adj to 0
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
> > > newsock 5 pipe 7 sock 8
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
> > > dupping: 4, 4
> > > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
> > 53332
> > > on 150.108.64.156 port 22 rdomain ""
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
> > > SSH-2.0-OpenSSH_8.4
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
> > > 2.0, remote software version OpenSSH_8.4
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
> > > OpenSSH* compat 0x04000000
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
> > 74/74
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
> > > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
> > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
> > received
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
> > > curve25519-sha256 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
> > > ecdsa-sha2-nistp256 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server
> > cipher:
> > > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client
> > cipher:
> > > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
> > > need=32 dh_need=32 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
> > > need=32 dh_need=32 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > > SSH2_MSG_KEX_ECDH_INIT [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
> > 4294967296
> > > blocks [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
> > SSH2_MSG_EXT_INFO
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > SSH2_MSG_NEWKEYS
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
> > received
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
> > 4294967296
> > > blocks [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method none [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
> > > "ouruser"
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST
> > to
> > > "x.x.x.x"
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
> > > "ssh"
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method keyboard-interactive [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive
> > devs
> > >  [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
> > > user=ouruser devs= [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices
> > 'pam'
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
> > > trying authentication method 'pam' [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive
> > for
> > > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
> > authentication
> > > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> > user=ouruser
> > > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
> > authentication
> > > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
> > > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
> > > user ouruser: 9 (Authentication service cannot retrieve authentication
> > info)
> > > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
> > failure
> > > for ouruser from x.x.x.x
> > > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam
> > for
> > > ouruser from x.x.x.x port 53332 ssh2
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method keyboard-interactive [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
> > > [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive
> > devs
> > >  [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
> > > user=ouruser devs= [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices
> > 'pam'
> > > [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
> > > trying authentication method 'pam' [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive
> > for
> > > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > >
> > >
> > > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
> > > /proc/self/oom_score_adj to 0
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
> > > newsock 5 pipe 7 sock 8
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
> > > dupping: 4, 4
> > > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
> > 53332
> > > on 150.108.64.156 port 22 rdomain ""
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
> > > SSH-2.0-OpenSSH_8.4
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
> > > 2.0, remote software version OpenSSH_8.4
> > > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
> > > OpenSSH* compat 0x04000000
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
> > 74/74
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
> > > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
> > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
> > received
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
> > > curve25519-sha256 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
> > > ecdsa-sha2-nistp256 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server
> > cipher:
> > > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client
> > cipher:
> > > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
> > > need=32 dh_need=32 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
> > > need=32 dh_need=32 [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > > SSH2_MSG_KEX_ECDH_INIT [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
> > 4294967296
> > > blocks [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
> > SSH2_MSG_EXT_INFO
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > SSH2_MSG_NEWKEYS
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
> > received
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
> > 4294967296
> > > blocks [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method none [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
> > > "ouruser"
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST
> > to
> > > "x.x.x.x"
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
> > > "ssh"
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method keyboard-interactive [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive
> > devs
> > >  [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
> > > user=ouruser devs= [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices
> > 'pam'
> > > [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
> > > trying authentication method 'pam' [preauth]
> > > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive
> > for
> > > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
> > authentication
> > > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> > user=ouruser
> > > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
> > authentication
> > > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
> > > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
> > > user ouruser: 9 (Authentication service cannot retrieve authentication
> > info)
> > > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
> > failure
> > > for ouruser from x.x.x.x
> > > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam
> > for
> > > ouruser from x.x.x.x port 53332 ssh2
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method keyboard-interactive [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
> > > [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive
> > devs
> > >  [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
> > > user=ouruser devs= [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices
> > 'pam'
> > > [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
> > > trying authentication method 'pam' [preauth]
> > > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive
> > for
> > > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > >
> > > With the NIS password the logs show this:
> >
> > Hi,
> >
> > did you drop what happened before or is this the only debug output for
> > the NIS password?
> >
>
>  The below here are just logs from /var/log/secure for the user that
>  successfully logs in with his/her NIS password.
>
>  By "drop what happened before" do you mean the original log snip? Yes I
>  removed those in an attempt to shorten the message content.
>
>  > Feb  9 11:16:57 debug1: do_pam_account: called
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env
> > strings 2
> > > Feb  9 11:16:57 ourserver sshd[536226]: Postponed
> > keyboard-interactive/pam
> > > for cai from 150.108.68.26 port 53646 ssh2 [preauth]
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called
> > > Feb  9 11:16:57 ourserver sshd[536226]: Accepted
> > keyboard-interactive/pam
> > > for cai from 150.108.68.26 port 53646 ssh2
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth:
> > cai
> > > has been authenticated by privileged process
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child
> > log
> > > fd closed
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled
> > > event 2
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid:
> > > 5879/200 (e=0/0)
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds:
> > Not
> > > a GSSAPI mechanism
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled
> > > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
> > > credentials
> > > Feb  9 11:16:57 ourserver systemd[536237]:
> > pam_unix(systemd-user:session):
> > > session opened for user cai(uid=5879) by (uid=0)
> > >
> > > What options should be set in /etc/ssh/sshd_config? Is sssd necessary
> > for
> > > this to work with the FreeIPA password
> >
>
>
>  Yes, SSSD must be configured and runnnig. ssd does appear to be working
>  fine and in /etc/ipa/ca.crt and the service is running correctly:
>
>  [domain/ourdomain.edu]
>
>  id_provider = ipa
>  ipa_server_mode = True
>  ipa_server = ourdomain.edu
>  ipa_domain = ourdomain.edu
>  ipa_hostname = ourdomain.edu
>  auth_provider = ipa
>  chpass_provider = ipa
>  access_provider = ipa
>  cache_credentials = True
>  ldap_tls_cacert = /etc/ipa/ca.crt
>  krb5_store_password_if_offline = True
>  [sssd]
>  services = nss, pam, ifp, ssh, sudo
>
>  domains = ourdomain.edu
>  [nss]
>  homedir_substring = /home
>  memcache_timeout = 600
>
>  [ifp]
>  allowed_uids = ipaapi, root
>
>  systemctl status sssd
>  * sssd.service - System Security Services Daemon
>       Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor
>  preset: enabled)
>       Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1 weeks 3
>  days ago
>
>

[Rob Crittenden]

Robert Kudyba via FreeIPA-users wrote:
>  I tried this on another test server, and configured NIS for the
> users,
>  which are different. Same issue. All the verbose output adds a lot of
>  log noise but I'm hoping it provides a clue.
>  
>  ipactl status
>  Directory Service: RUNNING
>  krb5kdc Service: RUNNING
>  kadmin Service: RUNNING
>  named Service: RUNNING
>  httpd Service: RUNNING
>  ipa-custodia Service: RUNNING
>  pki-tomcatd Service: RUNNING
>  ipa-otpd Service: RUNNING
>  ipa-dnskeysyncd Service: RUNNING
>  ipa: INFO: The ipactl command was successful
>  
>  here is ssh -o PubkeyAuthentication=no -vvv -k ouruser@ourserver
>  OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS  8 Dec 2020
>  debug1: Reading configuration data /etc/ssh/ssh_config
>  debug3: /etc/ssh/ssh_config line 55: Including file
>  /etc/ssh/ssh_config.d/50-redhat.conf depth 0
>  debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
>  debug2: checking match for 'final all' host ourserver originally ourserver
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'
>  debug2: match not found
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file
>  /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
>  debug1: Reading configuration data
>  /etc/crypto-policies/back-ends/openssh.config
>  debug3: gss kex names ok:
>  [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
>  debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org
>
> <mailto:[e-mail]>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
>  debug1: configuration requests final Match pass
>  debug1: re-parsing configuration
>  debug1: Reading configuration data /etc/ssh/ssh_config
>  debug3: /etc/ssh/ssh_config line 55: Including file
>  /etc/ssh/ssh_config.d/50-redhat.conf depth 0
>  debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
>  debug2: checking match for 'final all' host ourserver originally ourserver
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'
>  debug2: match found
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file
>  /etc/crypto-policies/back-ends/openssh.config depth 1
>  debug1: Reading configuration data
>  /etc/crypto-policies/back-ends/openssh.config
>  debug3: gss kex names ok:
>  [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
>  debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org
>
> <mailto:[e-mail]>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
>  debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' ->
>  '/root/.ssh/known_hosts'
>  debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' ->
>  '/root/.ssh/known_hosts2'
>  debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy
>  -p 22 ourserver
>  debug1: identity file /root/.ssh/id_rsa type 0
>  debug1: identity file /root/.ssh/id_rsa-cert type -1
>  debug1: identity file /root/.ssh/id_dsa type -1
>  debug1: identity file /root/.ssh/id_dsa-cert type -1
>  debug1: identity file /root/.ssh/id_ecdsa type -1
>  debug1: identity file /root/.ssh/id_ecdsa-cert type -1
>  debug1: identity file /root/.ssh/id_ecdsa_sk type -1
>  debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
>  debug1: identity file /root/.ssh/id_ed25519 type -1
>  debug1: identity file /root/.ssh/id_ed25519-cert type -1
>  debug1: identity file /root/.ssh/id_ed25519_sk type -1
>  debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
>  debug1: identity file /root/.ssh/id_xmss type -1
>  debug1: identity file /root/.ssh/id_xmss-cert type -1
>  debug1: Local version string SSH-2.0-OpenSSH_8.4
>  debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4
>  debug1: match: OpenSSH_8.4 pat OpenSSH* compat 0x04000000
>  debug2: fd 5 setting O_NONBLOCK
>  debug2: fd 4 setting O_NONBLOCK
>  debug1: Authenticating to ourserver:22 as 'ouruser'
>  debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
>  debug3: record_hostkey: found key type ECDSA in file
>  /root/.ssh/known_hosts:31
>  debug3: load_hostkeys: loaded 1 keys from ourserver
>  debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
>  debug3: order_hostkeyalgs: have matching best-preference key type
>  ecdsa-sha2-nistp256-cert-v01(a)openssh.com
>  <mailto:[e-mail]>, using
>  HostkeyAlgorithms verbatim
>  debug3: send packet: type 20
>  debug1: SSH2_MSG_KEXINIT sent
>  debug3: receive packet: type 20
>  debug1: SSH2_MSG_KEXINIT received
>  debug2: local client KEXINIT proposal
>  debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
>
> <mailto:[e-mail]>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c
>  debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01(a)openssh.com
>
> <mailto:[e-mail]>,[e-mail]
>
> <mailto:[e-mail]>,[e-mail]
>
> <mailto:[e-mail]>,[e-mail]
>
> <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>
> <mailto:[e-mail]>,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[e-mail]
>  <mailto:[e-mail]>,ssh-ed25519,[e-mail]
>  <mailto:[e-mail]>,rsa-sha2-512,rsa-sha2-256,ssh-rsa
>  debug2: ciphers ctos: aes256-gcm(a)openssh.com
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,aes256-ctr,[e-mail]
>  <mailto:[e-mail]>,aes128-ctr
>  debug2: ciphers stoc: aes256-gcm(a)openssh.com
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,aes256-ctr,[e-mail]
>  <mailto:[e-mail]>,aes128-ctr
>  debug2: MACs ctos: hmac-sha2-256-etm(a)openssh.com
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>
> <mailto:[e-mail]>,hmac-sha2-256,hmac-sha1,[e-mail]
>  <mailto:[e-mail]>,hmac-sha2-512
>  debug2: MACs stoc: hmac-sha2-256-etm(a)openssh.com
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>
> <mailto:[e-mail]>,hmac-sha2-256,hmac-sha1,[e-mail]
>  <mailto:[e-mail]>,hmac-sha2-512
>  debug2: compression ctos: none,zlib(a)openssh.com
>  <mailto:[e-mail]>,zlib
>  debug2: compression stoc: none,zlib(a)openssh.com
>  <mailto:[e-mail]>,zlib
>  debug2: languages ctos:
>  debug2: languages stoc:
>  debug2: first_kex_follows 0
>  debug2: reserved 0
>  debug2: peer server KEXINIT proposal
>  debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
>
> <mailto:[e-mail]>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
>  debug2: host key algorithms:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  debug2: ciphers ctos: chacha20-poly1305(a)openssh.com
>
> <mailto:[e-mail]>,aes128-ctr,aes192-ctr,aes256-ctr,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>
>  debug2: ciphers stoc: chacha20-poly1305(a)openssh.com
>
> <mailto:[e-mail]>,aes128-ctr,aes192-ctr,aes256-ctr,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>
>  debug2: MACs ctos: umac-64-etm(a)openssh.com
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,hmac-sha2-256,hmac-sha2-512,hmac-sha1
>  debug2: MACs stoc: umac-64-etm(a)openssh.com
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>,hmac-sha2-256,hmac-sha2-512,hmac-sha1
>  debug2: compression ctos: none,zlib(a)openssh.com <mailto:[e-mail]>
>  debug2: compression stoc: none,zlib(a)openssh.com <mailto:[e-mail]>
>  debug2: languages ctos:
>  debug2: languages stoc:
>  debug2: first_kex_follows 0
>  debug2: reserved 0
>  debug1: kex: algorithm: curve25519-sha256
>  debug1: kex: host key algorithm: ecdsa-sha2-nistp256
>  debug1: kex: server->client cipher: aes256-gcm(a)openssh.com
>  <mailto:[e-mail]> MAC: <implicit> compression: none
>  debug1: kex: client->server cipher: aes256-gcm(a)openssh.com
>  <mailto:[e-mail]> MAC: <implicit> compression: none
>  debug1: kex: curve25519-sha256 need=32 dh_need=32
>  debug1: kex: curve25519-sha256 need=32 dh_need=32
>  debug3: send packet: type 30
>  debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
>  debug3: receive packet: type 31
>  debug1: Server host key: ecdsa-sha2-nistp256
>  SHA256:yVm8V2ODZo0nAuvr9k2ydTJv0RtOgkl8Sp5Mkmp/F0M
>  debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
>  debug3: record_hostkey: found key type ECDSA in file
>  /root/.ssh/known_hosts:31
>  debug3: load_hostkeys: loaded 1 keys from ourserver
>  debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
>  debug1: Host 'ourserver' is known and matches the ECDSA host key.
>  debug1: Found key in /root/.ssh/known_hosts:31
>  debug3: send packet: type 21
>  debug2: set_newkeys: mode 1
>  debug1: rekey out after 4294967296 blocks
>  debug1: SSH2_MSG_NEWKEYS sent
>  debug1: expecting SSH2_MSG_NEWKEYS
>  debug3: receive packet: type 21
>  debug1: SSH2_MSG_NEWKEYS received
>  debug2: set_newkeys: mode 0
>  debug1: rekey in after 4294967296 blocks
>  debug1: Will attempt key: /root/.ssh/id_rsa RSA
>  SHA256:Sena4XB1wVt7x+o55Y9EI5WnQIyZ/SLFk+t6tmBFMYA
>  debug1: Will attempt key: /root/.ssh/id_dsa
>  debug1: Will attempt key: /root/.ssh/id_ecdsa
>  debug1: Will attempt key: /root/.ssh/id_ecdsa_sk
>  debug1: Will attempt key: /root/.ssh/id_ed25519
>  debug1: Will attempt key: /root/.ssh/id_ed25519_sk
>  debug1: Will attempt key: /root/.ssh/id_xmss
>  debug2: pubkey_prepare: done
>  debug3: send packet: type 5
>  debug3: receive packet: type 7
>  debug1: SSH2_MSG_EXT_INFO received
>  debug1: kex_input_ext_info:
>  server-sig-algs=&lt;ssh-ed25519,sk-ssh-ed25519(a)openssh.com
>
> <mailto:[e-mail]>,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[e-mail]
>
> <mailto:[e-mail]>,[e-mail]
>  <mailto:[e-mail]>>
>  debug3: receive packet: type 6
>  debug2: service_accept: ssh-userauth
>  debug1: SSH2_MSG_SERVICE_ACCEPT received
>  debug3: send packet: type 50
>  debug3: receive packet: type 51
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: start over, passed a different list
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: preferred gssapi-with-mic,keyboard-interactive,password
>  debug3: authmethod_lookup gssapi-with-mic
>  debug3: remaining preferred: keyboard-interactive,password
>  debug3: authmethod_is_enabled gssapi-with-mic
>  debug1: Next authentication method: gssapi-with-mic
>  debug1: Unspecified GSS failure.  Minor code may provide more information
>  Server host/ourserver.edu(a)ourserver.edu
>  <mailto:[e-mail]> not found in Kerberos database
>  debug3: send packet: type 50
>  debug2: we sent a gssapi-with-mic packet, wait for reply
>  debug3: receive packet: type 51
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug2: we did not send a packet, disable method
>  debug3: authmethod_lookup keyboard-interactive
>  debug3: remaining preferred: password
>  debug3: authmethod_is_enabled keyboard-interactive
>  debug1: Next authentication method: keyboard-interactive
>  debug2: userauth_kbdint
>  debug3: send packet: type 50
>  debug2: we sent a keyboard-interactive packet, wait for reply
>  debug3: receive packet: type 60
>  debug2: input_userauth_info_req
>  debug2: input_userauth_info_req: num_prompts 1
>  
>  And here are the server logs from /var/log secure and you can see sssd
>  is being used:
>  Feb 10 14:36:24 ourserver sshd[3024290]: debug1: Forked child 3084339.
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Set
>  /proc/self/oom_score_adj to 0
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rexec start in 4 out 4
>  newsock 4 pipe 6 sock 7
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: inetd sockets after
>  dupping: 4, 4
>  Feb 10 14:36:24 ourserver sshd[3084339]: Connection from x.x.x.x port
>  34160 on 150.108.68.128 port 22 rdomain ""
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Local version string
>  SSH-2.0-OpenSSH_8.4
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Remote protocol version
>  2.0, remote software version OpenSSH_8.4
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: match: OpenSSH_8.4 pat
>  OpenSSH* compat 0x04000000
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SELinux support
>  disabled [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: permanently_set_uid:
>  74/74 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: list_hostkey_types:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT sent
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT
>  received [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: algorithm:
>  curve25519-sha256 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: host key
>  algorithm: ecdsa-sha2-nistp256 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: client->server
>  cipher: aes256-gcm(a)openssh.com <mailto:[e-mail]> MAC:
>  <implicit> compression: none [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: server->client
>  cipher: aes256-gcm(a)openssh.com <mailto:[e-mail]> MAC:
>  <implicit> compression: none [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting
>  SSH2_MSG_KEX_ECDH_INIT [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey out after
>  4294967296 blocks [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS sent
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Sending
>  SSH2_MSG_EXT_INFO [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting
>  SSH2_MSG_NEWKEYS [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS
>  received [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey in after
>  4294967296 blocks [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: KEX done [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for
>  user ouruser service ssh-connection method none [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: connection from x.x.x.x
>  matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: initializing for
>  "ouruser"
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_RHOST
>  to "x.x.x.x"
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for
>  user ouruser service ssh-connection method gssapi-with-mic [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for
>  user ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 2 failures 0
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: keyboard-interactive
>  devs  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: Postponed keyboard-interactive
>  for ouruser from x.x.x.x port 34160 ssh2 [preauth]
>  Feb 10 14:36:28 ourserver sshd[3084344]: pam_unix(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>  rhost=x.x.x.x  user=ouruser
>  Feb 10 14:36:28 ourserver proxy_child: pam_unix(sssd-shadowutils:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>  rhost=x.x.x.x  user=ouruser
>  Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>  rhost=x.x.x.x user=ouruser
>  Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received
>  for user ouruser: 7 (Authentication failure)
>  Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication
>  failure for ouruser from x.x.x.x
>  Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam
>  for ouruser from x.x.x.x port 34160 ssh2
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for
>  user ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1
>  [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive
>  devs  [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive
>  for ouruser from x.x.x.x port 34160 ssh2 [preauth]
>  
>  I verified the FreeIPA password in both the GUI and via ipa user-mod.
>  The only time the user is able to log in is using the NIS
>  password. ldapsearch -x -D and kinit username work successfully. klist
>  displays the user details correctly.
>  
>  I can see that the installation script edits /etc/ssh/sshd_config with:
>  Include /etc/ssh/sshd_config.d/04-ipa.conf
>  
>  which has:
>  PubkeyAuthentication yes
>  KerberosAuthentication no
>  GSSAPIAuthentication yes
>  UsePAM yes
>  ChallengeResponseAuthentication yes
>  AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
>  AuthorizedKeysCommandUser nobody
>  
>  When the NIS password is used successfully here are the server logs:
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  user ouruser service ssh-connection method none [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x
>  matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for
>  "ouruser"
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST
>  to "x.x.x.x"
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  user ouruser service ssh-connection method gssapi-with-mic [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  user ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0
>  [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive
>  devs  [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive
>  for ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called
>  Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env strings 2
>  Feb 10 14:56:42 ourserver sshd[3085147]: Postponed
>  keyboard-interactive/pam for ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called
>  
>  I do see the error that sticks out is " Server
>  host/ourserver.edu(a)ourserver.edu <mailto:[e-mail]>
>  not found in Kerberos database" but we have students that log in from
>  all over the world so do all clients need to be added? iptables,
>  firewalld, and nftables are off and disabled. No hbac rules: 
What is ourserver.edu? In order to log in using Kerberos/GSSAPI then the
machine acting as the server needs to be enrolled as an IPA client so it
has a keytab.

rob

>  ipa hbacrule-find
>  --------------------
>  2 HBAC rules matched
>  --------------------
>    Rule name: allow_all
>    User category: all
>    Host category: all
>    Service category: all
>    Description: Allow all users to access any host from any host
>    Enabled: TRUE
>  
>    Rule name: allow_systemd-user
>    User category: all
>    Host category: all
>    Description: Allow pam_systemd to run user@.service to create a system
>  user session
>    Enabled: TRUE
>  ----------------------------
>  Number of entries returned 2
>  
>  Am I missing something obvious to regulars?
>  
>  
>  On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu
>  <mailto:[e-mail]>> wrote:
>  
>      On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users
>      <freeipa-users(a)lists.fedorahosted.org
>      <mailto:[e-mail]>> wrote:
>  
>          On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via
>          FreeIPA-users wrote:
>          > >
>          > > looks like sshd is trying to read
>          /home/ouruser/.ssh/authorized_keys and
>          > > is stuck. Can you read this file from the command line? Is
>          it e.g. on
>          > > NFS which might not be properly mounted?
>          > >
>          > > Does it work if you skip pubkey authentication
>          > >
>          > >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
>          > >
>          > > bye,
>          > > Sumit
>          > >
>          >
>          > Thanks for the suggestion. What happens is the NIS password
>          works. The
>          > FreeIPA password, which I update with:
>          > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with
>          the below
>          > errors/logs
>          >
>          > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child
>          536086.
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>          > /proc/self/oom_score_adj to 0
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in
>          5 out 5
>          > newsock 5 pipe 7 sock 8
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets
>          after
>          > dupping: 4, 4
>          > Feb  9 11:10:34 ourserver sshd[536086]: Connection from
>          x.x.x.x port 53332
>          > on 150.108.64.156 port 22 rdomain ""
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version
>          string
>          > SSH-2.0-OpenSSH_8.4
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote
>          protocol version
>          > 2.0, remote software version OpenSSH_8.4
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match:
>          OpenSSH_8.4 pat
>          > OpenSSH* compat 0x04000000
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux
>          support disabled
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          permanently_set_uid: 74/74
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          list_hostkey_types:
>          >
>          rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_KEXINIT sent
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_KEXINIT received
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>          > curve25519-sha256 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key
>          algorithm:
>          > ecdsa-sha2-nistp256 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          client->server cipher:
>          > aes256-gcm(a)openssh.com <mailto:[e-mail]> MAC:
>          <implicit> compression: none [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          server->client cipher:
>          > aes256-gcm(a)openssh.com <mailto:[e-mail]> MAC:
>          <implicit> compression: none [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          curve25519-sha256
>          > need=32 dh_need=32 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          curve25519-sha256
>          > need=32 dh_need=32 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>          > SSH2_MSG_KEX_ECDH_INIT [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out
>          after 4294967296
>          > blocks [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_NEWKEYS sent
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>          SSH2_MSG_EXT_INFO
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>          SSH2_MSG_NEWKEYS
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_NEWKEYS received
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
>          4294967296
>          > blocks [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          userauth-request for user
>          > ouruser service ssh-connection method none [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0
>          failures 0
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM:
>          initializing for
>          > "ouruser"
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>          PAM_RHOST to
>          > "x.x.x.x"
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>          PAM_TTY to
>          > "ssh"
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          userauth-request for user
>          > ouruser service ssh-connection method keyboard-interactive
>          [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1
>          failures 0
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          keyboard-interactive devs
>          >  [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>          > user=ouruser devs= [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc:
>          devices 'pam'
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          auth2_challenge_start:
>          > trying authentication method 'pam' [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
>          keyboard-interactive for
>          > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>          > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>          authentication
>          > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x 
>          user=ouruser
>          > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>          authentication
>          > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>          user=ouruser
>          > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>          received for
>          > user ouruser: 9 (Authentication service cannot retrieve
>          authentication info)
>          > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM:
>          Authentication failure
>          > for ouruser from x.x.x.x
>          > Feb  9 11:10:41 ourserver sshd[536086]: Failed
>          keyboard-interactive/pam for
>          > ouruser from x.x.x.x port 53332 ssh2
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>          userauth-request for user
>          > ouruser service ssh-connection method keyboard-interactive
>          [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2
>          failures 1
>          > [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>          keyboard-interactive devs
>          >  [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>          > user=ouruser devs= [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc:
>          devices 'pam'
>          > [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>          auth2_challenge_start:
>          > trying authentication method 'pam' [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
>          keyboard-interactive for
>          > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>          >
>          >
>          > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child
>          536086.
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>          > /proc/self/oom_score_adj to 0
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in
>          5 out 5
>          > newsock 5 pipe 7 sock 8
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets
>          after
>          > dupping: 4, 4
>          > Feb  9 11:10:34 ourserver sshd[536086]: Connection from
>          x.x.x.x port 53332
>          > on 150.108.64.156 port 22 rdomain ""
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version
>          string
>          > SSH-2.0-OpenSSH_8.4
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote
>          protocol version
>          > 2.0, remote software version OpenSSH_8.4
>          > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match:
>          OpenSSH_8.4 pat
>          > OpenSSH* compat 0x04000000
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux
>          support disabled
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          permanently_set_uid: 74/74
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          list_hostkey_types:
>          >
>          rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_KEXINIT sent
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_KEXINIT received
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>          > curve25519-sha256 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key
>          algorithm:
>          > ecdsa-sha2-nistp256 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          client->server cipher:
>          > aes256-gcm(a)openssh.com <mailto:[e-mail]> MAC:
>          <implicit> compression: none [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          server->client cipher:
>          > aes256-gcm(a)openssh.com <mailto:[e-mail]> MAC:
>          <implicit> compression: none [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          curve25519-sha256
>          > need=32 dh_need=32 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>          curve25519-sha256
>          > need=32 dh_need=32 [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>          > SSH2_MSG_KEX_ECDH_INIT [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out
>          after 4294967296
>          > blocks [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_NEWKEYS sent
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>          SSH2_MSG_EXT_INFO
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>          SSH2_MSG_NEWKEYS
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          SSH2_MSG_NEWKEYS received
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
>          4294967296
>          > blocks [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          userauth-request for user
>          > ouruser service ssh-connection method none [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0
>          failures 0
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM:
>          initializing for
>          > "ouruser"
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>          PAM_RHOST to
>          > "x.x.x.x"
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>          PAM_TTY to
>          > "ssh"
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          userauth-request for user
>          > ouruser service ssh-connection method keyboard-interactive
>          [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1
>          failures 0
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          keyboard-interactive devs
>          >  [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>          > user=ouruser devs= [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc:
>          devices 'pam'
>          > [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>          auth2_challenge_start:
>          > trying authentication method 'pam' [preauth]
>          > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
>          keyboard-interactive for
>          > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>          > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>          authentication
>          > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x 
>          user=ouruser
>          > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>          authentication
>          > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>          user=ouruser
>          > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>          received for
>          > user ouruser: 9 (Authentication service cannot retrieve
>          authentication info)
>          > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM:
>          Authentication failure
>          > for ouruser from x.x.x.x
>          > Feb  9 11:10:41 ourserver sshd[536086]: Failed
>          keyboard-interactive/pam for
>          > ouruser from x.x.x.x port 53332 ssh2
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>          userauth-request for user
>          > ouruser service ssh-connection method keyboard-interactive
>          [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2
>          failures 1
>          > [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>          keyboard-interactive devs
>          >  [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>          > user=ouruser devs= [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc:
>          devices 'pam'
>          > [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>          auth2_challenge_start:
>          > trying authentication method 'pam' [preauth]
>          > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
>          keyboard-interactive for
>          > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>          >
>          > With the NIS password the logs show this:
>  
>          Hi,
>  
>          did you drop what happened before or is this the only debug
>          output for
>          the NIS password?
>  
>  
>      The below here are just logs from /var/log/secure for the user that 
>      successfully logs in with his/her NIS password.
>  
>      By "drop what happened before" do you mean the original log snip?
>      Yes I removed those in an attempt to shorten the message content.
>  
>          > Feb  9 11:16:57 debug1: do_pam_account: called
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM
>          env strings 2
>          > Feb  9 11:16:57 ourserver sshd[536226]: Postponed
>          keyboard-interactive/pam
>          > for cai from 150.108.68.26 port 53646 ssh2 [preauth]
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>          do_pam_account: called
>          > Feb  9 11:16:57 ourserver sshd[536226]: Accepted
>          keyboard-interactive/pam
>          > for cai from 150.108.68.26 port 53646 ssh2
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>          monitor_child_preauth: cai
>          > has been authenticated by privileged process
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>          monitor_read_log: child log
>          > fd closed
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event:
>          unhandled
>          > event 2
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>          temporarily_use_uid:
>          > 5879/200 (e=0/0)
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>          ssh_gssapi_storecreds: Not
>          > a GSSAPI mechanism
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux
>          support disabled
>          > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
>          > credentials
>          > Feb  9 11:16:57 ourserver systemd[536237]:
>          pam_unix(systemd-user:session):
>          > session opened for user cai(uid=5879) by (uid=0)
>          >
>          > What options should be set in /etc/ssh/sshd_config? Is sssd
>          necessary for
>          > this to work with the FreeIPA password
>  
>  
>       
>      Yes, SSSD must be configured and runnnig. ssd does appear to be
>      working fine and in /etc/ipa/ca.crt and the service is running
>      correctly:
>  
>      [domain/ourdomain.edu <http://ourdomain.edu/>]
>  
>      id_provider = ipa
>      ipa_server_mode = True
>      ipa_server = ourdomain.edu <http://ourdomain.edu/>
>      ipa_domain = ourdomain.edu <http://ourdomain.edu/>
>      ipa_hostname = ourdomain.edu <http://ourdomain.edu/>
>      auth_provider = ipa
>      chpass_provider = ipa
>      access_provider = ipa
>      cache_credentials = True
>      ldap_tls_cacert = /etc/ipa/ca.crt
>      krb5_store_password_if_offline = True
>      [sssd]
>      services = nss, pam, ifp, ssh, sudo
>  
>      domains = ourdomain.edu <http://ourdomain.edu/>
>      [nss]
>      homedir_substring = /home
>      memcache_timeout = 600
>  
>      [ifp]
>      allowed_uids = ipaapi, root
>  
>      systemctl status sssd
>      * sssd.service - System Security Services Daemon
>           Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled;
>      vendor preset: enabled)
>           Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1
>      weeks 3 days ago
>       
>  
>  
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>

[Sumit Bose]

On Wed, Feb 10, 2021 at 03:09:37PM -0500, Robert Kudyba via FreeIPA-users wrote:
>  I tried this on another test server, and configured NIS for the
> users,
>  which are different. Same issue. All the verbose output adds a lot of log
>  noise but I'm hoping it provides a clue.
>  
>  ipactl status
>  Directory Service: RUNNING
>  krb5kdc Service: RUNNING
>  kadmin Service: RUNNING
>  named Service: RUNNING
>  httpd Service: RUNNING
>  ipa-custodia Service: RUNNING
>  pki-tomcatd Service: RUNNING
>  ipa-otpd Service: RUNNING
>  ipa-dnskeysyncd Service: RUNNING
>  ipa: INFO: The ipactl command was successful
>  
>  here is ssh -o PubkeyAuthentication=no -vvv -k ouruser@ourserver
>  OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS  8 Dec 2020
>  debug1: Reading configuration data /etc/ssh/ssh_config
>  debug3: /etc/ssh/ssh_config line 55: Including file
>  /etc/ssh/ssh_config.d/50-redhat.conf depth 0
>  debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
>  debug2: checking match for 'final all' host ourserver originally ourserver
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'
>  debug2: match not found
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file
>  /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
>  debug1: Reading configuration data
>  /etc/crypto-policies/back-ends/openssh.config
>  debug3: gss kex names ok:
>  [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
>  debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org
>
> ,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
>  debug1: configuration requests final Match pass
>  debug1: re-parsing configuration
>  debug1: Reading configuration data /etc/ssh/ssh_config
>  debug3: /etc/ssh/ssh_config line 55: Including file
>  /etc/ssh/ssh_config.d/50-redhat.conf depth 0
>  debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
>  debug2: checking match for 'final all' host ourserver originally ourserver
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'
>  debug2: match found
>  debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file
>  /etc/crypto-policies/back-ends/openssh.config depth 1
>  debug1: Reading configuration data
>  /etc/crypto-policies/back-ends/openssh.config
>  debug3: gss kex names ok:
>  [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
>  debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org
>
> ,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
>  debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' ->
>  '/root/.ssh/known_hosts'
>  debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' ->
>  '/root/.ssh/known_hosts2'
>  debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p
>  22 ourserver
>  debug1: identity file /root/.ssh/id_rsa type 0
>  debug1: identity file /root/.ssh/id_rsa-cert type -1
>  debug1: identity file /root/.ssh/id_dsa type -1
>  debug1: identity file /root/.ssh/id_dsa-cert type -1
>  debug1: identity file /root/.ssh/id_ecdsa type -1
>  debug1: identity file /root/.ssh/id_ecdsa-cert type -1
>  debug1: identity file /root/.ssh/id_ecdsa_sk type -1
>  debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
>  debug1: identity file /root/.ssh/id_ed25519 type -1
>  debug1: identity file /root/.ssh/id_ed25519-cert type -1
>  debug1: identity file /root/.ssh/id_ed25519_sk type -1
>  debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
>  debug1: identity file /root/.ssh/id_xmss type -1
>  debug1: identity file /root/.ssh/id_xmss-cert type -1
>  debug1: Local version string SSH-2.0-OpenSSH_8.4
>  debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4
>  debug1: match: OpenSSH_8.4 pat OpenSSH* compat 0x04000000
>  debug2: fd 5 setting O_NONBLOCK
>  debug2: fd 4 setting O_NONBLOCK
>  debug1: Authenticating to ourserver:22 as 'ouruser'
>  debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
>  debug3: record_hostkey: found key type ECDSA in file
>  /root/.ssh/known_hosts:31
>  debug3: load_hostkeys: loaded 1 keys from ourserver
>  debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
>  debug3: order_hostkeyalgs: have matching best-preference key type
>  ecdsa-sha2-nistp256-cert-v01(a)openssh.com, using HostkeyAlgorithms verbatim
>  debug3: send packet: type 20
>  debug1: SSH2_MSG_KEXINIT sent
>  debug3: receive packet: type 20
>  debug1: SSH2_MSG_KEXINIT received
>  debug2: local client KEXINIT proposal
>  debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
>
> ,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c
>  debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01(a)openssh.com,
>  ecdsa-sha2-nistp384-cert-v01(a)openssh.com,
>  ecdsa-sha2-nistp521-cert-v01(a)openssh.com,
>  [e-mail],ssh-ed25519-cert-v01(a)openssh.com
>  ,[e-mail],rsa-sha2-512-cert-v01(a)openssh.com,
>  [e-mail],ssh-rsa-cert-v01(a)openssh.com
>  ,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
>  [e-mail],ssh-ed25519,sk-ssh-ed25519(a)openssh.com
>  ,rsa-sha2-512,rsa-sha2-256,ssh-rsa
>  debug2: ciphers ctos: [e-mail],chacha20-poly1305(a)openssh.com
>  ,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
>  debug2: ciphers stoc: [e-mail],chacha20-poly1305(a)openssh.com
>  ,aes256-ctr,aes128-gcm(a)openssh.com,aes128-ctr
>  debug2: MACs ctos: [e-mail],hmac-sha1-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com
>  ,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
>  debug2: MACs stoc: [e-mail],hmac-sha1-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com
>  ,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
>  debug2: compression ctos: none,zlib(a)openssh.com,zlib
>  debug2: compression stoc: none,zlib(a)openssh.com,zlib
>  debug2: languages ctos:
>  debug2: languages stoc:
>  debug2: first_kex_follows 0
>  debug2: reserved 0
>  debug2: peer server KEXINIT proposal
>  debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org
>
> ,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
>  debug2: host key algorithms:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  debug2: ciphers ctos: chacha20-poly1305(a)openssh.com
>  ,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
>  aes256-gcm(a)openssh.com
>  debug2: ciphers stoc: chacha20-poly1305(a)openssh.com
>  ,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm(a)openssh.com,
>  aes256-gcm(a)openssh.com
>  debug2: MACs ctos: [e-mail],umac-128-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com,
>  [e-mail],[e-mail],umac-128(a)openssh.com
>  ,hmac-sha2-256,hmac-sha2-512,hmac-sha1
>  debug2: MACs stoc: [e-mail],umac-128-etm(a)openssh.com,
>  [e-mail],hmac-sha2-512-etm(a)openssh.com,
>  [e-mail],[e-mail],umac-128(a)openssh.com
>  ,hmac-sha2-256,hmac-sha2-512,hmac-sha1
>  debug2: compression ctos: none,zlib(a)openssh.com
>  debug2: compression stoc: none,zlib(a)openssh.com
>  debug2: languages ctos:
>  debug2: languages stoc:
>  debug2: first_kex_follows 0
>  debug2: reserved 0
>  debug1: kex: algorithm: curve25519-sha256
>  debug1: kex: host key algorithm: ecdsa-sha2-nistp256
>  debug1: kex: server->client cipher: aes256-gcm(a)openssh.com MAC: <implicit>
>  compression: none
>  debug1: kex: client->server cipher: aes256-gcm(a)openssh.com MAC: <implicit>
>  compression: none
>  debug1: kex: curve25519-sha256 need=32 dh_need=32
>  debug1: kex: curve25519-sha256 need=32 dh_need=32
>  debug3: send packet: type 30
>  debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
>  debug3: receive packet: type 31
>  debug1: Server host key: ecdsa-sha2-nistp256
>  SHA256:yVm8V2ODZo0nAuvr9k2ydTJv0RtOgkl8Sp5Mkmp/F0M
>  debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
>  debug3: record_hostkey: found key type ECDSA in file
>  /root/.ssh/known_hosts:31
>  debug3: load_hostkeys: loaded 1 keys from ourserver
>  debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts"
>  debug1: Host 'ourserver' is known and matches the ECDSA host key.
>  debug1: Found key in /root/.ssh/known_hosts:31
>  debug3: send packet: type 21
>  debug2: set_newkeys: mode 1
>  debug1: rekey out after 4294967296 blocks
>  debug1: SSH2_MSG_NEWKEYS sent
>  debug1: expecting SSH2_MSG_NEWKEYS
>  debug3: receive packet: type 21
>  debug1: SSH2_MSG_NEWKEYS received
>  debug2: set_newkeys: mode 0
>  debug1: rekey in after 4294967296 blocks
>  debug1: Will attempt key: /root/.ssh/id_rsa RSA
>  SHA256:Sena4XB1wVt7x+o55Y9EI5WnQIyZ/SLFk+t6tmBFMYA
>  debug1: Will attempt key: /root/.ssh/id_dsa
>  debug1: Will attempt key: /root/.ssh/id_ecdsa
>  debug1: Will attempt key: /root/.ssh/id_ecdsa_sk
>  debug1: Will attempt key: /root/.ssh/id_ed25519
>  debug1: Will attempt key: /root/.ssh/id_ed25519_sk
>  debug1: Will attempt key: /root/.ssh/id_xmss
>  debug2: pubkey_prepare: done
>  debug3: send packet: type 5
>  debug3: receive packet: type 7
>  debug1: SSH2_MSG_EXT_INFO received
>  debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,
>  sk-ssh-ed25519(a)openssh.com
>
> ,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
>  sk-ecdsa-sha2-nistp256(a)openssh.com,
>  webauthn-sk-ecdsa-sha2-nistp256(a)openssh.com&gt;
>  debug3: receive packet: type 6
>  debug2: service_accept: ssh-userauth
>  debug1: SSH2_MSG_SERVICE_ACCEPT received
>  debug3: send packet: type 50
>  debug3: receive packet: type 51
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: start over, passed a different list
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: preferred gssapi-with-mic,keyboard-interactive,password
>  debug3: authmethod_lookup gssapi-with-mic
>  debug3: remaining preferred: keyboard-interactive,password
>  debug3: authmethod_is_enabled gssapi-with-mic
>  debug1: Next authentication method: gssapi-with-mic
>  debug1: Unspecified GSS failure.  Minor code may provide more information
>  Server host/ourserver.edu(a)ourserver.edu not found in Kerberos database
>  debug3: send packet: type 50
>  debug2: we sent a gssapi-with-mic packet, wait for reply
>  debug3: receive packet: type 51
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug2: we did not send a packet, disable method
>  debug3: authmethod_lookup keyboard-interactive
>  debug3: remaining preferred: password
>  debug3: authmethod_is_enabled keyboard-interactive
>  debug1: Next authentication method: keyboard-interactive
>  debug2: userauth_kbdint
>  debug3: send packet: type 50
>  debug2: we sent a keyboard-interactive packet, wait for reply
>  debug3: receive packet: type 60
>  debug2: input_userauth_info_req
>  debug2: input_userauth_info_req: num_prompts 1
>  
>  And here are the server logs from /var/log secure and you can see sssd is
>  being used:
>  Feb 10 14:36:24 ourserver sshd[3024290]: debug1: Forked child 3084339.
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Set
>  /proc/self/oom_score_adj to 0
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rexec start in 4 out 4
>  newsock 4 pipe 6 sock 7
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: inetd sockets after
>  dupping: 4, 4
>  Feb 10 14:36:24 ourserver sshd[3084339]: Connection from x.x.x.x port 34160
>  on 150.108.68.128 port 22 rdomain ""
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Local version string
>  SSH-2.0-OpenSSH_8.4
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Remote protocol version
>  2.0, remote software version OpenSSH_8.4
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: match: OpenSSH_8.4 pat
>  OpenSSH* compat 0x04000000
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SELinux support disabled
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: permanently_set_uid: 74/74
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: list_hostkey_types:
>  rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT sent
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT received
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: algorithm:
>  curve25519-sha256 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: host key algorithm:
>  ecdsa-sha2-nistp256 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: client->server
>  cipher: aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: server->client
>  cipher: aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256
>  need=32 dh_need=32 [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting
>  SSH2_MSG_KEX_ECDH_INIT [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey out after 4294967296
>  blocks [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS sent
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Sending SSH2_MSG_EXT_INFO
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting SSH2_MSG_NEWKEYS
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS received
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey in after 4294967296
>  blocks [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: KEX done [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user
>  ouruser service ssh-connection method none [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: connection from x.x.x.x
>  matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: initializing for
>  "ouruser"
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_RHOST to
>  "x.x.x.x"
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user
>  ouruser service ssh-connection method gssapi-with-mic [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 2 failures 0
>  [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 10 14:36:24 ourserver sshd[3084339]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 34160 ssh2 [preauth]
>  Feb 10 14:36:28 ourserver sshd[3084344]: pam_unix(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>   user=ouruser
>  Feb 10 14:36:28 ourserver proxy_child: pam_unix(sssd-shadowutils:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>   user=ouruser 
Hi,

are you sure the sssd.conf you have send earlier it the right one?
SSSD'S 'proxy_child' should be only be called if the proxy provider is
configured?

bye,
Sumit

>  Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth):
> authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received for
>  user ouruser: 7 (Authentication failure)
>  Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication failure
>  for ouruser from x.x.x.x
>  Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam
>  for ouruser from x.x.x.x port 34160 ssh2
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1
>  [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 34160 ssh2 [preauth]
>  
>  I verified the FreeIPA password in both the GUI and via ipa user-mod. The
>  only time the user is able to log in is using the NIS password. ldapsearch
>  -x -D and kinit username work successfully. klist displays the user details
>  correctly.
>  
>  I can see that the installation script edits /etc/ssh/sshd_config with:
>  Include /etc/ssh/sshd_config.d/04-ipa.conf
>  
>  which has:
>  PubkeyAuthentication yes
>  KerberosAuthentication no
>  GSSAPIAuthentication yes
>  UsePAM yes
>  ChallengeResponseAuthentication yes
>  AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
>  AuthorizedKeysCommandUser nobody
>  
>  When the NIS password is used successfully here are the server logs:
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user
>  ouruser service ssh-connection method none [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x
>  matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for
>  "ouruser"
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST to
>  "x.x.x.x"
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user
>  ouruser service ssh-connection method gssapi-with-mic [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0
>  [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called
>  Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env strings 2
>  Feb 10 14:56:42 ourserver sshd[3085147]: Postponed keyboard-interactive/pam
>  for ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called
>  
>  I do see the error that sticks out is " Server host/
>  ourserver.edu(a)ourserver.edu not found in Kerberos database" but we have
>  students that log in from all over the world so do all clients need to be
>  added? iptables, firewalld, and nftables are off and disabled. No hbac
>  rules:
>  ipa hbacrule-find
>  --------------------
>  2 HBAC rules matched
>  --------------------
>    Rule name: allow_all
>    User category: all
>    Host category: all
>    Service category: all
>    Description: Allow all users to access any host from any host
>    Enabled: TRUE
>  
>    Rule name: allow_systemd-user
>    User category: all
>    Host category: all
>    Description: Allow pam_systemd to run user@.service to create a system
>  user session
>    Enabled: TRUE
>  ----------------------------
>  Number of entries returned 2
>  
>  Am I missing something obvious to regulars?
>  
>  
>  On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:
>  
>  > On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users <
>  > freeipa-users(a)lists.fedorahosted.org&gt; wrote:
>  >
>  >> On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via FreeIPA-users
>  >> wrote:
>  >> > >
>  >> > > looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys
>  >> and
>  >> > > is stuck. Can you read this file from the command line? Is it e.g. on
>  >> > > NFS which might not be properly mounted?
>  >> > >
>  >> > > Does it work if you skip pubkey authentication
>  >> > >
>  >> > >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
>  >> > >
>  >> > > bye,
>  >> > > Sumit
>  >> > >
>  >> >
>  >> > Thanks for the suggestion. What happens is the NIS password works. The
>  >> > FreeIPA password, which I update with:
>  >> > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with
> the below
>  >> > errors/logs
>  >> >
>  >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  >> > /proc/self/oom_score_adj to 0
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
>  >> > newsock 5 pipe 7 sock 8
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  >> > dupping: 4, 4
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
>  >> 53332
>  >> > on 150.108.64.156 port 22 rdomain ""
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  >> > SSH-2.0-OpenSSH_8.4
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
>  >> > 2.0, remote software version OpenSSH_8.4
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
>  >> > OpenSSH* compat 0x04000000
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
>  >> 74/74
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  >> [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
>  >> received
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  >> > curve25519-sha256 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
>  >> > ecdsa-sha2-nistp256 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server
>  >> cipher:
>  >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client
>  >> cipher:
>  >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  >> > need=32 dh_need=32 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  >> > need=32 dh_need=32 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
>  >> 4294967296
>  >> > blocks [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>  >> SSH2_MSG_EXT_INFO
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> SSH2_MSG_NEWKEYS
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
>  >> received
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
>  >> 4294967296
>  >> > blocks [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method none [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
>  >> > "ouruser"
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST
>  >> to
>  >> > "x.x.x.x"
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
>  >> > "ssh"
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive
>  >> devs
>  >> >  [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  >> > user=ouruser devs= [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  >> 'pam'
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  >> > trying authentication method 'pam' [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive
>  >> for
>  >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  >> authentication
>  >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  >> user=ouruser
>  >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  >> authentication
>  >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
>  >> > user ouruser: 9 (Authentication service cannot retrieve authentication
>  >> info)
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
>  >> failure
>  >> > for ouruser from x.x.x.x
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam
>  >> for
>  >> > ouruser from x.x.x.x port 53332 ssh2
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  >> > [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive
>  >> devs
>  >> >  [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  >> > user=ouruser devs= [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  >> 'pam'
>  >> > [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  >> > trying authentication method 'pam' [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive
>  >> for
>  >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> >
>  >> >
>  >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  >> > /proc/self/oom_score_adj to 0
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5
>  >> > newsock 5 pipe 7 sock 8
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  >> > dupping: 4, 4
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
>  >> 53332
>  >> > on 150.108.64.156 port 22 rdomain ""
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  >> > SSH-2.0-OpenSSH_8.4
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version
>  >> > 2.0, remote software version OpenSSH_8.4
>  >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat
>  >> > OpenSSH* compat 0x04000000
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
>  >> 74/74
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  >> [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
>  >> received
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  >> > curve25519-sha256 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm:
>  >> > ecdsa-sha2-nistp256 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server
>  >> cipher:
>  >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client
>  >> cipher:
>  >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  >> > need=32 dh_need=32 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256
>  >> > need=32 dh_need=32 [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
>  >> 4294967296
>  >> > blocks [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>  >> SSH2_MSG_EXT_INFO
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> SSH2_MSG_NEWKEYS
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
>  >> received
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
>  >> 4294967296
>  >> > blocks [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method none [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for
>  >> > "ouruser"
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST
>  >> to
>  >> > "x.x.x.x"
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to
>  >> > "ssh"
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive
>  >> devs
>  >> >  [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  >> > user=ouruser devs= [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  >> 'pam'
>  >> > [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  >> > trying authentication method 'pam' [preauth]
>  >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive
>  >> for
>  >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  >> authentication
>  >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  >> user=ouruser
>  >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  >> authentication
>  >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for
>  >> > user ouruser: 9 (Authentication service cannot retrieve authentication
>  >> info)
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
>  >> failure
>  >> > for ouruser from x.x.x.x
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam
>  >> for
>  >> > ouruser from x.x.x.x port 53332 ssh2
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  >> > [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive
>  >> devs
>  >> >  [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  >> > user=ouruser devs= [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices
>  >> 'pam'
>  >> > [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start:
>  >> > trying authentication method 'pam' [preauth]
>  >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive
>  >> for
>  >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> >
>  >> > With the NIS password the logs show this:
>  >>
>  >> Hi,
>  >>
>  >> did you drop what happened before or is this the only debug output for
>  >> the NIS password?
>  >>
>  >
>  > The below here are just logs from /var/log/secure for the user that
>  > successfully logs in with his/her NIS password.
>  >
>  > By "drop what happened before" do you mean the original log snip? Yes I
>  > removed those in an attempt to shorten the message content.
>  >
>  > > Feb  9 11:16:57 debug1: do_pam_account: called
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env
>  >> strings 2
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: Postponed
>  >> keyboard-interactive/pam
>  >> > for cai from 150.108.68.26 port 53646 ssh2 [preauth]
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: Accepted
>  >> keyboard-interactive/pam
>  >> > for cai from 150.108.68.26 port 53646 ssh2
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth:
>  >> cai
>  >> > has been authenticated by privileged process
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child
>  >> log
>  >> > fd closed
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled
>  >> > event 2
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid:
>  >> > 5879/200 (e=0/0)
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds:
>  >> Not
>  >> > a GSSAPI mechanism
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled
>  >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
>  >> > credentials
>  >> > Feb  9 11:16:57 ourserver systemd[536237]:
>  >> pam_unix(systemd-user:session):
>  >> > session opened for user cai(uid=5879) by (uid=0)
>  >> >
>  >> > What options should be set in /etc/ssh/sshd_config? Is sssd necessary
>  >> for
>  >> > this to work with the FreeIPA password
>  >>
>  >
>  >
>  > Yes, SSSD must be configured and runnnig. ssd does appear to be working
>  > fine and in /etc/ipa/ca.crt and the service is running correctly:
>  >
>  > [domain/ourdomain.edu]
>  >
>  > id_provider = ipa
>  > ipa_server_mode = True
>  > ipa_server = ourdomain.edu
>  > ipa_domain = ourdomain.edu
>  > ipa_hostname = ourdomain.edu
>  > auth_provider = ipa
>  > chpass_provider = ipa
>  > access_provider = ipa
>  > cache_credentials = True
>  > ldap_tls_cacert = /etc/ipa/ca.crt
>  > krb5_store_password_if_offline = True
>  > [sssd]
>  > services = nss, pam, ifp, ssh, sudo
>  >
>  > domains = ourdomain.edu
>  > [nss]
>  > homedir_substring = /home
>  > memcache_timeout = 600
>  >
>  > [ifp]
>  > allowed_uids = ipaapi, root
>  >
>  > systemctl status sssd
>  > * sssd.service - System Security Services Daemon
>  >      Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor
>  > preset: enabled)
>  >      Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1 weeks 3
>  > days ago
>  >
>  > 
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...

[Robert Kudyba]

>  What is ourserver.edu? In order to log in using Kerberos/GSSAPI then the
>  machine acting as the server needs to be enrolled as an IPA client so it
>  has a keytab.
>  rob 

OK I added a Fedora server as a client. From ipa host-show
client.ourserver.edu
  Host name: client.ourserver.edu
  Platform: x86_64
  Operating system: 5.10.13-200.fc33.x86_64
  Principal name: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU
  Principal alias: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU
  SSH public key fingerprint: SHA256:xxx(ecdsa-sha2-nistp256),
                              SHA256:xxx (ssh-ed25519),
                              SHA256:xxx (ssh-rsa)
  Password: False
  Keytab: True
  Managed by: client.ourserver.edu

ssh -k still fails.

I created a new user in via ipa user-add and that user also fails even with
a su -:

su: user testuser does not exist or the user entry does not contain all the
required fields

Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey out after 4294967296
blocks [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS sent
[preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: Sending SSH2_MSG_EXT_INFO
[preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: expecting SSH2_MSG_NEWKEYS
[preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS received
[preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey in after 4294967296
blocks [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: KEX done [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user
yume service ssh-connection method none [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 0 failures 0
[preauth]
Feb 11 15:07:21 ourserver sshd[609424]: Invalid user testuser from x.x.x.x
port 36264
Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: initializing for
"testuser"
Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_RHOST to
"x.x.x.x"
Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_TTY to
"ssh"
Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user
testuser service ssh-connection method keyboard-interactive [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 1 failures 0
[preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: keyboard-interactive devs
 [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge:
user=testuser devs= [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb 11 15:07:21 ourserver sshd[609424]: Postponed keyboard-interactive for
invalid user testuser from x.x.x.x port 36264 ssh2 [preauth]
Feb 11 15:07:24 ourserver sshd[609424]: Connection closed by invalid user
testuser x.x.x.x port 36264 [preauth]

are you sure the sssd.conf you have send earlier it the right one?
>  SSSD'S 'proxy_child' should be only be called if the
> proxy provider is
>  configured?
>
>  bye,
>  Sumit
>  

Sorry, you are correct. I'm testing between 2 test NIS servers to see if
there's a difference in behavior but the same problem, ssh -k never works
with the FreeIPA/Kerberos password


>  > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth):
>  authentication
>  > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received for
>  > user ouruser: 7 (Authentication failure)
>  > Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication
>  failure
>  > for ouruser from x.x.x.x
>  > Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam
>  > for ouruser from x.x.x.x port 34160 ssh2
>  > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for
>  user
>  > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1
>  > [preauth]
>  > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive
>  devs
>  >  [preauth]
>  > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge:
>  > user=ouruser devs= [preauth]
>  > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
>  > 'pam' [preauth]
>  > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start:
>  > trying authentication method 'pam' [preauth]
>  > Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive
>  for
>  > ouruser from x.x.x.x port 34160 ssh2 [preauth]
>  >
>  > I verified the FreeIPA password in both the GUI and via ipa user-mod. The
>  > only time the user is able to log in is using the NIS password.
>  ldapsearch
>  > -x -D and kinit username work successfully. klist displays the user
>  details
>  > correctly.
>  >
>  > I can see that the installation script edits /etc/ssh/sshd_config with:
>  > Include /etc/ssh/sshd_config.d/04-ipa.conf
>  >
>  > which has:
>  > PubkeyAuthentication yes
>  > KerberosAuthentication no
>  > GSSAPIAuthentication yes
>  > UsePAM yes
>  > ChallengeResponseAuthentication yes
>  > AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
>  > AuthorizedKeysCommandUser nobody
>  >
>  > When the NIS password is used successfully here are the server logs:
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  user
>  > ouruser service ssh-connection method none [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0
>  > [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x
>  > matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for
>  > "ouruser"
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST
>  to
>  > "x.x.x.x"
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to
>  > "ssh"
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  user
>  > ouruser service ssh-connection method gssapi-with-mic [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0
>  > [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  user
>  > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0
>  > [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive
>  devs
>  >  [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge:
>  > user=ouruser devs= [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices
>  > 'pam' [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start:
>  > trying authentication method 'pam' [preauth]
>  > Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive
>  for
>  > ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  > Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called
>  > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env
>  strings 2
>  > Feb 10 14:56:42 ourserver sshd[3085147]: Postponed
>  keyboard-interactive/pam
>  > for ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called
>  >
>  > I do see the error that sticks out is " Server host/
>  > ourserver.edu(a)ourserver.edu not found in Kerberos database" but we have
>  > students that log in from all over the world so do all clients need to be
>  > added? iptables, firewalld, and nftables are off and disabled. No hbac
>  > rules:
>  > ipa hbacrule-find
>  > --------------------
>  > 2 HBAC rules matched
>  > --------------------
>  >   Rule name: allow_all
>  >   User category: all
>  >   Host category: all
>  >   Service category: all
>  >   Description: Allow all users to access any host from any host
>  >   Enabled: TRUE
>  >
>  >   Rule name: allow_systemd-user
>  >   User category: all
>  >   Host category: all
>  >   Description: Allow pam_systemd to run user@.service to create a system
>  > user session
>  >   Enabled: TRUE
>  > ----------------------------
>  > Number of entries returned 2
>  >
>  > Am I missing something obvious to regulars?
>  >
>  >
>  > On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu&gt;
>  wrote:
>  >
>  > > On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users <
>  > > freeipa-users(a)lists.fedorahosted.org&gt; wrote:
>  > >
>  > >> On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via
>  FreeIPA-users
>  > >> wrote:
>  > >> > >
>  > >> > > looks like sshd is trying to read
>  /home/ouruser/.ssh/authorized_keys
>  > >> and
>  > >> > > is stuck. Can you read this file from the command line? Is it
>  e.g. on
>  > >> > > NFS which might not be properly mounted?
>  > >> > >
>  > >> > > Does it work if you skip pubkey authentication
>  > >> > >
>  > >> > >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
>  > >> > >
>  > >> > > bye,
>  > >> > > Sumit
>  > >> > >
>  > >> >
>  > >> > Thanks for the suggestion. What happens is the NIS password works.
>  The
>  > >> > FreeIPA password, which I update with:
>  > >> > ipa user-mod ouruser --setattr "userpassword=xxxx", fails
> with the
>  below
>  > >> > errors/logs
>  > >> >
>  > >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  > >> > /proc/self/oom_score_adj to 0
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5
>  out 5
>  > >> > newsock 5 pipe 7 sock 8
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  > >> > dupping: 4, 4
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
>  > >> 53332
>  > >> > on 150.108.64.156 port 22 rdomain ""
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  > >> > SSH-2.0-OpenSSH_8.4
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol
>  version
>  > >> > 2.0, remote software version OpenSSH_8.4
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4
>  pat
>  > >> > OpenSSH* compat 0x04000000
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support
>  disabled
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
>  > >> 74/74
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  > >> [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
>  sent
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
>  > >> received
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  > >> > curve25519-sha256 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key
>  algorithm:
>  > >> > ecdsa-sha2-nistp256 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> client->server
>  > >> cipher:
>  > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> server->client
>  > >> cipher:
>  > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  curve25519-sha256
>  > >> > need=32 dh_need=32 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  curve25519-sha256
>  > >> > need=32 dh_need=32 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  > >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
>  > >> 4294967296
>  > >> > blocks [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
>  sent
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>  > >> SSH2_MSG_EXT_INFO
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  > >> SSH2_MSG_NEWKEYS
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
>  > >> received
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
>  > >> 4294967296
>  > >> > blocks [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  > >> user
>  > >> > ouruser service ssh-connection method none [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing
>  for
>  > >> > "ouruser"
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  PAM_RHOST
>  > >> to
>  > >> > "x.x.x.x"
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  PAM_TTY to
>  > >> > "ssh"
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  > >> user
>  > >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive
>  > >> devs
>  > >> >  [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  > >> > user=ouruser devs= [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc:
>  devices
>  > >> 'pam'
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  auth2_challenge_start:
>  > >> > trying authentication method 'pam' [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
>  keyboard-interactive
>  > >> for
>  > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  > >> authentication
>  > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  > >> user=ouruser
>  > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  > >> authentication
>  > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  user=ouruser
>  > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  received for
>  > >> > user ouruser: 9 (Authentication service cannot retrieve
>  authentication
>  > >> info)
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
>  > >> failure
>  > >> > for ouruser from x.x.x.x
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed
>  keyboard-interactive/pam
>  > >> for
>  > >> > ouruser from x.x.x.x port 53332 ssh2
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for
>  > >> user
>  > >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  > >> > [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive
>  > >> devs
>  > >> >  [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  > >> > user=ouruser devs= [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc:
>  devices
>  > >> 'pam'
>  > >> > [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>  auth2_challenge_start:
>  > >> > trying authentication method 'pam' [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
>  keyboard-interactive
>  > >> for
>  > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  > >> >
>  > >> >
>  > >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086.
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  > >> > /proc/self/oom_score_adj to 0
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5
>  out 5
>  > >> > newsock 5 pipe 7 sock 8
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after
>  > >> > dupping: 4, 4
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port
>  > >> 53332
>  > >> > on 150.108.64.156 port 22 rdomain ""
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string
>  > >> > SSH-2.0-OpenSSH_8.4
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol
>  version
>  > >> > 2.0, remote software version OpenSSH_8.4
>  > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4
>  pat
>  > >> > OpenSSH* compat 0x04000000
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support
>  disabled
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid:
>  > >> 74/74
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types:
>  > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  > >> [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
>  sent
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
>  > >> received
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
>  > >> > curve25519-sha256 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key
>  algorithm:
>  > >> > ecdsa-sha2-nistp256 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> client->server
>  > >> cipher:
>  > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> server->client
>  > >> cipher:
>  > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  curve25519-sha256
>  > >> > need=32 dh_need=32 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  curve25519-sha256
>  > >> > need=32 dh_need=32 [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  > >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
>  > >> 4294967296
>  > >> > blocks [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
>  sent
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>  > >> SSH2_MSG_EXT_INFO
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  > >> SSH2_MSG_NEWKEYS
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
>  > >> received
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
>  > >> 4294967296
>  > >> > blocks [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  > >> user
>  > >> > ouruser service ssh-connection method none [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing
>  for
>  > >> > "ouruser"
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  PAM_RHOST
>  > >> to
>  > >> > "x.x.x.x"
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  PAM_TTY to
>  > >> > "ssh"
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for
>  > >> user
>  > >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive
>  > >> devs
>  > >> >  [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
>  > >> > user=ouruser devs= [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc:
>  devices
>  > >> 'pam'
>  > >> > [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  auth2_challenge_start:
>  > >> > trying authentication method 'pam' [preauth]
>  > >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
>  keyboard-interactive
>  > >> for
>  > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  > >> authentication
>  > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  > >> user=ouruser
>  > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  > >> authentication
>  > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  user=ouruser
>  > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  received for
>  > >> > user ouruser: 9 (Authentication service cannot retrieve
>  authentication
>  > >> info)
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication
>  > >> failure
>  > >> > for ouruser from x.x.x.x
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed
>  keyboard-interactive/pam
>  > >> for
>  > >> > ouruser from x.x.x.x port 53332 ssh2
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for
>  > >> user
>  > >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1
>  > >> > [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive
>  > >> devs
>  > >> >  [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
>  > >> > user=ouruser devs= [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc:
>  devices
>  > >> 'pam'
>  > >> > [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>  auth2_challenge_start:
>  > >> > trying authentication method 'pam' [preauth]
>  > >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
>  keyboard-interactive
>  > >> for
>  > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  > >> >
>  > >> > With the NIS password the logs show this:
>  > >>
>  > >> Hi,
>  > >>
>  > >> did you drop what happened before or is this the only debug output for
>  > >> the NIS password?
>  > >>
>  > >
>  > > The below here are just logs from /var/log/secure for the user that
>  > > successfully logs in with his/her NIS password.
>  > >
>  > > By "drop what happened before" do you mean the original log snip? Yes
> I
>  > > removed those in an attempt to shorten the message content.
>  > >
>  > > > Feb  9 11:16:57 debug1: do_pam_account: called
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env
>  > >> strings 2
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: Postponed
>  > >> keyboard-interactive/pam
>  > >> > for cai from 150.108.68.26 port 53646 ssh2 [preauth]
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account:
>  called
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: Accepted
>  > >> keyboard-interactive/pam
>  > >> > for cai from 150.108.68.26 port 53646 ssh2
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>  monitor_child_preauth:
>  > >> cai
>  > >> > has been authenticated by privileged process
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log:
>  child
>  > >> log
>  > >> > fd closed
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event:
>  unhandled
>  > >> > event 2
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid:
>  > >> > 5879/200 (e=0/0)
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>  ssh_gssapi_storecreds:
>  > >> Not
>  > >> > a GSSAPI mechanism
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support
>  disabled
>  > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
>  > >> > credentials
>  > >> > Feb  9 11:16:57 ourserver systemd[536237]:
>  > >> pam_unix(systemd-user:session):
>  > >> > session opened for user cai(uid=5879) by (uid=0)
>  > >> >
>  > >> > What options should be set in /etc/ssh/sshd_config? Is sssd
>  necessary
>  > >> for
>  > >> > this to work with the FreeIPA password
>  > >>
>  > >
>  > >
>  > > Yes, SSSD must be configured and runnnig. ssd does appear to be working
>  > > fine and in /etc/ipa/ca.crt and the service is running correctly:
>  > >
>  > > [domain/ourdomain.edu]
>  > >
>  > > id_provider = ipa
>  > > ipa_server_mode = True
>  > > ipa_server = ourdomain.edu
>  > > ipa_domain = ourdomain.edu
>  > > ipa_hostname = ourdomain.edu
>  > > auth_provider = ipa
>  > > chpass_provider = ipa
>  > > access_provider = ipa
>  > > cache_credentials = True
>  > > ldap_tls_cacert = /etc/ipa/ca.crt
>  > > krb5_store_password_if_offline = True
>  > > [sssd]
>  > > services = nss, pam, ifp, ssh, sudo
>  > >
>  > > domains = ourdomain.edu
>  > > [nss]
>  > > homedir_substring = /home
>  > > memcache_timeout = 600
>  > >
>  > > [ifp]
>  > > allowed_uids = ipaapi, root
>  > >
>  > > systemctl status sssd
>  > > * sssd.service - System Security Services Daemon
>  > >      Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled;
>  vendor
>  > > preset: enabled)
>  > >      Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1
>  weeks 3
>  > > days ago
>  > >
>  > >
>
>  > _______________________________________________
>  > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  > To unsubscribe send an email to
>  freeipa-users-leave(a)lists.fedorahosted.org
>  > Fedora Code of Conduct:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
>  > List Guidelines:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi...
>  > List Archives:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o...
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
>  List Guidelines:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi...
>  List Archives:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o...
>  Do not reply to spam on the list, report it:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__pagure.io_fedora-2Di...
>

[Robert Kudyba]

One one of the test servers sssd fails to start correctly. Here are the
errors and their respective logs, with debug enabled.
freeipa-server-4.9.1-1.fc33.x86_64

systemctl status sssd --no-pager -l
* sssd.service - System Security Services Daemon
     Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor
preset: enabled)
     Active: activating (start) since Tue 2021-02-16 15:15:32 EST; 1s ago
   Main PID: 809519 (sssd)
      Tasks: 2 (limit: 9484)
     Memory: 10.9M
        CPU: 998ms
     CGroup: /system.slice/sssd.service
             |-809519 /usr/sbin/sssd -i --logger=files
             `-809520 /usr/libexec/sssd/sssd_be --domain implicit_files
--uid 0 --gid 0 --logger=files

Feb 16 15:15:32 ourdomain.edu systemd[1]: Starting System Security Services
Daemon...
Feb 16 15:15:32 ourdomain.edu sssd[809519]: NSCD socket was detected and
seems to be configured to cache some of the databases controlled by SSSD
[passwd,group,netgroup,services]. It is recommended not to run NSCD in
parallel with SSSD, unless NSCD is configured not to cache these.
Feb 16 15:15:33 ourdomain.edu sssd[809519]: Starting up

However:
systemctl status nscd
* nscd.service - Name Service Cache Daemon
     Loaded: loaded (/usr/lib/systemd/system/nscd.service; disabled; vendor
preset: disabled)
     Active: inactive (dead)

 /var/log/sssd/sssd_ourdomain.edu.log
(2021-02-16 15:06:30): [be[ourdomain.edu]] [ipa_init_dyndns] (0x0100):
Dynamic DNS updates are off.
(2021-02-16 15:06:30): [be[ourdomain.edu]] [setup_tls_config] (0x0020):
Unknown value for tls_reqcert 'never (or allow'.
(2021-02-16 15:06:30): [be[ourdomain.edu]] [ipa_init_misc] (0x0020): Unable
to get TLS options [22]: Invalid argument
(2021-02-16 15:06:30): [be[ourdomain.edu]] [sssm_ipa_init] (0x0020): Unable
to init IPA module [22]: Invalid argument
(2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_module_run_constructor]
(0x0010): Module [ipa] constructor failed [22]: Invalid argument
(2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_load_module] (0x0020):
Unable to create DP module.
(2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_target_init] (0x0010):
Unable to load module ipa
(2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_load_targets] (0x0020):
Unable to load target [id] [80]: Accessing a corrupted shared library.
(2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_init_done] (0x0020): Unable
to initialize DP targets [1432158209]: Internal Error
(2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_terminate_active_requests]
(0x0400): Terminating active data provider requests
(2021-02-16 15:06:30): [be[ourdomain.edu]] [sbus_server_symlink_remove]
(0x0400): Symlink removed [/var/lib/sss/pipes/private/sbus-dp_ourdomain.edu]

 /var/log/sssd/sssd.log
(2021-02-16 15:13:21): [sssd] [monitor_cleanup] (0x0010): Error removing
pidfile! (2 [No such file or directory])
(2021-02-16 15:13:22): [sssd] [server_setup] (0x0040): Starting with debug
level = 0x0070
(2021-02-16 15:13:22): [sssd] [svc_child_info] (0x0040): Child [809126] ('
ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
(2021-02-16 15:13:22): [sssd] [svc_child_info] (0x0040): Child [809127] ('
ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
(2021-02-16 15:13:24): [sssd] [svc_child_info] (0x0040): Child [809133] ('
ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
(2021-02-16 15:13:27): [sssd] [services_startup_timeout] (0x0020):
Providers did not start in time, forcing services startup!
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809137]
('nss':'nss') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809139]
('ifp':'ifp') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809141]
('sudo':'sudo') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809138]
('pam':'pam') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809140]
('ssh':'ssh') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809142]
('pac':'pac') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809143]
('nss':'nss') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809144]
('ifp':'ifp') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809145]
('sudo':'sudo') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809147]
('ssh':'ssh') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809146]
('pam':'pam') exited with code [3]
(2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809148]
('pac':'pac') exited with code [3]
(2021-02-16 15:13:28): [sssd] [svc_child_info] (0x0040): Child [809153] ('
ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
(2021-02-16 15:13:28): [sssd] [monitor_restart_service] (0x0010): Process [
ourdomain.edu], definitely stopped!
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0040): Returned with: 1
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
[pac][809148]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
[pac][809148]: [No such process]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
[ssh][809147]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
[ssh][809147]: [No such process]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
[pam][809146]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
[pam][809146]: [No such process]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
[sudo][809145]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
[sudo][809145]: [No such process]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
[ifp][809144]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
[ifp][809144]: [No such process]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
[nss][809143]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
[nss][809143]: [No such process]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
[implicit_files][809125]
(2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Child
[implicit_files] exited gracefully
(2021-02-16 15:13:28): [sssd] [monitor_cleanup] (0x0010): Error removing
pidfile! (2 [No such file or directory])

In /var/log/sssd/sssd_pam.log,  /var/log/sssd/sssd_sudo.log,
/var/log/sssd/sssd_ifp.log, /var/log/sssd/sssd_pac.log  &
/var/log/sssd/sssd_nss.log all these logs end with:
(2021-02-16 15:08:30): [pam] [server_setup] (0x0040): Starting with debug
level = 0x0070
(2021-02-16 15:08:30): [pam] [sss_iface_connect_address] (0x0020):
check_file failed for [/var/lib/sss/pipes/private/sbus-dp_ourdomain.edu].
(2021-02-16 15:08:30): [pam] [sss_dp_init] (0x0010): Failed to connect to
backend server.
(2021-02-16 15:08:30): [pam] [sss_process_init] (0x0010): fatal error
setting up backend connector
(2021-02-16 15:08:30): [pam] [pam_process_init] (0x0010):
sss_process_init() failed

/var/log/sssd/sssd_implicit_files.log
(2021-02-16 15:12:24): [be[implicit_files]] [sysdb_set_cache_entry_attr]
(0x0040): Error: 17 (File exists)
(2021-02-16 15:12:24): [be[implicit_files]] [sysdb_set_cache_entry_attr]
(0x0040): Error: 17 (File exists)
(2021-02-16 15:12:24): [be[implicit_files]] [sysdb_store_new_group]
(0x0040): sysdb_add_group failed (while renaming group) for:
wireshark@implicit_files [1630].
(2021-02-16 15:12:24): [be[implicit_files]] [sysdb_store_group] (0x0040):
Cache update failed: 17
(2021-02-16 15:12:24): [be[implicit_files]] [save_file_group] (0x0040):
Could not add group to cache
(2021-02-16 15:12:26): [be[implicit_files]] [orderly_shutdown] (0x0010):
SIGTERM: killing children
(2021-02-16 15:12:26): [be[implicit_files]] [orderly_shutdown] (0x0040):
Shutting down (status = 0)
(2021-02-16 15:12:26): [be[implicit_files]] [server_setup] (0x0040):
Starting with debug level = 0x0070

However ipactl status does not show any errors:
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful

Anything sticking out?

On Thu, Feb 11, 2021 at 4:51 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:

>  What is ourserver.edu? In order to log in using Kerberos/GSSAPI then
> the
> > machine acting as the server needs to be enrolled as an IPA client so it
> > has a keytab.
> > rob
>
>
>  OK I added a Fedora server as a client. From ipa host-show
>  client.ourserver.edu
>    Host name: client.ourserver.edu
>    Platform: x86_64
>    Operating system: 5.10.13-200.fc33.x86_64
>    Principal name: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU
>    Principal alias: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU
>    SSH public key fingerprint: SHA256:xxx(ecdsa-sha2-nistp256),
>                                SHA256:xxx (ssh-ed25519),
>                                SHA256:xxx (ssh-rsa)
>    Password: False
>    Keytab: True
>    Managed by: client.ourserver.edu
>
>  ssh -k still fails.
>
>  I created a new user in via ipa user-add and that user also fails even
>  with a su -:
>
>  su: user testuser does not exist or the user entry does not contain all
>  the required fields
>
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey out after 4294967296
>  blocks [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS sent
>  [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: Sending SSH2_MSG_EXT_INFO
>  [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: expecting SSH2_MSG_NEWKEYS
>  [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS received
>  [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey in after 4294967296
>  blocks [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: KEX done [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user
>  yume service ssh-connection method none [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 0 failures 0
>  [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: Invalid user testuser from x.x.x.x
>  port 36264
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: initializing for
>  "testuser"
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_RHOST to
>  "x.x.x.x"
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user
>  testuser service ssh-connection method keyboard-interactive [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 1 failures 0
>  [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge:
>  user=testuser devs= [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 11 15:07:21 ourserver sshd[609424]: Postponed keyboard-interactive for
>  invalid user testuser from x.x.x.x port 36264 ssh2 [preauth]
>  Feb 11 15:07:24 ourserver sshd[609424]: Connection closed by invalid user
>  testuser x.x.x.x port 36264 [preauth]
>
>  are you sure the sssd.conf you have send earlier it the right one?
> > SSSD'S 'proxy_child' should be only be called if the proxy provider is
> > configured?
> >
> > bye,
> > Sumit
> >
>
>
>  Sorry, you are correct. I'm testing between 2 test NIS servers to see if
>  there's a difference in behavior but the same problem, ssh -k never works
>  with the FreeIPA/Kerberos password
>
>
> >
> > > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth):
> > authentication
> > > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
> > > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received
> > for
> > > user ouruser: 7 (Authentication failure)
> > > Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication
> > failure
> > > for ouruser from x.x.x.x
> > > Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam
> > > for ouruser from x.x.x.x port 34160 ssh2
> > > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method keyboard-interactive [preauth]
> > > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1
> > > [preauth]
> > > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive
> > devs
> > >  [preauth]
> > > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge:
> > > user=ouruser devs= [preauth]
> > > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
> > > 'pam' [preauth]
> > > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start:
> > > trying authentication method 'pam' [preauth]
> > > Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive
> > for
> > > ouruser from x.x.x.x port 34160 ssh2 [preauth]
> > >
> > > I verified the FreeIPA password in both the GUI and via ipa user-mod.
> > The
> > > only time the user is able to log in is using the NIS password.
> > ldapsearch
> > > -x -D and kinit username work successfully. klist displays the user
> > details
> > > correctly.
> > >
> > > I can see that the installation script edits /etc/ssh/sshd_config with:
> > > Include /etc/ssh/sshd_config.d/04-ipa.conf
> > >
> > > which has:
> > > PubkeyAuthentication yes
> > > KerberosAuthentication no
> > > GSSAPIAuthentication yes
> > > UsePAM yes
> > > ChallengeResponseAuthentication yes
> > > AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
> > > AuthorizedKeysCommandUser nobody
> > >
> > > When the NIS password is used successfully here are the server logs:
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method none [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0
> > > [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x
> > > matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for
> > > "ouruser"
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST
> > to
> > > "x.x.x.x"
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to
> > > "ssh"
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method gssapi-with-mic [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0
> > > [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
> > user
> > > ouruser service ssh-connection method keyboard-interactive [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0
> > > [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive
> > devs
> > >  [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge:
> > > user=ouruser devs= [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices
> > > 'pam' [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start:
> > > trying authentication method 'pam' [preauth]
> > > Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive
> > for
> > > ouruser from x.x.x.x port 35046 ssh2 [preauth]
> > > Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called
> > > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env
> > strings 2
> > > Feb 10 14:56:42 ourserver sshd[3085147]: Postponed
> > keyboard-interactive/pam
> > > for ouruser from x.x.x.x port 35046 ssh2 [preauth]
> > > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called
> > >
> > > I do see the error that sticks out is " Server host/
> > > ourserver.edu(a)ourserver.edu not found in Kerberos database" but we have
> > > students that log in from all over the world so do all clients need to
> > be
> > > added? iptables, firewalld, and nftables are off and disabled. No hbac
> > > rules:
> > > ipa hbacrule-find
> > > --------------------
> > > 2 HBAC rules matched
> > > --------------------
> > >   Rule name: allow_all
> > >   User category: all
> > >   Host category: all
> > >   Service category: all
> > >   Description: Allow all users to access any host from any host
> > >   Enabled: TRUE
> > >
> > >   Rule name: allow_systemd-user
> > >   User category: all
> > >   Host category: all
> > >   Description: Allow pam_systemd to run user@.service to create a
> > system
> > > user session
> > >   Enabled: TRUE
> > > ----------------------------
> > > Number of entries returned 2
> > >
> > > Am I missing something obvious to regulars?
> > >
> > >
> > > On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu&gt;
> > wrote:
> > >
> > > > On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users <
> > > > freeipa-users(a)lists.fedorahosted.org&gt; wrote:
> > > >
> > > >> On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via
> > FreeIPA-users
> > > >> wrote:
> > > >> > >
> > > >> > > looks like sshd is trying to read
> > /home/ouruser/.ssh/authorized_keys
> > > >> and
> > > >> > > is stuck. Can you read this file from the command line? Is
> it
> > e.g. on
> > > >> > > NFS which might not be properly mounted?
> > > >> > >
> > > >> > > Does it work if you skip pubkey authentication
> > > >> > >
> > > >> > >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver
> > > >> > >
> > > >> > > bye,
> > > >> > > Sumit
> > > >> > >
> > > >> >
> > > >> > Thanks for the suggestion. What happens is the NIS password
> works.
> > The
> > > >> > FreeIPA password, which I update with:
> > > >> > ipa user-mod ouruser --setattr "userpassword=xxxx",
> fails with the
> > below
> > > >> > errors/logs
> > > >> >
> > > >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child
> > 536086.
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
> > > >> > /proc/self/oom_score_adj to 0
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5
> > out 5
> > > >> > newsock 5 pipe 7 sock 8
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets
> after
> > > >> > dupping: 4, 4
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x
> > port
> > > >> 53332
> > > >> > on 150.108.64.156 port 22 rdomain ""
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version
> > string
> > > >> > SSH-2.0-OpenSSH_8.4
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol
> > version
> > > >> > 2.0, remote software version OpenSSH_8.4
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match:
> OpenSSH_8.4
> > pat
> > > >> > OpenSSH* compat 0x04000000
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support
> > disabled
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> > permanently_set_uid:
> > > >> 74/74
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> list_hostkey_types:
> > > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
> > > >> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
> > sent
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
> > > >> received
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
> > > >> > curve25519-sha256 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key
> > algorithm:
> > > >> > ecdsa-sha2-nistp256 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> client->server
> > > >> cipher:
> > > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> server->client
> > > >> cipher:
> > > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> > curve25519-sha256
> > > >> > need=32 dh_need=32 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> > curve25519-sha256
> > > >> > need=32 dh_need=32 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > > >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
> > > >> 4294967296
> > > >> > blocks [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
> > sent
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
> > > >> SSH2_MSG_EXT_INFO
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > > >> SSH2_MSG_NEWKEYS
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
> > > >> received
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
> > > >> 4294967296
> > > >> > blocks [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done
> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request
> > for
> > > >> user
> > > >> > ouruser service ssh-connection method none [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0
> failures
> > 0
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing
> > for
> > > >> > "ouruser"
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
> > PAM_RHOST
> > > >> to
> > > >> > "x.x.x.x"
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
> > PAM_TTY to
> > > >> > "ssh"
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request
> > for
> > > >> user
> > > >> > ouruser service ssh-connection method keyboard-interactive
> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1
> failures
> > 0
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> > keyboard-interactive
> > > >> devs
> > > >> >  [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
> > > >> > user=ouruser devs= [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc:
> > devices
> > > >> 'pam'
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> > auth2_challenge_start:
> > > >> > trying authentication method 'pam' [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
> > keyboard-interactive
> > > >> for
> > > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
> > > >> authentication
> > > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> > > >> user=ouruser
> > > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
> > > >> authentication
> > > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> > user=ouruser
> > > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
> > received for
> > > >> > user ouruser: 9 (Authentication service cannot retrieve
> > authentication
> > > >> info)
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM:
> Authentication
> > > >> failure
> > > >> > for ouruser from x.x.x.x
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed
> > keyboard-interactive/pam
> > > >> for
> > > >> > ouruser from x.x.x.x port 53332 ssh2
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request
> > for
> > > >> user
> > > >> > ouruser service ssh-connection method keyboard-interactive
> > [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2
> failures
> > 1
> > > >> > [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> > keyboard-interactive
> > > >> devs
> > > >> >  [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
> > > >> > user=ouruser devs= [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc:
> > devices
> > > >> 'pam'
> > > >> > [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> > auth2_challenge_start:
> > > >> > trying authentication method 'pam' [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
> > keyboard-interactive
> > > >> for
> > > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > > >> >
> > > >> >
> > > >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child
> > 536086.
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
> > > >> > /proc/self/oom_score_adj to 0
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5
> > out 5
> > > >> > newsock 5 pipe 7 sock 8
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets
> after
> > > >> > dupping: 4, 4
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x
> > port
> > > >> 53332
> > > >> > on 150.108.64.156 port 22 rdomain ""
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version
> > string
> > > >> > SSH-2.0-OpenSSH_8.4
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol
> > version
> > > >> > 2.0, remote software version OpenSSH_8.4
> > > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match:
> OpenSSH_8.4
> > pat
> > > >> > OpenSSH* compat 0x04000000
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support
> > disabled
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> > permanently_set_uid:
> > > >> 74/74
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> list_hostkey_types:
> > > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
> > > >> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
> > sent
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT
> > > >> received
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm:
> > > >> > curve25519-sha256 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key
> > algorithm:
> > > >> > ecdsa-sha2-nistp256 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> client->server
> > > >> cipher:
> > > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> server->client
> > > >> cipher:
> > > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none
> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> > curve25519-sha256
> > > >> > need=32 dh_need=32 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> > curve25519-sha256
> > > >> > need=32 dh_need=32 [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > > >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after
> > > >> 4294967296
> > > >> > blocks [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
> > sent
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
> > > >> SSH2_MSG_EXT_INFO
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
> > > >> SSH2_MSG_NEWKEYS
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS
> > > >> received
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after
> > > >> 4294967296
> > > >> > blocks [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done
> [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request
> > for
> > > >> user
> > > >> > ouruser service ssh-connection method none [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0
> failures
> > 0
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing
> > for
> > > >> > "ouruser"
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
> > PAM_RHOST
> > > >> to
> > > >> > "x.x.x.x"
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
> > PAM_TTY to
> > > >> > "ssh"
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request
> > for
> > > >> user
> > > >> > ouruser service ssh-connection method keyboard-interactive
> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1
> failures
> > 0
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> > keyboard-interactive
> > > >> devs
> > > >> >  [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge:
> > > >> > user=ouruser devs= [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc:
> > devices
> > > >> 'pam'
> > > >> > [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> > auth2_challenge_start:
> > > >> > trying authentication method 'pam' [preauth]
> > > >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
> > keyboard-interactive
> > > >> for
> > > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
> > > >> authentication
> > > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> > > >> user=ouruser
> > > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
> > > >> authentication
> > > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> > user=ouruser
> > > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
> > received for
> > > >> > user ouruser: 9 (Authentication service cannot retrieve
> > authentication
> > > >> info)
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM:
> Authentication
> > > >> failure
> > > >> > for ouruser from x.x.x.x
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed
> > keyboard-interactive/pam
> > > >> for
> > > >> > ouruser from x.x.x.x port 53332 ssh2
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request
> > for
> > > >> user
> > > >> > ouruser service ssh-connection method keyboard-interactive
> > [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2
> failures
> > 1
> > > >> > [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> > keyboard-interactive
> > > >> devs
> > > >> >  [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge:
> > > >> > user=ouruser devs= [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc:
> > devices
> > > >> 'pam'
> > > >> > [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> > auth2_challenge_start:
> > > >> > trying authentication method 'pam' [preauth]
> > > >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
> > keyboard-interactive
> > > >> for
> > > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
> > > >> >
> > > >> > With the NIS password the logs show this:
> > > >>
> > > >> Hi,
> > > >>
> > > >> did you drop what happened before or is this the only debug output
> > for
> > > >> the NIS password?
> > > >>
> > > >
> > > > The below here are just logs from /var/log/secure for the user that
> > > > successfully logs in with his/her NIS password.
> > > >
> > > > By "drop what happened before" do you mean the original log snip?
> Yes
> > I
> > > > removed those in an attempt to shorten the message content.
> > > >
> > > > > Feb  9 11:16:57 debug1: do_pam_account: called
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env
> > > >> strings 2
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: Postponed
> > > >> keyboard-interactive/pam
> > > >> > for cai from 150.108.68.26 port 53646 ssh2 [preauth]
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account:
> > called
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: Accepted
> > > >> keyboard-interactive/pam
> > > >> > for cai from 150.108.68.26 port 53646 ssh2
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
> > monitor_child_preauth:
> > > >> cai
> > > >> > has been authenticated by privileged process
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log:
> > child
> > > >> log
> > > >> > fd closed
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event:
> > unhandled
> > > >> > event 2
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
> > temporarily_use_uid:
> > > >> > 5879/200 (e=0/0)
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
> > ssh_gssapi_storecreds:
> > > >> Not
> > > >> > a GSSAPI mechanism
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support
> > disabled
> > > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing
> > > >> > credentials
> > > >> > Feb  9 11:16:57 ourserver systemd[536237]:
> > > >> pam_unix(systemd-user:session):
> > > >> > session opened for user cai(uid=5879) by (uid=0)
> > > >> >
> > > >> > What options should be set in /etc/ssh/sshd_config? Is sssd
> > necessary
> > > >> for
> > > >> > this to work with the FreeIPA password
> > > >>
> > > >
> > > >
> > > > Yes, SSSD must be configured and runnnig. ssd does appear to be
> > working
> > > > fine and in /etc/ipa/ca.crt and the service is running correctly:
> > > >
> > > > [domain/ourdomain.edu]
> > > >
> > > > id_provider = ipa
> > > > ipa_server_mode = True
> > > > ipa_server = ourdomain.edu
> > > > ipa_domain = ourdomain.edu
> > > > ipa_hostname = ourdomain.edu
> > > > auth_provider = ipa
> > > > chpass_provider = ipa
> > > > access_provider = ipa
> > > > cache_credentials = True
> > > > ldap_tls_cacert = /etc/ipa/ca.crt
> > > > krb5_store_password_if_offline = True
> > > > [sssd]
> > > > services = nss, pam, ifp, ssh, sudo
> > > >
> > > > domains = ourdomain.edu
> > > > [nss]
> > > > homedir_substring = /home
> > > > memcache_timeout = 600
> > > >
> > > > [ifp]
> > > > allowed_uids = ipaapi, root
> > > >
> > > > systemctl status sssd
> > > > * sssd.service - System Security Services Daemon
> > > >      Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled;
> > vendor
> > > > preset: enabled)
> > > >      Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1
> > weeks 3
> > > > days ago
> > > >
> > > >
> >
> > > _______________________________________________
> > > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> > > To unsubscribe send an email to
> > freeipa-users-leave(a)lists.fedorahosted.org
> > > Fedora Code of Conduct:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
> > > List Guidelines:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi...
> > > List Archives:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o...
> > _______________________________________________
> > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to
> > freeipa-users-leave(a)lists.fedorahosted.org
> > Fedora Code of Conduct:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
> > List Guidelines:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi...
> > List Archives:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o...
> > Do not reply to spam on the list, report it:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__pagure.io_fedora-2Di...
> >
>

[Sumit Bose]

On Tue, Feb 16, 2021 at 03:36:59PM -0500, Robert Kudyba via FreeIPA-users wrote:
>  One one of the test servers sssd fails to start correctly. Here are
> the
>  errors and their respective logs, with debug enabled.
>  freeipa-server-4.9.1-1.fc33.x86_64
>  
>  systemctl status sssd --no-pager -l
>  * sssd.service - System Security Services Daemon
>       Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor
>  preset: enabled)
>       Active: activating (start) since Tue 2021-02-16 15:15:32 EST; 1s ago
>     Main PID: 809519 (sssd)
>        Tasks: 2 (limit: 9484)
>       Memory: 10.9M
>          CPU: 998ms
>       CGroup: /system.slice/sssd.service
>               |-809519 /usr/sbin/sssd -i --logger=files
>               `-809520 /usr/libexec/sssd/sssd_be --domain implicit_files
>  --uid 0 --gid 0 --logger=files
>  
>  Feb 16 15:15:32 ourdomain.edu systemd[1]: Starting System Security Services
>  Daemon...
>  Feb 16 15:15:32 ourdomain.edu sssd[809519]: NSCD socket was detected and
>  seems to be configured to cache some of the databases controlled by SSSD
>  [passwd,group,netgroup,services]. It is recommended not to run NSCD in
>  parallel with SSSD, unless NSCD is configured not to cache these.
>  Feb 16 15:15:33 ourdomain.edu sssd[809519]: Starting up 
Hi,

that's odd, can you check with ps if nscd is running? Does
/var/run/nscd/socket exists?

>  
>  However:
>  systemctl status nscd
>  * nscd.service - Name Service Cache Daemon
>       Loaded: loaded (/usr/lib/systemd/system/nscd.service; disabled; vendor
>  preset: disabled)
>       Active: inactive (dead)
>  
>   /var/log/sssd/sssd_ourdomain.edu.log
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [ipa_init_dyndns] (0x0100):
>  Dynamic DNS updates are off.
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [setup_tls_config] (0x0020):
>  Unknown value for tls_reqcert 'never (or allow'. 
Can you check your sssd.conf if the 'tls_reqcert' option is set to
'never (or allow'? I guess you wanted to set it to 'never'.

HTH

bye,
Sumit

>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [ipa_init_misc] (0x0020):
> Unable
>  to get TLS options [22]: Invalid argument
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [sssm_ipa_init] (0x0020): Unable
>  to init IPA module [22]: Invalid argument
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_module_run_constructor]
>  (0x0010): Module [ipa] constructor failed [22]: Invalid argument
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_load_module] (0x0020):
>  Unable to create DP module.
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_target_init] (0x0010):
>  Unable to load module ipa
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_load_targets] (0x0020):
>  Unable to load target [id] [80]: Accessing a corrupted shared library.
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_init_done] (0x0020): Unable
>  to initialize DP targets [1432158209]: Internal Error
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_terminate_active_requests]
>  (0x0400): Terminating active data provider requests
>  (2021-02-16 15:06:30): [be[ourdomain.edu]] [sbus_server_symlink_remove]
>  (0x0400): Symlink removed [/var/lib/sss/pipes/private/sbus-dp_ourdomain.edu]
>  
>   /var/log/sssd/sssd.log
>  (2021-02-16 15:13:21): [sssd] [monitor_cleanup] (0x0010): Error removing
>  pidfile! (2 [No such file or directory])
>  (2021-02-16 15:13:22): [sssd] [server_setup] (0x0040): Starting with debug
>  level = 0x0070
>  (2021-02-16 15:13:22): [sssd] [svc_child_info] (0x0040): Child [809126] ('
>  ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
>  (2021-02-16 15:13:22): [sssd] [svc_child_info] (0x0040): Child [809127] ('
>  ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
>  (2021-02-16 15:13:24): [sssd] [svc_child_info] (0x0040): Child [809133] ('
>  ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [services_startup_timeout] (0x0020):
>  Providers did not start in time, forcing services startup!
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809137]
>  ('nss':'nss') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809139]
>  ('ifp':'ifp') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809141]
>  ('sudo':'sudo') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809138]
>  ('pam':'pam') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809140]
>  ('ssh':'ssh') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809142]
>  ('pac':'pac') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809143]
>  ('nss':'nss') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809144]
>  ('ifp':'ifp') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809145]
>  ('sudo':'sudo') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809147]
>  ('ssh':'ssh') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809146]
>  ('pam':'pam') exited with code [3]
>  (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809148]
>  ('pac':'pac') exited with code [3]
>  (2021-02-16 15:13:28): [sssd] [svc_child_info] (0x0040): Child [809153] ('
>  ourdomain.edu':'%BE_ourdomain.edu') exited with code [3]
>  (2021-02-16 15:13:28): [sssd] [monitor_restart_service] (0x0010): Process [
>  ourdomain.edu], definitely stopped!
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0040): Returned with: 1
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
>  [pac][809148]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
>  [pac][809148]: [No such process]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
>  [ssh][809147]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
>  [ssh][809147]: [No such process]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
>  [pam][809146]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
>  [pam][809146]: [No such process]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
>  [sudo][809145]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
>  [sudo][809145]: [No such process]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
>  [ifp][809144]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
>  [ifp][809144]: [No such process]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
>  [nss][809143]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill
>  [nss][809143]: [No such process]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating
>  [implicit_files][809125]
>  (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Child
>  [implicit_files] exited gracefully
>  (2021-02-16 15:13:28): [sssd] [monitor_cleanup] (0x0010): Error removing
>  pidfile! (2 [No such file or directory])
>  
>  In /var/log/sssd/sssd_pam.log,  /var/log/sssd/sssd_sudo.log,
>  /var/log/sssd/sssd_ifp.log, /var/log/sssd/sssd_pac.log  &
>  /var/log/sssd/sssd_nss.log all these logs end with:
>  (2021-02-16 15:08:30): [pam] [server_setup] (0x0040): Starting with debug
>  level = 0x0070
>  (2021-02-16 15:08:30): [pam] [sss_iface_connect_address] (0x0020):
>  check_file failed for [/var/lib/sss/pipes/private/sbus-dp_ourdomain.edu].
>  (2021-02-16 15:08:30): [pam] [sss_dp_init] (0x0010): Failed to connect to
>  backend server.
>  (2021-02-16 15:08:30): [pam] [sss_process_init] (0x0010): fatal error
>  setting up backend connector
>  (2021-02-16 15:08:30): [pam] [pam_process_init] (0x0010):
>  sss_process_init() failed
>  
>  /var/log/sssd/sssd_implicit_files.log
>  (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_set_cache_entry_attr]
>  (0x0040): Error: 17 (File exists)
>  (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_set_cache_entry_attr]
>  (0x0040): Error: 17 (File exists)
>  (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_store_new_group]
>  (0x0040): sysdb_add_group failed (while renaming group) for:
>  wireshark@implicit_files [1630].
>  (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_store_group] (0x0040):
>  Cache update failed: 17
>  (2021-02-16 15:12:24): [be[implicit_files]] [save_file_group] (0x0040):
>  Could not add group to cache
>  (2021-02-16 15:12:26): [be[implicit_files]] [orderly_shutdown] (0x0010):
>  SIGTERM: killing children
>  (2021-02-16 15:12:26): [be[implicit_files]] [orderly_shutdown] (0x0040):
>  Shutting down (status = 0)
>  (2021-02-16 15:12:26): [be[implicit_files]] [server_setup] (0x0040):
>  Starting with debug level = 0x0070
>  
>  However ipactl status does not show any errors:
>  Directory Service: RUNNING
>  krb5kdc Service: RUNNING
>  kadmin Service: RUNNING
>  named Service: RUNNING
>  httpd Service: RUNNING
>  ipa-custodia Service: RUNNING
>  pki-tomcatd Service: RUNNING
>  ipa-otpd Service: RUNNING
>  ipa-dnskeysyncd Service: RUNNING
>  ipa: INFO: The ipactl command was successful
>  
>  Anything sticking out?
>  
>  On Thu, Feb 11, 2021 at 4:51 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:
>  
>  > What is ourserver.edu? In order to log in using Kerberos/GSSAPI then the
>  >> machine acting as the server needs to be enrolled as an IPA client so it
>  >> has a keytab.
>  >> rob
>  >
>  >
>  > OK I added a Fedora server as a client. From ipa host-show
>  > client.ourserver.edu
>  >   Host name: client.ourserver.edu
>  >   Platform: x86_64
>  >   Operating system: 5.10.13-200.fc33.x86_64
>  >   Principal name: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU
>  >   Principal alias: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU
>  >   SSH public key fingerprint: SHA256:xxx(ecdsa-sha2-nistp256),
>  >                               SHA256:xxx (ssh-ed25519),
>  >                               SHA256:xxx (ssh-rsa)
>  >   Password: False
>  >   Keytab: True
>  >   Managed by: client.ourserver.edu
>  >
>  > ssh -k still fails.
>  >
>  > I created a new user in via ipa user-add and that user also fails even
>  > with a su -:
>  >
>  > su: user testuser does not exist or the user entry does not contain all
>  > the required fields
>  >
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey out after 4294967296
>  > blocks [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS sent
>  > [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: Sending SSH2_MSG_EXT_INFO
>  > [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: expecting SSH2_MSG_NEWKEYS
>  > [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS received
>  > [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey in after 4294967296
>  > blocks [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: KEX done [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user
>  > yume service ssh-connection method none [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 0 failures 0
>  > [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: Invalid user testuser from x.x.x.x
>  > port 36264
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: initializing for
>  > "testuser"
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_RHOST to
>  > "x.x.x.x"
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_TTY to
>  > "ssh"
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user
>  > testuser service ssh-connection method keyboard-interactive [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 1 failures 0
>  > [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: keyboard-interactive devs
>  >  [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge:
>  > user=testuser devs= [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: kbdint_alloc: devices
>  > 'pam' [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge_start:
>  > trying authentication method 'pam' [preauth]
>  > Feb 11 15:07:21 ourserver sshd[609424]: Postponed keyboard-interactive for
>  > invalid user testuser from x.x.x.x port 36264 ssh2 [preauth]
>  > Feb 11 15:07:24 ourserver sshd[609424]: Connection closed by invalid user
>  > testuser x.x.x.x port 36264 [preauth]
>  >
>  > are you sure the sssd.conf you have send earlier it the right one?
>  >> SSSD'S 'proxy_child' should be only be called if the proxy provider
> is
>  >> configured?
>  >>
>  >> bye,
>  >> Sumit
>  >>
>  >
>  >
>  > Sorry, you are correct. I'm testing between 2 test NIS servers to see if
>  > there's a difference in behavior but the same problem, ssh -k never works
>  > with the FreeIPA/Kerberos password
>  >
>  >
>  >>
>  >> > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth):
>  >> authentication
>  >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  >> > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received
>  >> for
>  >> > user ouruser: 7 (Authentication failure)
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication
>  >> failure
>  >> > for ouruser from x.x.x.x
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam
>  >> > for ouruser from x.x.x.x port 34160 ssh2
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1
>  >> > [preauth]
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive
>  >> devs
>  >> >  [preauth]
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge:
>  >> > user=ouruser devs= [preauth]
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices
>  >> > 'pam' [preauth]
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start:
>  >> > trying authentication method 'pam' [preauth]
>  >> > Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive
>  >> for
>  >> > ouruser from x.x.x.x port 34160 ssh2 [preauth]
>  >> >
>  >> > I verified the FreeIPA password in both the GUI and via ipa user-mod.
>  >> The
>  >> > only time the user is able to log in is using the NIS password.
>  >> ldapsearch
>  >> > -x -D and kinit username work successfully. klist displays the user
>  >> details
>  >> > correctly.
>  >> >
>  >> > I can see that the installation script edits /etc/ssh/sshd_config with:
>  >> > Include /etc/ssh/sshd_config.d/04-ipa.conf
>  >> >
>  >> > which has:
>  >> > PubkeyAuthentication yes
>  >> > KerberosAuthentication no
>  >> > GSSAPIAuthentication yes
>  >> > UsePAM yes
>  >> > ChallengeResponseAuthentication yes
>  >> > AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
>  >> > AuthorizedKeysCommandUser nobody
>  >> >
>  >> > When the NIS password is used successfully here are the server logs:
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method none [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0
>  >> > [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x
>  >> > matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for
>  >> > "ouruser"
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST
>  >> to
>  >> > "x.x.x.x"
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to
>  >> > "ssh"
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method gssapi-with-mic [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0
>  >> > [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for
>  >> user
>  >> > ouruser service ssh-connection method keyboard-interactive [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0
>  >> > [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive
>  >> devs
>  >> >  [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge:
>  >> > user=ouruser devs= [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices
>  >> > 'pam' [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start:
>  >> > trying authentication method 'pam' [preauth]
>  >> > Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive
>  >> for
>  >> > ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  >> > Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called
>  >> > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env
>  >> strings 2
>  >> > Feb 10 14:56:42 ourserver sshd[3085147]: Postponed
>  >> keyboard-interactive/pam
>  >> > for ouruser from x.x.x.x port 35046 ssh2 [preauth]
>  >> > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called
>  >> >
>  >> > I do see the error that sticks out is " Server host/
>  >> > ourserver.edu(a)ourserver.edu not found in Kerberos database" but we
> have
>  >> > students that log in from all over the world so do all clients need to
>  >> be
>  >> > added? iptables, firewalld, and nftables are off and disabled. No hbac
>  >> > rules:
>  >> > ipa hbacrule-find
>  >> > --------------------
>  >> > 2 HBAC rules matched
>  >> > --------------------
>  >> >   Rule name: allow_all
>  >> >   User category: all
>  >> >   Host category: all
>  >> >   Service category: all
>  >> >   Description: Allow all users to access any host from any host
>  >> >   Enabled: TRUE
>  >> >
>  >> >   Rule name: allow_systemd-user
>  >> >   User category: all
>  >> >   Host category: all
>  >> >   Description: Allow pam_systemd to run user@.service to create a
>  >> system
>  >> > user session
>  >> >   Enabled: TRUE
>  >> > ----------------------------
>  >> > Number of entries returned 2
>  >> >
>  >> > Am I missing something obvious to regulars?
>  >> >
>  >> >
>  >> > On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu&gt;
>  >> wrote:
>  >> >
>  >> > > On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users <
>  >> > > freeipa-users(a)lists.fedorahosted.org&gt; wrote:
>  >> > >
>  >> > >> On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via
>  >> FreeIPA-users
>  >> > >> wrote:
>  >> > >> > >
>  >> > >> > > looks like sshd is trying to read
>  >> /home/ouruser/.ssh/authorized_keys
>  >> > >> and
>  >> > >> > > is stuck. Can you read this file from the command line?
> Is it
>  >> e.g. on
>  >> > >> > > NFS which might not be properly mounted?
>  >> > >> > >
>  >> > >> > > Does it work if you skip pubkey authentication
>  >> > >> > >
>  >> > >> > >     ssh -o PubkeyAuthentication=no -vv -k
> ouruser@ourserver
>  >> > >> > >
>  >> > >> > > bye,
>  >> > >> > > Sumit
>  >> > >> > >
>  >> > >> >
>  >> > >> > Thanks for the suggestion. What happens is the NIS password
> works.
>  >> The
>  >> > >> > FreeIPA password, which I update with:
>  >> > >> > ipa user-mod ouruser --setattr "userpassword=xxxx",
> fails with the
>  >> below
>  >> > >> > errors/logs
>  >> > >> >
>  >> > >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child
>  >> 536086.
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  >> > >> > /proc/self/oom_score_adj to 0
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start
> in 5
>  >> out 5
>  >> > >> > newsock 5 pipe 7 sock 8
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets
> after
>  >> > >> > dupping: 4, 4
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from
> x.x.x.x
>  >> port
>  >> > >> 53332
>  >> > >> > on 150.108.64.156 port 22 rdomain ""
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local
> version
>  >> string
>  >> > >> > SSH-2.0-OpenSSH_8.4
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote
> protocol
>  >> version
>  >> > >> > 2.0, remote software version OpenSSH_8.4
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match:
> OpenSSH_8.4
>  >> pat
>  >> > >> > OpenSSH* compat 0x04000000
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux
> support
>  >> disabled
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  >> permanently_set_uid:
>  >> > >> 74/74
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> list_hostkey_types:
>  >> > >> >
> rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  >> > >> [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_KEXINIT
>  >> sent
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_KEXINIT
>  >> > >> received
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> algorithm:
>  >> > >> > curve25519-sha256 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host
> key
>  >> algorithm:
>  >> > >> > ecdsa-sha2-nistp256 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> client->server
>  >> > >> cipher:
>  >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression:
> none [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> server->client
>  >> > >> cipher:
>  >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression:
> none [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  >> curve25519-sha256
>  >> > >> > need=32 dh_need=32 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  >> curve25519-sha256
>  >> > >> > need=32 dh_need=32 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> > >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out
> after
>  >> > >> 4294967296
>  >> > >> > blocks [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_NEWKEYS
>  >> sent
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>  >> > >> SSH2_MSG_EXT_INFO
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> > >> SSH2_MSG_NEWKEYS
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_NEWKEYS
>  >> > >> received
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in
> after
>  >> > >> 4294967296
>  >> > >> > blocks [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done
> [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> userauth-request
>  >> for
>  >> > >> user
>  >> > >> > ouruser service ssh-connection method none [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0
> failures
>  >> 0
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM:
> initializing
>  >> for
>  >> > >> > "ouruser"
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  >> PAM_RHOST
>  >> > >> to
>  >> > >> > "x.x.x.x"
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  >> PAM_TTY to
>  >> > >> > "ssh"
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> userauth-request
>  >> for
>  >> > >> user
>  >> > >> > ouruser service ssh-connection method keyboard-interactive
>  >> [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1
> failures
>  >> 0
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  >> keyboard-interactive
>  >> > >> devs
>  >> > >> >  [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> auth2_challenge:
>  >> > >> > user=ouruser devs= [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> kbdint_alloc:
>  >> devices
>  >> > >> 'pam'
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  >> auth2_challenge_start:
>  >> > >> > trying authentication method 'pam' [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
>  >> keyboard-interactive
>  >> > >> for
>  >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  >> > >> authentication
>  >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  >> > >> user=ouruser
>  >> > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  >> > >> authentication
>  >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  >> user=ouruser
>  >> > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  >> received for
>  >> > >> > user ouruser: 9 (Authentication service cannot retrieve
>  >> authentication
>  >> > >> info)
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM:
> Authentication
>  >> > >> failure
>  >> > >> > for ouruser from x.x.x.x
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed
>  >> keyboard-interactive/pam
>  >> > >> for
>  >> > >> > ouruser from x.x.x.x port 53332 ssh2
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> userauth-request
>  >> for
>  >> > >> user
>  >> > >> > ouruser service ssh-connection method keyboard-interactive
>  >> [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2
> failures
>  >> 1
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>  >> keyboard-interactive
>  >> > >> devs
>  >> > >> >  [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> auth2_challenge:
>  >> > >> > user=ouruser devs= [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> kbdint_alloc:
>  >> devices
>  >> > >> 'pam'
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>  >> auth2_challenge_start:
>  >> > >> > trying authentication method 'pam' [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
>  >> keyboard-interactive
>  >> > >> for
>  >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> > >> >
>  >> > >> >
>  >> > >> > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child
>  >> 536086.
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set
>  >> > >> > /proc/self/oom_score_adj to 0
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start
> in 5
>  >> out 5
>  >> > >> > newsock 5 pipe 7 sock 8
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets
> after
>  >> > >> > dupping: 4, 4
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: Connection from
> x.x.x.x
>  >> port
>  >> > >> 53332
>  >> > >> > on 150.108.64.156 port 22 rdomain ""
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local
> version
>  >> string
>  >> > >> > SSH-2.0-OpenSSH_8.4
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote
> protocol
>  >> version
>  >> > >> > 2.0, remote software version OpenSSH_8.4
>  >> > >> > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match:
> OpenSSH_8.4
>  >> pat
>  >> > >> > OpenSSH* compat 0x04000000
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux
> support
>  >> disabled
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  >> permanently_set_uid:
>  >> > >> 74/74
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> list_hostkey_types:
>  >> > >> >
> rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
>  >> > >> [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_KEXINIT
>  >> sent
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_KEXINIT
>  >> > >> received
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> algorithm:
>  >> > >> > curve25519-sha256 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host
> key
>  >> algorithm:
>  >> > >> > ecdsa-sha2-nistp256 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> client->server
>  >> > >> cipher:
>  >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression:
> none [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
> server->client
>  >> > >> cipher:
>  >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression:
> none [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  >> curve25519-sha256
>  >> > >> > need=32 dh_need=32 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex:
>  >> curve25519-sha256
>  >> > >> > need=32 dh_need=32 [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> > >> > SSH2_MSG_KEX_ECDH_INIT [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out
> after
>  >> > >> 4294967296
>  >> > >> > blocks [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_NEWKEYS
>  >> sent
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending
>  >> > >> SSH2_MSG_EXT_INFO
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting
>  >> > >> SSH2_MSG_NEWKEYS
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> SSH2_MSG_NEWKEYS
>  >> > >> received
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in
> after
>  >> > >> 4294967296
>  >> > >> > blocks [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done
> [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> userauth-request
>  >> for
>  >> > >> user
>  >> > >> > ouruser service ssh-connection method none [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0
> failures
>  >> 0
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM:
> initializing
>  >> for
>  >> > >> > "ouruser"
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  >> PAM_RHOST
>  >> > >> to
>  >> > >> > "x.x.x.x"
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting
>  >> PAM_TTY to
>  >> > >> > "ssh"
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> userauth-request
>  >> for
>  >> > >> user
>  >> > >> > ouruser service ssh-connection method keyboard-interactive
>  >> [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1
> failures
>  >> 0
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  >> keyboard-interactive
>  >> > >> devs
>  >> > >> >  [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> auth2_challenge:
>  >> > >> > user=ouruser devs= [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
> kbdint_alloc:
>  >> devices
>  >> > >> 'pam'
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: debug1:
>  >> auth2_challenge_start:
>  >> > >> > trying authentication method 'pam' [preauth]
>  >> > >> > Feb  9 11:10:35 ourserver sshd[536086]: Postponed
>  >> keyboard-interactive
>  >> > >> for
>  >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth):
>  >> > >> authentication
>  >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  >> > >> user=ouruser
>  >> > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  >> > >> authentication
>  >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  >> user=ouruser
>  >> > >> > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth):
>  >> received for
>  >> > >> > user ouruser: 9 (Authentication service cannot retrieve
>  >> authentication
>  >> > >> info)
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM:
> Authentication
>  >> > >> failure
>  >> > >> > for ouruser from x.x.x.x
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: Failed
>  >> keyboard-interactive/pam
>  >> > >> for
>  >> > >> > ouruser from x.x.x.x port 53332 ssh2
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> userauth-request
>  >> for
>  >> > >> user
>  >> > >> > ouruser service ssh-connection method keyboard-interactive
>  >> [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2
> failures
>  >> 1
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>  >> keyboard-interactive
>  >> > >> devs
>  >> > >> >  [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> auth2_challenge:
>  >> > >> > user=ouruser devs= [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
> kbdint_alloc:
>  >> devices
>  >> > >> 'pam'
>  >> > >> > [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: debug1:
>  >> auth2_challenge_start:
>  >> > >> > trying authentication method 'pam' [preauth]
>  >> > >> > Feb  9 11:10:41 ourserver sshd[536086]: Postponed
>  >> keyboard-interactive
>  >> > >> for
>  >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth]
>  >> > >> >
>  >> > >> > With the NIS password the logs show this:
>  >> > >>
>  >> > >> Hi,
>  >> > >>
>  >> > >> did you drop what happened before or is this the only debug
> output
>  >> for
>  >> > >> the NIS password?
>  >> > >>
>  >> > >
>  >> > > The below here are just logs from /var/log/secure for the user that
>  >> > > successfully logs in with his/her NIS password.
>  >> > >
>  >> > > By "drop what happened before" do you mean the original log
> snip? Yes
>  >> I
>  >> > > removed those in an attempt to shorten the message content.
>  >> > >
>  >> > > > Feb  9 11:16:57 debug1: do_pam_account: called
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM
> env
>  >> > >> strings 2
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: Postponed
>  >> > >> keyboard-interactive/pam
>  >> > >> > for cai from 150.108.68.26 port 53646 ssh2 [preauth]
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
> do_pam_account:
>  >> called
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: Accepted
>  >> > >> keyboard-interactive/pam
>  >> > >> > for cai from 150.108.68.26 port 53646 ssh2
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>  >> monitor_child_preauth:
>  >> > >> cai
>  >> > >> > has been authenticated by privileged process
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
> monitor_read_log:
>  >> child
>  >> > >> log
>  >> > >> > fd closed
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event:
>  >> unhandled
>  >> > >> > event 2
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>  >> temporarily_use_uid:
>  >> > >> > 5879/200 (e=0/0)
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1:
>  >> ssh_gssapi_storecreds:
>  >> > >> Not
>  >> > >> > a GSSAPI mechanism
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid:
> 0/0
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux
> support
>  >> disabled
>  >> > >> > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM:
> establishing
>  >> > >> > credentials
>  >> > >> > Feb  9 11:16:57 ourserver systemd[536237]:
>  >> > >> pam_unix(systemd-user:session):
>  >> > >> > session opened for user cai(uid=5879) by (uid=0)
>  >> > >> >
>  >> > >> > What options should be set in /etc/ssh/sshd_config? Is sssd
>  >> necessary
>  >> > >> for
>  >> > >> > this to work with the FreeIPA password
>  >> > >>
>  >> > >
>  >> > >
>  >> > > Yes, SSSD must be configured and runnnig. ssd does appear to be
>  >> working
>  >> > > fine and in /etc/ipa/ca.crt and the service is running correctly:
>  >> > >
>  >> > > [domain/ourdomain.edu]
>  >> > >
>  >> > > id_provider = ipa
>  >> > > ipa_server_mode = True
>  >> > > ipa_server = ourdomain.edu
>  >> > > ipa_domain = ourdomain.edu
>  >> > > ipa_hostname = ourdomain.edu
>  >> > > auth_provider = ipa
>  >> > > chpass_provider = ipa
>  >> > > access_provider = ipa
>  >> > > cache_credentials = True
>  >> > > ldap_tls_cacert = /etc/ipa/ca.crt
>  >> > > krb5_store_password_if_offline = True
>  >> > > [sssd]
>  >> > > services = nss, pam, ifp, ssh, sudo
>  >> > >
>  >> > > domains = ourdomain.edu
>  >> > > [nss]
>  >> > > homedir_substring = /home
>  >> > > memcache_timeout = 600
>  >> > >
>  >> > > [ifp]
>  >> > > allowed_uids = ipaapi, root
>  >> > >
>  >> > > systemctl status sssd
>  >> > > * sssd.service - System Security Services Daemon
>  >> > >      Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled;
>  >> vendor
>  >> > > preset: enabled)
>  >> > >      Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1
>  >> weeks 3
>  >> > > days ago
>  >> > >
>  >> > >
>  >>
>  >> > _______________________________________________
>  >> > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  >> > To unsubscribe send an email to
>  >> freeipa-users-leave(a)lists.fedorahosted.org
>  >> > Fedora Code of Conduct:
>  >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
>  >> > List Guidelines:
>  >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi...
>  >> > List Archives:
>  >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o...
>  >> _______________________________________________
>  >> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  >> To unsubscribe send an email to
>  >> freeipa-users-leave(a)lists.fedorahosted.org
>  >> Fedora Code of Conduct:
>  >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
>  >> List Guidelines:
>  >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi...
>  >> List Archives:
>  >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o...
>  >> Do not reply to spam on the list, report it:
>  >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__pagure.io_fedora-2Di...
>  >>
>  > 
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Robert Kudyba]

>  that's odd, can you check with ps if nscd is running? 

It is not.


>  Does /var/run/nscd/socket exists?
>  
Yes it does, I then deleted it.


>  > (2021-02-16 15:06:30): [be[ourdomain.edu]] [setup_tls_config]
> (0x0020):
>  > Unknown value for tls_reqcert 'never (or allow'.
>
>  Can you check your sssd.conf if the 'tls_reqcert' option is set to
>  'never (or allow'? I guess you wanted to set it to 'never'.
>  
Nice catch. Bad copy/paste for me as I tried to enable more verbose debug
logs. Now sssd at least starts.

So new users created in IPA/LDAP can now ssh -k and su correctly. The debug
ssh server logs look like this:
Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: userauth-request for user
tuser service ssh-connection method keyboard-interactive [preauth]
Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: attempt 3 failures 1
[preauth]
Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: keyboard-interactive devs
 [preauth]
Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: auth2_challenge: user=tuser
devs= [preauth]
Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Feb 17 09:44:48 ourdomainsshd[1016078]: Postponed keyboard-interactive for
tuser from x.x.x.x port 48614 ssh2 [preauth]
Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.xuser=tuser
Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:auth): received for
user tuser: 12 (Authentication token is no longer valid; new one required)
Feb 17 09:44:52 ourdomainsshd[1016085]: debug1: do_pam_account: called
Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:account): User info
message: Password expired. Change your password now.
Feb 17 09:44:52 ourdomainsshd[1016085]: pam_unix(sshd:chauthtok): user
"tuser" does not exist in /etc/passwd
Feb 17 09:44:52 ourdomainsshd[1016078]: Postponed keyboard-interactive/pam
for tuser from x.x.x.x port 48614 ssh2 [preauth]

ssh -vvv has this:
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,
sk-ssh-ed25519(a)openssh.com
,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
sk-ecdsa-sha2-nistp256(a)openssh.com,
webauthn-sk-ecdsa-sha2-nistp256(a)openssh.com&gt;
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: start over, passed a different list
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: preferred password
debug3: authmethod_lookup password
debug3: remaining preferred:
debug3: authmethod_is_enabled password
debug1: Next authentication method: password

As for an imported NIS user, only the NIS password continues to allow
logins. I also stopped ypserv and ypbind to see if there was any difference
and there is not.

Here are the ssh server logs for the NIS password that succeeds:
Feb 17 09:34:56 ourdomain sshd [1015632]: debug1: userauth-request for user
ouruser service ssh-connection method password [preauth]
Feb 17 09:34:56 ourdomain sshd [1015632]: debug1: attempt 2 failures 1
[preauth]
Feb 17 09:34:56 ourdomain sshd [1015632]: pam_sss(sshd:auth):
authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
user=ouruser
Feb 17 09:34:56 ourdomain sshd [1015632]: pam_sss(sshd:auth): received for
user ouruser: 9 (Authentication service cannot retrieve authentication info)
Feb 17 09:34:58 ourdomain sshd [1015632]: debug1: PAM: password
authentication failed for ouruser: Authentication failure
Feb 17 09:34:58 ourdomain sshd [1015632]: Failed password for ouruser from
x.x.x.x port 48578 ssh2
Feb 17 09:35:07 ourdomain sshd [1015632]: debug1: userauth-request for user
ouruser service ssh-connection method password [preauth]
Feb 17 09:35:07 ourdomain sshd [1015632]: debug1: attempt 3 failures 2
[preauth]
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1:* PAM: password
authentication accepted for ouruser*
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: do_pam_account: called
Feb 17 09:35:08 ourdomain sshd [1015632]: Accepted password for ouruser
from x.x.x.x port 48578 ssh2
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: monitor_child_preauth:
ouruser has been authenticated by privileged process
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: monitor_read_log: child
log fd closed
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: audit_event: unhandled
event 2
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: temporarily_use_uid:
5920/200 (e=0/0)
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1:* ssh_gssapi_storecreds:
Not a GSSAPI mechanism*
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: restore_uid: 0/0
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: SELinux support disabled
Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: PAM: establishing
credentials
Feb 17 09:35:08 ourdomain systemd[1015655]: pam_unix(systemd-user:session):
session opened for user ouruser (uid=5920) by (uid=0)
Feb 17 09:36:38 ourdomain sshd [1015632]: pam_unix(sshd:session): session
opened for user ouruser (uid=5920) by (uid=0)
Feb 17 09:36:39 ourdomain sshd [1015632]: User child is on pid 1015696
Feb 17 09:36:39 ourdomain sshd [1015696]: debug1: PAM: establishing
credentials

What sticks out to me is what I bolded, that PAM password auth works and
that this is NOT a GSSAPI mechanism. Am I missing a step?

Here are the related logs from sssd_ourdomain.edu.log:
(2021-02-17 10:45:25): [be[ourdomain.edu]*] [dp_get_account_info_send]
(0x0200): Got request for
[0x1][BE_REQ_USER][name=pam_usertype_non_existent:@ourdomain.edu
<pam_usertype_non_existent%3A(a)ourdomain.edu&gt;]*
(2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_attach_req] (0x0400): DP
Request [Account #70]: New request. Flags [0x0001].
(2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_attach_req] (0x0400): Number
of active DP request: 1
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_next_base]
(0x0400): Searching for users with base [cn=accounts,dc=ourdomain,dc=edu]
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_ext_step]
(0x0400): calling ldap_search_ext with
[(&(uid=pam_usertype_non_existent:)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))][cn=accounts,dc=ourdomain,dc=edu].
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_op_finished]
(0x0400): Search result: Success(0), no errmsg set
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_process]
(0x0400): *Search for users, returned 0 results*.
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_next_base]
(0x0400): Searching for users with base [cn=trusts,dc=ourdomain,dc=edu]
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_ext_step]
(0x0400): calling ldap_search_ext with
[(&(&(uid=pam_usertype_non_existent:)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))(objectClass=ipaIDObject))][cn=trusts,dc=ourdomain,dc=edu].
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_op_finished]
(0x0400): Search result: Success(0), no errmsg set
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_process]
(0x0400): *Search for users, returned 0 results*.
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_search_by_name] (0x0400):
No such entry
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_delete_user] (0x0400):
Error: 2 (No such file or directory)
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_search_by_name] (0x0400):
No such entry
(2021-02-17 10:45:25): [be[ourdomain.edu]]
[ipa_id_get_account_info_orig_done] (0x0080): Object not found, ending
request
(2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_done] (0x0400): DP
Request [Account #70]: Request handler finished [0]: Success
(2021-02-17 10:45:25): [be[ourdomain.edu]] [_dp_req_recv] (0x0400): DP
Request [Account #70]: Receiving request data.
(2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_destructor] (0x0400): DP
Request [Account #70]: Request removed.
(2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_destructor] (0x0400):
Number of active DP request: 0
(2021-02-17 10:45:25): [be[ourdomain.edu]] [sbus_issue_request_done]
(0x0400): sssd.dataprovider.getAccountInfo: Success

So the log "*name=pam_usertype_non_existent*:" seems rather pertinent. I
see from another thread
https://freeipa-users.redhat.narkive.com/GYwxOWHr/understanding-the-migra...,
I also used --setattr userpassword={crypt}yourencryptedpass.

ipa user-show returns "Kerberos keys available: False" on the migrated NIS
users. I tested changing passwords of users via the GUI and then ipa
user-show returns "Kerberos keys available: True"

Anything stick out here?

[Sumit Bose]

On Wed, Feb 17, 2021 at 10:58:54AM -0500, Robert Kudyba via FreeIPA-users wrote:
>  >
>  > that's odd, can you check with ps if nscd is running?
>  
>  
>  It is not.
>  
>  
>  > Does /var/run/nscd/socket exists?
>  >
>  
>  Yes it does, I then deleted it.
>  
>  
>  > > (2021-02-16 15:06:30): [be[ourdomain.edu]] [setup_tls_config] (0x0020):
>  > > Unknown value for tls_reqcert 'never (or allow'.
>  >
>  > Can you check your sssd.conf if the 'tls_reqcert' option is set to
>  > 'never (or allow'? I guess you wanted to set it to 'never'.
>  >
>  
>  Nice catch. Bad copy/paste for me as I tried to enable more verbose debug
>  logs. Now sssd at least starts.
>  
>  So new users created in IPA/LDAP can now ssh -k and su correctly. The debug
>  ssh server logs look like this:
>  Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: userauth-request for user
>  tuser service ssh-connection method keyboard-interactive [preauth]
>  Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: attempt 3 failures 1
>  [preauth]
>  Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: keyboard-interactive devs
>   [preauth]
>  Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: auth2_challenge: user=tuser
>  devs= [preauth]
>  Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: kbdint_alloc: devices 'pam'
>  [preauth]
>  Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Feb 17 09:44:48 ourdomainsshd[1016078]: Postponed keyboard-interactive for
>  tuser from x.x.x.x port 48614 ssh2 [preauth]
>  Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.xuser=tuser
>  Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:auth): received for
>  user tuser: 12 (Authentication token is no longer valid; new one required)
>  Feb 17 09:44:52 ourdomainsshd[1016085]: debug1: do_pam_account: called
>  Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:account): User info
>  message: Password expired. Change your password now.
>  Feb 17 09:44:52 ourdomainsshd[1016085]: pam_unix(sshd:chauthtok): user
>  "tuser" does not exist in /etc/passwd
>  Feb 17 09:44:52 ourdomainsshd[1016078]: Postponed keyboard-interactive/pam
>  for tuser from x.x.x.x port 48614 ssh2 [preauth]
>  
>  ssh -vvv has this:
>  debug2: pubkey_prepare: done
>  debug3: send packet: type 5
>  debug3: receive packet: type 7
>  debug1: SSH2_MSG_EXT_INFO received
>  debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,
>  sk-ssh-ed25519(a)openssh.com
>
> ,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
>  sk-ecdsa-sha2-nistp256(a)openssh.com,
>  webauthn-sk-ecdsa-sha2-nistp256(a)openssh.com&gt;
>  debug3: receive packet: type 6
>  debug2: service_accept: ssh-userauth
>  debug1: SSH2_MSG_SERVICE_ACCEPT received
>  debug3: send packet: type 50
>  debug3: receive packet: type 51
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: start over, passed a different list
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: preferred password
>  debug3: authmethod_lookup password
>  debug3: remaining preferred:
>  debug3: authmethod_is_enabled password
>  debug1: Next authentication method: password
>  
>  As for an imported NIS user, only the NIS password continues to allow
>  logins. I also stopped ypserv and ypbind to see if there was any difference
>  and there is not.
>  
>  Here are the ssh server logs for the NIS password that succeeds:
>  Feb 17 09:34:56 ourdomain sshd [1015632]: debug1: userauth-request for user
>  ouruser service ssh-connection method password [preauth]
>  Feb 17 09:34:56 ourdomain sshd [1015632]: debug1: attempt 2 failures 1
>  [preauth]
>  Feb 17 09:34:56 ourdomain sshd [1015632]: pam_sss(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>  user=ouruser
>  Feb 17 09:34:56 ourdomain sshd [1015632]: pam_sss(sshd:auth): received for
>  user ouruser: 9 (Authentication service cannot retrieve authentication info)
>  Feb 17 09:34:58 ourdomain sshd [1015632]: debug1: PAM: password
>  authentication failed for ouruser: Authentication failure
>  Feb 17 09:34:58 ourdomain sshd [1015632]: Failed password for ouruser from
>  x.x.x.x port 48578 ssh2
>  Feb 17 09:35:07 ourdomain sshd [1015632]: debug1: userauth-request for user
>  ouruser service ssh-connection method password [preauth]
>  Feb 17 09:35:07 ourdomain sshd [1015632]: debug1: attempt 3 failures 2
>  [preauth]
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1:* PAM: password
>  authentication accepted for ouruser*
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: do_pam_account: called
>  Feb 17 09:35:08 ourdomain sshd [1015632]: Accepted password for ouruser
>  from x.x.x.x port 48578 ssh2
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: monitor_child_preauth:
>  ouruser has been authenticated by privileged process
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: monitor_read_log: child
>  log fd closed
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: audit_event: unhandled
>  event 2
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: temporarily_use_uid:
>  5920/200 (e=0/0)
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1:* ssh_gssapi_storecreds:
>  Not a GSSAPI mechanism*
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: restore_uid: 0/0
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: SELinux support disabled
>  Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: PAM: establishing
>  credentials
>  Feb 17 09:35:08 ourdomain systemd[1015655]: pam_unix(systemd-user:session):
>  session opened for user ouruser (uid=5920) by (uid=0)
>  Feb 17 09:36:38 ourdomain sshd [1015632]: pam_unix(sshd:session): session
>  opened for user ouruser (uid=5920) by (uid=0)
>  Feb 17 09:36:39 ourdomain sshd [1015632]: User child is on pid 1015696
>  Feb 17 09:36:39 ourdomain sshd [1015696]: debug1: PAM: establishing
>  credentials
>  
>  What sticks out to me is what I bolded, that PAM password auth works and
>  that this is NOT a GSSAPI mechanism. Am I missing a step?
>  
>  Here are the related logs from sssd_ourdomain.edu.log:
>  (2021-02-17 10:45:25): [be[ourdomain.edu]*] [dp_get_account_info_send]
>  (0x0200): Got request for
>  [0x1][BE_REQ_USER][name=pam_usertype_non_existent:@ourdomain.edu
>  <pam_usertype_non_existent%3A(a)ourdomain.edu&gt;]*
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_attach_req] (0x0400): DP
>  Request [Account #70]: New request. Flags [0x0001].
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_attach_req] (0x0400): Number
>  of active DP request: 1
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_next_base]
>  (0x0400): Searching for users with base [cn=accounts,dc=ourdomain,dc=edu]
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_ext_step]
>  (0x0400): calling ldap_search_ext with
>
> [(&(uid=pam_usertype_non_existent:)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))][cn=accounts,dc=ourdomain,dc=edu].
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_op_finished]
>  (0x0400): Search result: Success(0), no errmsg set
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_process]
>  (0x0400): *Search for users, returned 0 results*.
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_next_base]
>  (0x0400): Searching for users with base [cn=trusts,dc=ourdomain,dc=edu]
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_ext_step]
>  (0x0400): calling ldap_search_ext with
>
> [(&(&(uid=pam_usertype_non_existent:)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))(objectClass=ipaIDObject))][cn=trusts,dc=ourdomain,dc=edu].
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_op_finished]
>  (0x0400): Search result: Success(0), no errmsg set
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_process]
>  (0x0400): *Search for users, returned 0 results*.
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_search_by_name] (0x0400):
>  No such entry
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_delete_user] (0x0400):
>  Error: 2 (No such file or directory)
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_search_by_name] (0x0400):
>  No such entry
>  (2021-02-17 10:45:25): [be[ourdomain.edu]]
>  [ipa_id_get_account_info_orig_done] (0x0080): Object not found, ending
>  request
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_done] (0x0400): DP
>  Request [Account #70]: Request handler finished [0]: Success
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [_dp_req_recv] (0x0400): DP
>  Request [Account #70]: Receiving request data.
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_destructor] (0x0400): DP
>  Request [Account #70]: Request removed.
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_destructor] (0x0400):
>  Number of active DP request: 0
>  (2021-02-17 10:45:25): [be[ourdomain.edu]] [sbus_issue_request_done]
>  (0x0400): sssd.dataprovider.getAccountInfo: Success
>  
>  So the log "*name=pam_usertype_non_existent*:" seems rather pertinent. I
>  see from another thread
>  https://freeipa-users.redhat.narkive.com/GYwxOWHr/understanding-the-migra...,
>  I also used --setattr userpassword={crypt}yourencryptedpass.
>  
>  ipa user-show returns "Kerberos keys available: False" on the migrated NIS
>  users. I tested changing passwords of users via the GUI and then ipa
>  user-show returns "Kerberos keys available: True" 
Hi,

have you enabled the migration mode with

    ipa config-mod --enable-migration=True

With this authentication with SSSD should fall back to LDAP
authentication if the Kerberos keys are not available and this would
trigger a creation of the Kerberos keys for the user trying to log in.

bye,
Sumit

>  
>  Anything stick out here? 
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Robert Kudyba]

>  have you enabled the migration mode with
>
>      ipa config-mod --enable-migration=True
>  
I've tried it with True and False. At what point should this be changed to
False?


>  With this authentication with SSSD should fall back to LDAP
>  authentication if the Kerberos keys are not available and this would
>  trigger a creation of the Kerberos keys for the user trying to log in.
>  
The fallback appears to be NIS.

The good news is the user can log in to the GUI, i.e.i
https://ourserver/ipa/ui and change their password but I do see this error
in the Apache error log which sounds like this issue
<https://pagure.io/freeipa/issue/7032>;:
[Wed Mar 03 13:53:07.526386 2021] [wsgi:error] [pid 16169:tid 16554]
[remote xx.xx.xx.xx:63098] ipa: DEBUG: Destroyed connection
context.ldap2_140265125387520
[Wed Mar 03 13:53:07.563873 2021] [:warn] [pid 16174:tid 16239] [client
xx.xx.xx.xx:63098] failed to set perms (3140) on file
(/run/ipa/ccaches/admin(a)OURDOMAIN.EDU-jhCS0U)!, referer:
https://ourdomain.edu/ipa/ui/
[Wed Mar 03 13:53:07.564720 2021] [wsgi:error] [pid 16170:tid 16545]
[remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI wsgi_dispatch.__call__:
[Wed Mar 03 13:53:07.564838 2021] [wsgi:error] [pid 16170:tid 16545]
[remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI jsonserver_session.__call__:
[Wed Mar 03 13:53:07.570164 2021] [:warn] [pid 16174:tid 16285] [client
xx.xx.xx.xx:63076] failed to set perms (3140) on file
(/run/ipa/ccaches/adminOURDOMAIN.EDU-jhCS0U)!, referer:
https://ourdomain.edu/ipa/ui/

Now from ssh -vvv -k this is what we see:
debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: start over, passed a different list
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic




*debug1: Next authentication method: gssapi-with-micdebug1: Unspecified GSS
failure.  Minor code may provide more informationNo Kerberos credentials
available (default cache: KEYRING:persistent:0)debug1: Unspecified GSS
failure.  Minor code may provide more informationNo Kerberos credentials
available (default cache: KEYRING:persistent:0)*
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /ouruser/.ssh/id_rsa RSA
SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug1: Trying private key: /ouruser/.ssh/id_dsa
debug3: no such identity: /ouruser/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /ouruser/.ssh/id_ecdsa
debug3: no such identity: /ouruser/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /ouruser/.ssh/id_ecdsa_sk
debug3: no such identity: /ouruser/.ssh/id_ecdsa_sk: No such file or
directory
debug1: Trying private key: /ouruser/.ssh/id_ed25519
debug3: no such identity: /ouruser/.ssh/id_ed25519: No such file or
directory
debug1: Trying private key: /ouruser/.ssh/id_ed25519_sk
debug3: no such identity: /ouruser/.ssh/id_ed25519_sk: No such file or
directory
debug1: Trying private key: /ouruser/.ssh/id_xmss
debug3: no such identity: /ouruser/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication

Is this a clue?
Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KEYRING:persistent:0)

From the ssh server logs, set to debug:
Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 0 failures 0
[preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: initializing for
"ouruser"
Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting PAM_RHOST to
"x.x.x.x"
Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting PAM_TTY to "ssh"
Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for user
ouruser service ssh-connection method publickey [preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 1 failures 0
[preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth_pubkey: test pkalg
rsa-sha2-512 pkblob RSA SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
[preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid:
6915/200 (e=0/0)
Mar  3 14:00:46 ourserver sshd[79161]: debug1: trying public key file
/home/ouruser/.ssh/authorized_keys
Mar  3 14:00:46 ourserver sshd[79161]: debug1: Could not open authorized
keys '/home/ouruser/.ssh/authorized_keys': No such file or directory
Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid: 99/99
(e=0/0)
Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid: 99/99
(e=0/0)
Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
Mar  3 14:00:46 ourserver sshd[79161]: Failed publickey for ouruser from
x.x.x.x port 40248 ssh2: RSA
SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 2 failures 1
[preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: keyboard-interactive devs
 [preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Mar  3 14:00:46 ourserver sshd[79161]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Mar  3 14:00:46 ourserver sshd[79161]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 40248 ssh2 [preauth]
Mar  3 14:00:50 ourserver sshd[79168]: pam_unix(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
Mar  3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
Mar  3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth): received for
user ouruser: 9 (Authentication service cannot retrieve authentication info)
Mar  3 14:00:52 ourserver sshd[79161]: error: PAM: Authentication failure
for ouruser from x.x.x.x
Mar  3 14:00:52 ourserver sshd[79161]: Failed keyboard-interactive/pam for
ouruser from x.x.x.x port 40248 ssh2
Mar  3 14:00:52 ourserver sshd[79161]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Mar  3 14:00:52 ourserver sshd[79161]: debug1: attempt 3 failures 2
[preauth]
Mar  3 14:00:52 ourserver sshd[79161]: debug1: keyboard-interactive devs
 [preauth]
Mar  3 14:00:52 ourserver sshd[79161]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Mar  3 14:00:52 ourserver sshd[79161]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Mar  3 14:00:52 ourserver sshd[79161]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]

Here are the server logs on a successful login with the NIS password:
Mar  3 14:06:09 ourserver sshd[79292]: Accepted keyboard-interactive/pam
for ouruser from xx.xx.xx.xx port 40252 ssh2
Mar  3 14:06:09 ourserver sshd[79292]: debug1: monitor_child_preauth:
ouruser has been authenticated by privileged process
Mar  3 14:06:09 ourserver sshd[79292]: debug1: monitor_read_log: child log
fd closed
Mar  3 14:06:09 ourserver sshd[79292]: debug1: audit_event: unhandled event
2
Mar  3 14:06:09 ourserver sshd[79292]: debug1: temporarily_use_uid:
6915/200 (e=0/0)
Mar  3 14:06:09 ourserver sshd[79292]: debug1: *ssh_gssapi_storecreds: Not
a GSSAPI mechanism*
Mar  3 14:06:09 ourserver sshd[79292]: debug1: restore_uid: 0/0
Mar  3 14:06:09 ourserver sshd[79292]: debug1: SELinux support disabled
Mar  3 14:06:09 ourserver sshd[79292]: debug1: PAM: establishing credentials
Mar  3 14:06:09 ourserver systemd[79307]: pam_unix(systemd-user:session):
session opened for user ouruser(uid=6915) by (uid=0)
Mar  3 14:06:10 ourserver sshd[79292]: pam_unix(sshd:session): session
opened for user ouruser(uid=6915) by (uid=0)
Mar  3 14:06:10 ourserver sshd[79292]: User child is on pid 79320
Mar  3 14:06:10 ourserver sshd[79320]: debug1: PAM: establishing credentials

So it clearly says it's "Not a GSSAPI mechanism".

/etc/nsswitch.conf (which is a symbolic link to
/etc/authselect/nsswitch.conf)
passwd:     sss files systemd
group:      sss files systemd
netgroup:   sss files
automount:  sss files
services:   sss files
sudoers:    files sss
shadow:     files nis
hosts:      files nis mdns4_minimal [NOTFOUND=return] dns myhostname
mymachines

And /etc/authselect/user-nsswitch.conf has
passwd:      files nis systemd
shadow:     files nis
group:       files nis systemd
hosts:      files nis mdns4_minimal [NOTFOUND=return] dns myhostname
mymachines

Why is the Kerberos login failing?

[Robert Kudyba]

I believe we've made some progress but not quite there yet. Just to recap,
any NEW user created via CLI or GUI can connect via ssh. All imported NIS
users can only log in with their NIS password. I change the user's password
in the UI and check the Password checkbox in User authentication type and
click Save. I successfully added a client:
ipa host-add-managedby --hosts=ourdomain.edu client.ourdomain.edu
  Host name: client.ourdomain.edu
  Platform: x86_64
  Operating system: 5.10.9-201.fc33.x86_64
  Principal name: host/client.ourdomain.edu(a)OURDOMAIN.EDU
  Principal alias: host/client.ourdomain.edu(a)OURDOMAIN.EDU
  Managed by: client.ourdomain.edu, ourdomain.edu
-------------------------
Number of members added 1
-------------------------
[root@ourdomain ~]# ipa-getkeytab -s ourdomain.edu -p host/
client.ourdomain.edu -k /tmp/client.keytab
Keytab successfully retrieved and stored in: /tmp/client.keytab

Based on this SF discussion
<https://serverfault.com/questions/609086/freeipa-command-line-tools-do-no...;,
I changed: in  /etc/krb5.conf
default_ccache_name = FILE:/tmp/krb5cc_%{uid}

Before the change in the krb5.log file I would see:
LOOKING_UP_SERVER: authtime 0, etypes {rep=UNSUPPORTED:(0)} host/
ourdomain.edu(a)OURDOMAIN.EDU for nfs/client.ourdomain.edu(a)OUDOMAIN.EDU,
Server not found in Kerberos database

Now I see:
Mar 11 13:38:28 ourdomain.edu u krb5kdc[369141](info): AS_REQ (6 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17),
aes256-cts-hmac-sha384-192(20), camellia256-cts-cmac(26),
aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25)}) 150.108.64.55:
NEEDED_PREAUTH: host/client. ourdomain.edu .edu(a)OURDOMAIN.EDU for krbtgt/
OURDOMAIN.EDU @ OURDOMAIN.EDU , Additional pre-authentication required
Mar 11 13:38:28 ourdomain.edu krb5kdc[369141](info): closing down fd 11
Mar 11 13:38:28 ourdomain.edu krb5kdc[369144](info): preauth (spake) verify
failure: Preauthentication failed

I had run kinit admin a few times but it seems to expire with the old 1969
date but another kinit admin fixes that:
klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: host/ourdomain.edu(a)OURDOMAIN.EDU

Valid starting     Expires            Service principal
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:

[root@ourdomain ~]# kinit admin
Password for admin(a)OURDOMAIN.EDU:

[root@ourdomain ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin(a)OURDOMAIN.EDU

Valid starting     Expires            Service principal
03/11/21 13:38:27  03/12/21 13:38:24  krbtgt/OURDOMAIN.EDU(a)OURDOMAIN.EDU

ipa -vv pwpolicy-show global_policy
ipa: INFO: Request: {
    "id": 0,
    "method": "pwpolicy_show/1",
    "params": [
        [
            "global_policy"
        ],
        {
            "version": "2.240"
        }
    ]
}
ipa: INFO: Response: {
    "error": null,
    "id": 0,
    "principal": &quot;admin(a)OURDOMAIN.EDU&quot;,
    "result": {
        "result": {
            "cn": [
                "global_policy"
            ],
            "dn": "cn=global_policy,cn=OURDOMAIN.EDU
,cn=kerberos,dc=ourdomain,dc=edu",
            "krbmaxpwdlife": [
                "90"
            ],
            "krbminpwdlife": [
                "1"
            ],
            "krbpwdfailurecountinterval": [
                "60"
            ],
            "krbpwdhistorylength": [
                "0"
            ],
            "krbpwdlockoutduration": [
                "600"
            ],
            "krbpwdmaxfailure": [
                "6"
            ],
            "krbpwdmindiffchars": [
                "0"
            ],
            "krbpwdminlength": [
                "8"
            ]
        },
        "summary": null,
        "value": "global_policy"
    },
    "version": "4.9.2"
}
  Group: global_policy
  Max lifetime (days): 90
  Min lifetime (hours): 1
  History size: 0
  Character classes: 0
  Min length: 8
  Max failures: 6
  Failure reset interval: 60
  Lockout duration: 600

Alas ssh -k fails again with these SSH server logs:
Mar 11 13:38:58 ourdomain sshd[372719]: debug1: attempt 2 failures 1
[preauth]
Mar 11 13:38:58 ourdomain sshd[372719]: debug1: keyboard-interactive devs
 [preauth]
Mar 11 13:38:58 ourdomain sshd[372719]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Mar 11 13:38:58 ourdomain sshd[372719]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Mar 11 13:38:58 ourdomain sshd[372719]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Mar 11 13:38:58 ourdomain sshd[372719]: Postponed keyboard-interactive for
ouruser from xx.xx.xx.xx port 41834 ssh2 [preauth]
Mar 11 13:39:01 ourdomain sshd[372726]: pam_unix(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=xx.xx.xx.xx
 user=ouruser
Mar 11 13:39:01 ourdomain sshd[372726]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=xx.xx.xx.xx user=ouruser
Mar 11 13:39:01 ourdomain sshd[372726]: pam_sss(sshd:auth): received for
user ouruser: 9 (Authentication service cannot retrieve authentication info)
Mar 11 13:39:04 ourdomain sshd[372719]: error: PAM: Authentication failure
for ouruser from xx.xx.xx.xx

And with ssh -vv I still see:

debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: KEYRING:persistent:0)
*debug1: Unspecified GSS failure.  Minor code may provide more information*
*No Kerberos credentials available (default cache: KEYRING:persistent:0)*
debug2: we did not send a packet, disable method
debug1: Next authentication method: publickey
debug1: Offering public key: /root/.ssh/id_rsa RSA
SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
debug2: we sent a publickey packet, wait for reply
*debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive*

What am I missing?

On Wed, Mar 3, 2021 at 2:24 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:

>  have you enabled the migration mode with
> >
> >     ipa config-mod --enable-migration=True
> >
>
>  I've tried it with True and False. At what point should this be changed to
>  False?
>
>
> > With this authentication with SSSD should fall back to LDAP
> > authentication if the Kerberos keys are not available and this would
> > trigger a creation of the Kerberos keys for the user trying to log in.
> >
>
>  The fallback appears to be NIS.
>
>  The good news is the user can log in to the GUI, i.e.i
>  https://ourserver/ipa/ui and change their password but I do see this
>  error in the Apache error log which sounds like this issue
>  <https://pagure.io/freeipa/issue/7032>;:
>  [Wed Mar 03 13:53:07.526386 2021] [wsgi:error] [pid 16169:tid 16554]
>  [remote xx.xx.xx.xx:63098] ipa: DEBUG: Destroyed connection
>  context.ldap2_140265125387520
>  [Wed Mar 03 13:53:07.563873 2021] [:warn] [pid 16174:tid 16239] [client
>  xx.xx.xx.xx:63098] failed to set perms (3140) on file
>  (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU-jhCS0U)!, referer:
>  https://ourdomain.edu/ipa/ui/
>  [Wed Mar 03 13:53:07.564720 2021] [wsgi:error] [pid 16170:tid 16545]
>  [remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI wsgi_dispatch.__call__:
>  [Wed Mar 03 13:53:07.564838 2021] [wsgi:error] [pid 16170:tid 16545]
>  [remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI jsonserver_session.__call__:
>  [Wed Mar 03 13:53:07.570164 2021] [:warn] [pid 16174:tid 16285] [client
>  xx.xx.xx.xx:63076] failed to set perms (3140) on file
>  (/run/ipa/ccaches/adminOURDOMAIN.EDU-jhCS0U)!, referer:
>  https://ourdomain.edu/ipa/ui/
>
>  Now from ssh -vvv -k this is what we see:
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: start over, passed a different list
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
>  debug3: authmethod_lookup gssapi-with-mic
>  debug3: remaining preferred: publickey,keyboard-interactive,password
>  debug3: authmethod_is_enabled gssapi-with-mic
>
>
>
>
>  *debug1: Next authentication method: gssapi-with-micdebug1: Unspecified
>  GSS failure.  Minor code may provide more informationNo Kerberos
>  credentials available (default cache: KEYRING:persistent:0)debug1:
>  Unspecified GSS failure.  Minor code may provide more informationNo
>  Kerberos credentials available (default cache: KEYRING:persistent:0)*
>  debug2: we did not send a packet, disable method
>  debug3: authmethod_lookup publickey
>  debug3: remaining preferred: keyboard-interactive,password
>  debug3: authmethod_is_enabled publickey
>  debug1: Next authentication method: publickey
>  debug1: Offering public key: /ouruser/.ssh/id_rsa RSA
>  SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
>  debug3: send packet: type 50
>  debug2: we sent a publickey packet, wait for reply
>  debug3: receive packet: type 51
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug1: Trying private key: /ouruser/.ssh/id_dsa
>  debug3: no such identity: /ouruser/.ssh/id_dsa: No such file or directory
>  debug1: Trying private key: /ouruser/.ssh/id_ecdsa
>  debug3: no such identity: /ouruser/.ssh/id_ecdsa: No such file or directory
>  debug1: Trying private key: /ouruser/.ssh/id_ecdsa_sk
>  debug3: no such identity: /ouruser/.ssh/id_ecdsa_sk: No such file or
>  directory
>  debug1: Trying private key: /ouruser/.ssh/id_ed25519
>  debug3: no such identity: /ouruser/.ssh/id_ed25519: No such file or
>  directory
>  debug1: Trying private key: /ouruser/.ssh/id_ed25519_sk
>  debug3: no such identity: /ouruser/.ssh/id_ed25519_sk: No such file or
>  directory
>  debug1: Trying private key: /ouruser/.ssh/id_xmss
>  debug3: no such identity: /ouruser/.ssh/id_xmss: No such file or directory
>  debug2: we did not send a packet, disable method
>  debug3: authmethod_lookup keyboard-interactive
>  debug3: remaining preferred: password
>  debug3: authmethod_is_enabled keyboard-interactive
>  debug1: Next authentication
>
>  Is this a clue?
>  Unspecified GSS failure.  Minor code may provide more information
>  No Kerberos credentials available (default cache: KEYRING:persistent:0)
>
>  From the ssh server logs, set to debug:
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 0 failures 0
>  [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: initializing for
>  "ouruser"
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting PAM_RHOST to
>  "x.x.x.x"
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting PAM_TTY to
>  "ssh"
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for user
>  ouruser service ssh-connection method publickey [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 1 failures 0
>  [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth_pubkey: test pkalg
>  rsa-sha2-512 pkblob RSA SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
>  [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid:
>  6915/200 (e=0/0)
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: trying public key file
>  /home/ouruser/.ssh/authorized_keys
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: Could not open authorized
>  keys '/home/ouruser/.ssh/authorized_keys': No such file or directory
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid: 99/99
>  (e=0/0)
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid: 99/99
>  (e=0/0)
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
>  Mar  3 14:00:46 ourserver sshd[79161]: Failed publickey for ouruser from
>  x.x.x.x port 40248 ssh2: RSA
>  SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 2 failures 1
>  [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: keyboard-interactive devs
>   [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: kbdint_alloc: devices 'pam'
>  [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Mar  3 14:00:46 ourserver sshd[79161]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 40248 ssh2 [preauth]
>  Mar  3 14:00:50 ourserver sshd[79168]: pam_unix(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
>  Mar  3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  Mar  3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth): received for
>  user ouruser: 9 (Authentication service cannot retrieve authentication info)
>  Mar  3 14:00:52 ourserver sshd[79161]: error: PAM: Authentication failure
>  for ouruser from x.x.x.x
>  Mar  3 14:00:52 ourserver sshd[79161]: Failed keyboard-interactive/pam for
>  ouruser from x.x.x.x port 40248 ssh2
>  Mar  3 14:00:52 ourserver sshd[79161]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Mar  3 14:00:52 ourserver sshd[79161]: debug1: attempt 3 failures 2
>  [preauth]
>  Mar  3 14:00:52 ourserver sshd[79161]: debug1: keyboard-interactive devs
>   [preauth]
>  Mar  3 14:00:52 ourserver sshd[79161]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Mar  3 14:00:52 ourserver sshd[79161]: debug1: kbdint_alloc: devices 'pam'
>  [preauth]
>  Mar  3 14:00:52 ourserver sshd[79161]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>
>  Here are the server logs on a successful login with the NIS password:
>  Mar  3 14:06:09 ourserver sshd[79292]: Accepted keyboard-interactive/pam
>  for ouruser from xx.xx.xx.xx port 40252 ssh2
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: monitor_child_preauth:
>  ouruser has been authenticated by privileged process
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: monitor_read_log: child log
>  fd closed
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: audit_event: unhandled
>  event 2
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: temporarily_use_uid:
>  6915/200 (e=0/0)
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: *ssh_gssapi_storecreds:
>  Not a GSSAPI mechanism*
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: restore_uid: 0/0
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: SELinux support disabled
>  Mar  3 14:06:09 ourserver sshd[79292]: debug1: PAM: establishing
>  credentials
>  Mar  3 14:06:09 ourserver systemd[79307]: pam_unix(systemd-user:session):
>  session opened for user ouruser(uid=6915) by (uid=0)
>  Mar  3 14:06:10 ourserver sshd[79292]: pam_unix(sshd:session): session
>  opened for user ouruser(uid=6915) by (uid=0)
>  Mar  3 14:06:10 ourserver sshd[79292]: User child is on pid 79320
>  Mar  3 14:06:10 ourserver sshd[79320]: debug1: PAM: establishing
>  credentials
>
>  So it clearly says it's "Not a GSSAPI mechanism".
>
>  /etc/nsswitch.conf (which is a symbolic link to
>  /etc/authselect/nsswitch.conf)
>  passwd:     sss files systemd
>  group:      sss files systemd
>  netgroup:   sss files
>  automount:  sss files
>  services:   sss files
>  sudoers:    files sss
>  shadow:     files nis
>  hosts:      files nis mdns4_minimal [NOTFOUND=return] dns myhostname
>  mymachines
>
>  And /etc/authselect/user-nsswitch.conf has
>  passwd:      files nis systemd
>  shadow:     files nis
>  group:       files nis systemd
>  hosts:      files nis mdns4_minimal [NOTFOUND=return] dns myhostname
>  mymachines
>
>  Why is the Kerberos login failing?
>

[Rob Crittenden]

Robert Kudyba via FreeIPA-users wrote:
>  I believe we've made some progress but not quite there yet. Just
> to
>  recap, any NEW user created via CLI or GUI can connect via ssh. All
>  imported NIS users can only log in with their NIS password. I change the
>  user's password in the UI and check the Password checkbox in User
>  authentication type and click Save. I successfully added a client:
>  ipa host-add-managedby --hosts=ourdomain.edu <http://ourdomain.edu>
>  client.ourdomain.edu <http://client.ourdomain.edu>
>    Host name: client.ourdomain.edu <http://client.ourdomain.edu>
>    Platform: x86_64
>    Operating system: 5.10.9-201.fc33.x86_64
>    Principal name: host/client.ourdomain.edu(a)OURDOMAIN.EDU
>  <mailto:[e-mail]>
>    Principal alias: host/client.ourdomain.edu(a)OURDOMAIN.EDU
>  <mailto:[e-mail]>
>    Managed by: client.ourdomain.edu <http://client.ourdomain.edu>;,
>  ourdomain.edu <http://ourdomain.edu>
>  -------------------------
>  Number of members added 1
>  -------------------------
>  [root@ourdomain ~]# ipa-getkeytab -s ourdomain.edu
>  <http://ourdomain.edu> -p host/client.ourdomain.edu
>  <http://client.ourdomain.edu> -k /tmp/client.keytab
>  Keytab successfully retrieved and stored in: /tmp/client.keytab 
This is why SSSD isn't working. SSSD uses the host keytab in
/etc/krb5.keytab and you invalidated it with the above command.

>  Based on this SF discussion
>
> <https://serverfault.com/questions/609086/freeipa-command-line-tools-do-no...;,
>  I changed: in  /etc/krb5.conf
>  default_ccache_name = FILE:/tmp/krb5cc_%{uid} 
I don't think this is necessary.

>  
>  Before the change in the krb5.log file I would see:
>  LOOKING_UP_SERVER: authtime 0, etypes {rep=UNSUPPORTED:(0)}
>  host/ourdomain.edu(a)OURDOMAIN.EDU <mailto:[e-mail]>
>  for nfs/client.ourdomain.edu(a)OUDOMAIN.EDU
>  <mailto:[e-mail]>, Server not found in Kerberos
>  database
>  
>  Now I see:
>  Mar 11 13:38:28 ourdomain.edu <http://ourdomain.edu> u
>  krb5kdc[369141](info): AS_REQ (6 etypes {aes256-cts-hmac-sha1-96(18),
>  aes128-cts-hmac-sha1-96(17), aes256-cts-hmac-sha384-192(20),
>  camellia256-cts-cmac(26), aes128-cts-hmac-sha256-128(19),
>  camellia128-cts-cmac(25)}) 150.108.64.55 <http://150.108.64.55>;:
>  NEEDED_PREAUTH: host/client. ourdomain.edu <http://ourdomain.edu>
>  .edu(a)OURDOMAIN.EDU <mailto:[e-mail]> for krbtgt/ OURDOMAIN.EDU
>  <http://OURDOMAIN.EDU> @ OURDOMAIN.EDU <http://OURDOMAIN.EDU> ,
>  Additional pre-authentication required
>  Mar 11 13:38:28 ourdomain.edu <http://ourdomain.edu>
>  krb5kdc[369141](info): closing down fd 11
>  Mar 11 13:38:28 ourdomain.edu <http://ourdomain.edu>
>  krb5kdc[369144](info): preauth (spake) verify failure: Preauthentication
>  failed
>  
>  I had run kinit admin a few times but it seems to expire with the old
>  1969 date but another kinit admin fixes that:
>  klist
>  Ticket cache: FILE:/tmp/krb5cc_0
>  Default principal: host/ourdomain.edu(a)OURDOMAIN.EDU
>  <mailto:[e-mail]>
>  
>  Valid starting     Expires            Service principal
>  12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY: 
kinit doesn't change the credentials that SSSD has (or had).

>  
>  [root@ourdomain ~]# kinit admin
>  Password for admin(a)OURDOMAIN.EDU <mailto:[e-mail]>:
>  
>  [root@ourdomain ~]# klist
>  Ticket cache: FILE:/tmp/krb5cc_0
>  Default principal: admin(a)OURDOMAIN.EDU <mailto:[e-mail]>
>  
>  Valid starting     Expires            Service principal
>  03/11/21 13:38:27  03/12/21 13:38:24  krbtgt/OURDOMAIN.EDU(a)OURDOMAIN.EDU
>  <mailto:[e-mail]>
>  
>  ipa -vv pwpolicy-show global_policy
>  ipa: INFO: Request: {
>      "id": 0,
>      "method": "pwpolicy_show/1",
>      "params": [
>          [
>              "global_policy"
>          ],
>          {
>              "version": "2.240"
>          }
>      ]
>  }
>  ipa: INFO: Response: {
>      "error": null,
>      "id": 0,
>      "principal": &quot;admin(a)OURDOMAIN.EDU
> <mailto:[e-mail]>",
>      "result": {
>          "result": {
>              "cn": [
>                  "global_policy"
>              ],
>              "dn": "cn=global_policy,cn=OURDOMAIN.EDU
>  <http://OURDOMAIN.EDU>,cn=kerberos,dc=ourdomain,dc=edu";,
>              "krbmaxpwdlife": [
>                  "90"
>              ],
>              "krbminpwdlife": [
>                  "1"
>              ],
>              "krbpwdfailurecountinterval": [
>                  "60"
>              ],
>              "krbpwdhistorylength": [
>                  "0"
>              ],
>              "krbpwdlockoutduration": [
>                  "600"
>              ],
>              "krbpwdmaxfailure": [
>                  "6"
>              ],
>              "krbpwdmindiffchars": [
>                  "0"
>              ],
>              "krbpwdminlength": [
>                  "8"
>              ]
>          },
>          "summary": null,
>          "value": "global_policy"
>      },
>      "version": "4.9.2"
>  }
>    Group: global_policy
>    Max lifetime (days): 90
>    Min lifetime (hours): 1
>    History size: 0
>    Character classes: 0
>    Min length: 8
>    Max failures: 6
>    Failure reset interval: 60
>    Lockout duration: 600
>  
>  Alas ssh -k fails again with these SSH server logs:
>  Mar 11 13:38:58 ourdomain sshd[372719]: debug1: attempt 2 failures 1
>  [preauth]
>  Mar 11 13:38:58 ourdomain sshd[372719]: debug1: keyboard-interactive
>  devs  [preauth]
>  Mar 11 13:38:58 ourdomain sshd[372719]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Mar 11 13:38:58 ourdomain sshd[372719]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Mar 11 13:38:58 ourdomain sshd[372719]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Mar 11 13:38:58 ourdomain sshd[372719]: Postponed keyboard-interactive
>  for ouruser from xx.xx.xx.xx port 41834 ssh2 [preauth]
>  Mar 11 13:39:01 ourdomain sshd[372726]: pam_unix(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>  rhost=xx.xx.xx.xx  user=ouruser
>  Mar 11 13:39:01 ourdomain sshd[372726]: pam_sss(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>  rhost=xx.xx.xx.xx user=ouruser
>  Mar 11 13:39:01 ourdomain sshd[372726]: pam_sss(sshd:auth): received for
>  user ouruser: 9 (Authentication service cannot retrieve authentication info)
>  Mar 11 13:39:04 ourdomain sshd[372719]: error: PAM: Authentication
>  failure for ouruser from xx.xx.xx.xx
>  
>  And with ssh -vv I still see:
>  
>  debug1: Next authentication method: gssapi-with-mic
>  debug1: Unspecified GSS failure.  Minor code may provide more information
>  No Kerberos credentials available (default cache: KEYRING:persistent:0)
>  *debug1: Unspecified GSS failure.  Minor code may provide more information*
>  *No Kerberos credentials available (default cache: KEYRING:persistent:0)*
>  debug2: we did not send a packet, disable method
>  debug1: Next authentication method: publickey
>  debug1: Offering public key: /root/.ssh/id_rsa RSA
>  SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
>  debug2: we sent a publickey packet, wait for reply
>  *debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive*
>  
>  What am I missing?
>  
>  On Wed, Mar 3, 2021 at 2:24 PM Robert Kudyba <rkudyba(a)fordham.edu
>  <mailto:[e-mail]>> wrote:
>  
>          have you enabled the migration mode with
>  
>              ipa config-mod --enable-migration=True
>  
>  
>      I've tried it with True and False. At what point should this be
>      changed to False?
>       
>  
>          With this authentication with SSSD should fall back to LDAP
>          authentication if the Kerberos keys are not available and this would
>          trigger a creation of the Kerberos keys for the user trying to
>          log in.
>  
>  
>      The fallback appears to be NIS. 
>  
>      The good news is the user can log in to the GUI, i.e.i
>      https://ourserver/ipa/ui and change their password but I do see this
>      error in the Apache error log which sounds like this issue
>      <https://pagure.io/freeipa/issue/7032>;:
>      [Wed Mar 03 13:53:07.526386 2021] [wsgi:error] [pid 16169:tid 16554]
>      [remote xx.xx.xx.xx:63098] ipa: DEBUG: Destroyed connection
>      context.ldap2_140265125387520
>      [Wed Mar 03 13:53:07.563873 2021] [:warn] [pid 16174:tid 16239]
>      [client xx.xx.xx.xx:63098] failed to set perms (3140) on file
>      (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU-jhCS0U)!, referer:
>      https://ourdomain.edu/ipa/ui/
>      [Wed Mar 03 13:53:07.564720 2021] [wsgi:error] [pid 16170:tid 16545]
>      [remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI wsgi_dispatch.__call__:
>      [Wed Mar 03 13:53:07.564838 2021] [wsgi:error] [pid 16170:tid 16545]
>      [remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI jsonserver_session.__call__:
>      [Wed Mar 03 13:53:07.570164 2021] [:warn] [pid 16174:tid 16285]
>      [client xx.xx.xx.xx:63076] failed to set perms (3140) on file
>      (/run/ipa/ccaches/adminOURDOMAIN.EDU-jhCS0U)!, referer:
>      https://ourdomain.edu/ipa/ui/
>  
>      Now from ssh -vvv -k this is what we see:
>      debug1: Authentications that can continue:
>      publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>      debug3: start over, passed a different list
>      publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>      debug3: preferred
>      gssapi-with-mic,publickey,keyboard-interactive,password
>      debug3: authmethod_lookup gssapi-with-mic
>      debug3: remaining preferred: publickey,keyboard-interactive,password
>      debug3: authmethod_is_enabled gssapi-with-mic
>      *debug1: Next authentication method: gssapi-with-mic
>      debug1: Unspecified GSS failure.  Minor code may provide more
>      information
>      No Kerberos credentials available (default cache: KEYRING:persistent:0)
>      debug1: Unspecified GSS failure.  Minor code may provide more
>      information
>      No Kerberos credentials available (default cache: KEYRING:persistent:0)*
>      debug2: we did not send a packet, disable method
>      debug3: authmethod_lookup publickey
>      debug3: remaining preferred: keyboard-interactive,password
>      debug3: authmethod_is_enabled publickey
>      debug1: Next authentication method: publickey
>      debug1: Offering public key: /ouruser/.ssh/id_rsa RSA
>      SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
>      debug3: send packet: type 50
>      debug2: we sent a publickey packet, wait for reply
>      debug3: receive packet: type 51
>      debug1: Authentications that can continue:
>      publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>      debug1: Trying private key: /ouruser/.ssh/id_dsa
>      debug3: no such identity: /ouruser/.ssh/id_dsa: No such file or
>      directory
>      debug1: Trying private key: /ouruser/.ssh/id_ecdsa
>      debug3: no such identity: /ouruser/.ssh/id_ecdsa: No such file or
>      directory
>      debug1: Trying private key: /ouruser/.ssh/id_ecdsa_sk
>      debug3: no such identity: /ouruser/.ssh/id_ecdsa_sk: No such file or
>      directory
>      debug1: Trying private key: /ouruser/.ssh/id_ed25519
>      debug3: no such identity: /ouruser/.ssh/id_ed25519: No such file or
>      directory
>      debug1: Trying private key: /ouruser/.ssh/id_ed25519_sk
>      debug3: no such identity: /ouruser/.ssh/id_ed25519_sk: No such file
>      or directory
>      debug1: Trying private key: /ouruser/.ssh/id_xmss
>      debug3: no such identity: /ouruser/.ssh/id_xmss: No such file or
>      directory
>      debug2: we did not send a packet, disable method
>      debug3: authmethod_lookup keyboard-interactive
>      debug3: remaining preferred: password
>      debug3: authmethod_is_enabled keyboard-interactive
>      debug1: Next authentication
>  
>      Is this a clue?
>      Unspecified GSS failure.  Minor code may provide more information
>      No Kerberos credentials available (default cache: KEYRING:persistent:0)
>       
>      From the ssh server logs, set to debug:
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 0 failures 0
>      [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: initializing for
>      "ouruser"
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting
>      PAM_RHOST to "x.x.x.x"
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting PAM_TTY
>      to "ssh"
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for
>      user ouruser service ssh-connection method publickey [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 1 failures 0
>      [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth_pubkey: test
>      pkalg rsa-sha2-512 pkblob RSA
>      SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid:
>      6915/200 (e=0/0)
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: trying public key
>      file /home/ouruser/.ssh/authorized_keys
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: Could not open
>      authorized keys '/home/ouruser/.ssh/authorized_keys': No such file
>      or directory
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid:
>      99/99 (e=0/0)
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid:
>      99/99 (e=0/0)
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0
>      Mar  3 14:00:46 ourserver sshd[79161]: Failed publickey for ouruser
>      from x.x.x.x port 40248 ssh2: RSA
>      SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for
>      user ouruser service ssh-connection method keyboard-interactive
>      [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: attempt 2 failures 1
>      [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: keyboard-interactive
>      devs  [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: auth2_challenge:
>      user=ouruser devs= [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1: kbdint_alloc: devices
>      'pam' [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: debug1:
>      auth2_challenge_start: trying authentication method 'pam' [preauth]
>      Mar  3 14:00:46 ourserver sshd[79161]: Postponed
>      keyboard-interactive for ouruser from x.x.x.x port 40248 ssh2 [preauth]
>      Mar  3 14:00:50 ourserver sshd[79168]: pam_unix(sshd:auth):
>      authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>      rhost=x.x.x.x  user=ouruser
>      Mar  3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth):
>      authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>      rhost=x.x.x.x user=ouruser
>      Mar  3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth): received
>      for user ouruser: 9 (Authentication service cannot retrieve
>      authentication info)
>      Mar  3 14:00:52 ourserver sshd[79161]: error: PAM: Authentication
>      failure for ouruser from x.x.x.x
>      Mar  3 14:00:52 ourserver sshd[79161]: Failed
>      keyboard-interactive/pam for ouruser from x.x.x.x port 40248 ssh2
>      Mar  3 14:00:52 ourserver sshd[79161]: debug1: userauth-request for
>      user ouruser service ssh-connection method keyboard-interactive
>      [preauth]
>      Mar  3 14:00:52 ourserver sshd[79161]: debug1: attempt 3 failures 2
>      [preauth]
>      Mar  3 14:00:52 ourserver sshd[79161]: debug1: keyboard-interactive
>      devs  [preauth]
>      Mar  3 14:00:52 ourserver sshd[79161]: debug1: auth2_challenge:
>      user=ouruser devs= [preauth]
>      Mar  3 14:00:52 ourserver sshd[79161]: debug1: kbdint_alloc: devices
>      'pam' [preauth]
>      Mar  3 14:00:52 ourserver sshd[79161]: debug1:
>      auth2_challenge_start: trying authentication method 'pam' [preauth]
>  
>      Here are the server logs on a successful login with the NIS password:
>      Mar  3 14:06:09 ourserver sshd[79292]: Accepted
>      keyboard-interactive/pam for ouruser from xx.xx.xx.xx port 40252 ssh2
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1:
>      monitor_child_preauth: ouruser has been authenticated by privileged
>      process
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1: monitor_read_log:
>      child log fd closed
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1: audit_event:
>      unhandled event 2
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1: temporarily_use_uid:
>      6915/200 (e=0/0)
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1:
>      *ssh_gssapi_storecreds: Not a GSSAPI mechanism*
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1: restore_uid: 0/0
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1: SELinux support disabled
>      Mar  3 14:06:09 ourserver sshd[79292]: debug1: PAM: establishing
>      credentials
>      Mar  3 14:06:09 ourserver systemd[79307]:
>      pam_unix(systemd-user:session): session opened for user
>      ouruser(uid=6915) by (uid=0)
>      Mar  3 14:06:10 ourserver sshd[79292]: pam_unix(sshd:session):
>      session opened for user ouruser(uid=6915) by (uid=0)
>      Mar  3 14:06:10 ourserver sshd[79292]: User child is on pid 79320
>      Mar  3 14:06:10 ourserver sshd[79320]: debug1: PAM: establishing
>      credentials
>  
>      So it clearly says it's "Not a GSSAPI mechanism".
>  
>      /etc/nsswitch.conf (which is a symbolic link to
>      /etc/authselect/nsswitch.conf)
>      passwd:     sss files systemd
>      group:      sss files systemd
>      netgroup:   sss files
>      automount:  sss files
>      services:   sss files
>      sudoers:    files sss
>      shadow:     files nis
>      hosts:      files nis mdns4_minimal [NOTFOUND=return] dns myhostname
>      mymachines
>  
>      And/etc/authselect/user-nsswitch.conf has
>      passwd:      files nis systemd
>      shadow:     files nis
>      group:       files nis systemd
>      hosts:      files nis mdns4_minimal [NOTFOUND=return] dns myhostname
>      mymachines
>  
>      Why is the Kerberos login failing?
>  
>  
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>

[Robert Kudyba]

On Thu, Mar 11, 2021 at 2:31 PM Rob Crittenden <rcritten(a)redhat.com&gt; wrote:

>  Robert Kudyba via FreeIPA-users wrote:
>  I believe we've made some progress but not quite there yet. Just to recap,
>  any NEW user created via CLI or GUI can connect via ssh. All imported NIS
>  users can only log in with their NIS password. I change the user's password
>  in the UI and check the Password checkbox in User authentication type and
>  click Save. I successfully added a client: ipa host-add-managedby --hosts=
>  ourdomain.edu client.ourdomain.edu Host name: client.ourdomain.edu
>  Platform: x86_64 Operating system: 5.10.9-201.fc33.x86_64 Principal name:
>  host/client.ourdomain.edu(a)OURDOMAIN.EDU Principal alias: host/
>  client.ourdomain.edu(a)OURDOMAIN.EDU Managed by: client.ourdomain.edu,
>  ourdomain.edu ------------------------- Number of members added 1
>  ------------------------- [root@ourdomain ~]# ipa-getkeytab -s
>  ourdomain.edu -p host/ client.ourdomain.edu -k /tmp/client.keytab
>  > Keytab successfully retrieved and stored in: /tmp/client.keytab
>
>  This is why SSSD isn't working. SSSD uses the host keytab in
>  /etc/krb5.keytab and you invalidated it with the above command.
>  
OK what do I need to do to fix this? I got this from
https://docs.fedoraproject.org/en-US/Fedora/18/html/FreeIPA_Guide/Install...
(which I realize is old),


>  > Based on this SF discussion
>  > <
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__serverfault.com_ques...
>  >,
>  > I changed: in  /etc/krb5.conf
>  > default_ccache_name = FILE:/tmp/krb5cc_%{uid}
>
>  I don't think this is necessary.
>  
OK Thanks for letting me know.

Are these SSH logs helpful:

NEEDED_PREAUTH: host/client. ourdomain.edu .edu(a)OURDOMAIN.EDU for krbtgt/
OURDOMAIN.EDU <http://ourdomain.edu/> @ OURDOMAIN.EDU
<http://ourdomain.edu/>;, Additional pre-authentication required Mar 11
13:38:28 ourdomain.edu krb5kdc[369141](info): closing down fd 11 Mar 11
13:38:28 ourdomain.edu krb5kdc[369144](info): preauth (spake) verify
failure: Preauthentication failed

Does this have to do with your comment above about SSSD not working?

[Rob Crittenden]

Robert Kudyba wrote:
>  
>  
>  On Thu, Mar 11, 2021 at 2:31 PM Rob Crittenden <rcritten(a)redhat.com
>  <mailto:[e-mail]>> wrote:
>  
>      Robert Kudyba via FreeIPA-users wrote:
>      I believe we've made some progress but not quite there yet. Just to
>      recap, any NEW user created via CLI or GUI can connect via ssh. All
>      imported NIS users can only log in with their NIS password. I change
>      the user's password in the UI and check the Password checkbox in
>      User authentication type and click Save. I successfully added a
>      client: ipa host-add-managedby --hosts=ourdomain.edu
>      <http://ourdomain.edu> client.ourdomain.edu
>      <http://client.ourdomain.edu/>Host name: client.ourdomain.edu
>      <http://client.ourdomain.edu/>Platform: x86_64 Operating system:
>      5.10.9-201.fc33.x86_64 Principal name: host/client.ourdomain.edu
>      <http://client.ourdomain.edu>(a)OURDOMAIN.EDU <http://OURDOMAIN.EDU>
>      Principal alias: host/client.ourdomain.edu
>      <http://client.ourdomain.edu>(a)OURDOMAIN.EDU <http://OURDOMAIN.EDU>
>      Managed by: client.ourdomain.edu <http://client.ourdomain.edu/>;,
>      ourdomain.edu <http://ourdomain.edu/>-------------------------
>      Number of members added 1 ------------------------- [root@ourdomain
>      ~]# ipa-getkeytab -s ourdomain.edu <http://ourdomain.edu/>-p host/
>      client.ourdomain.edu <http://client.ourdomain.edu/>-k
>      /tmp/client.keytab 
>      > Keytab successfully retrieved and stored in: /tmp/client.keytab
>  
>      This is why SSSD isn't working. SSSD uses the host keytab in
>      /etc/krb5.keytab and you invalidated it with the above command.
>  
>  
>  OK what do I need to do to fix this? I got this
>
> from https://docs.fedoraproject.org/en-US/Fedora/18/html/FreeIPA_Guide/In...
>  (which I realize is old), 
Please do not use these documents. We'd remove them if it were in our
control.

You can re-run your getkeytab command using /etc/krb5.keytab instead to
sync up the keys.

>      > Based on this SF discussion
>      >
>     
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__serverfault.com_ques...
>      >,
>      > I changed: in  /etc/krb5.conf
>      > default_ccache_name = FILE:/tmp/krb5cc_%{uid}
>  
>      I don't think this is necessary.
>  
>  
>  OK Thanks for letting me know. 
>  
>  Are these SSH logs helpful:
>  
>  NEEDED_PREAUTH: host/client. ourdomain.edu
>  <http://ourdomain.edu/>.edu(a)OURDOMAIN.EDU <http://OURDOMAIN.EDU> for
>  krbtgt/OURDOMAIN.EDU <http://ourdomain.edu/>@ OURDOMAIN.EDU
>  <http://ourdomain.edu/>;, Additional pre-authentication required Mar 11
>  13:38:28 ourdomain.edu <http://ourdomain.edu/>krb5kdc[369141](info):
>  closing down fd 11 Mar 11 13:38:28 ourdomain.edu
>  <http://ourdomain.edu/>krb5kdc[369144](info): preauth (spake) verify
>  failure: Preauthentication failed
>  
>  Does this have to do with your comment above about SSSD not working? 
Yes. A keytab is a password and this is effectively a "bad password" error.

rob

[Robert Kudyba]

>  >     > Keytab successfully retrieved and stored in: /tmp/client.keytab
>  >
>  >     This is why SSSD isn't working. SSSD uses the host keytab in
>  >     /etc/krb5.keytab and you invalidated it with the above command.
>  >
>  >
>  > OK what do I need to do to fix this? I got this
>  > from
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
>  > (which I realize is old),
>
>  Please do not use these documents. We'd remove them if it were in our
>  control.
>  
Understood.


You can re-run your getkeytab command using /etc/krb5.keytab instead to
>  sync up the keys.
>  
OK I ran:

ipa-getkeytab -s ourdomain.edu -p host/client.ourdomain.edu -k
/etc/krb5.keytab
Keytab successfully retrieved and stored in: /etc/krb5.keytab

Then I scp the keytab to the client /etc directory. Now in the krb5.log I
see:

Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): TGS_REQ (6 etypes
{aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20),
camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17),
aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25)}) 150.108.64.55:
LOOKING_UP_SERVER: authtime 0, etypes {rep=UNSUPPORTED:(0)}
host/client.ourdomain edu(a)OURDOMAIN.EDU for DNS/
ds-dc31.ds.ourdomain.edu(a)OURDOMAIN.EDU, Server not found in Kerberos
database
Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): closing down fd 11

ssh logs:
Mar 11 15:39:27 ourdomain sshd[375517]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Mar 11 15:39:27 ourdomain sshd[375517]: debug1: attempt 2 failures 1
[preauth]
Mar 11 15:39:27 ourdomain sshd[375517]: debug1: keyboard-interactive devs
 [preauth]
Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Mar 11 15:39:27 ourdomain sshd[375517]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Mar 11 15:39:27 ourdomain sshd[375517]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 44136 ssh2 [preauth]
Mar 11 15:39:30 ourdomain sshd[375524]: pam_unix(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth): received for
user ouruser: 9 (Authentication service cannot retrieve authentication info)


>  > Does this have to do with your comment above about SSSD not
> working?
>
>  Yes. A keytab is a password and this is effectively a "bad password" error.

Do I have to restart any services?

[Robert Kudyba]

Still seeing:
preauth (spake) verify failure: Preauthentication failed

kvno ldap/ourdomain
kvno = 2
kvno http/ourdomain
kvno = 1

klist -k
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
----
--------------------------------------------------------------------------
   2 host/ourdomain.edu(a)OURDOMAIN.EDU
   2 host/ourdomain.edu(a)OURDOMAIN.EDU
   2 host/ourdomain.edu(a)OURDOMAIN.EDU
   2 host/ourdomain.edu(a)OURDOMAIN.EDU
   2 host/ourdomain.edu(a)OURDOMAIN.EDU
   2 host/ourdomain.edu(a)OURDOMAIN.EDU
   4 host/ourclient.edu(a)OURCLIENT.EDU
   4 host/ourclient.edu(a)OURCLIENT.EDU
   5 host/ourclient.edu(a)OURCLIENT.EDU
   5 host/ourclient.edu(a)OURCLIENT.EDU
   6 host/ourclient.edu(a)OURCLIENT.EDU
   6 host/ourclient.edu(a)OURCLIENT.EDU
   7 host/ourclient.edu(a)OURCLIENT.EDU
   7 host/ourclient.edu(a)OURCLIENT.EDU


On Thu, Mar 11, 2021 at 3:47 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:

>  >     > Keytab successfully retrieved and stored in:
> /tmp/client.keytab
> > >
> > >     This is why SSSD isn't working. SSSD uses the host keytab in
> > >     /etc/krb5.keytab and you invalidated it with the above command.
> > >
> > >
> > > OK what do I need to do to fix this? I got this
> > > from
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
> > > (which I realize is old),
> >
> > Please do not use these documents. We'd remove them if it were in our
> > control.
> >
>
>  Understood.
>
>
>  You can re-run your getkeytab command using /etc/krb5.keytab instead to
> > sync up the keys.
> >
>
>  OK I ran:
>
>  ipa-getkeytab -s ourdomain.edu -p host/client.ourdomain.edu -k
>  /etc/krb5.keytab
>  Keytab successfully retrieved and stored in: /etc/krb5.keytab
>
>  Then I scp the keytab to the client /etc directory. Now in the krb5.log I
>  see:
>
>  Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): TGS_REQ (6 etypes
>  {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20),
>  camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17),
>  aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25)}) 150.108.64.55:
>  LOOKING_UP_SERVER: authtime 0, etypes {rep=UNSUPPORTED:(0)}
>  host/client.ourdomain edu(a)OURDOMAIN.EDU for DNS/
>  ds-dc31.ds.ourdomain.edu(a)OURDOMAIN.EDU, Server not found in Kerberos
>  database
>  Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): closing down fd 11
>
>  ssh logs:
>  Mar 11 15:39:27 ourdomain sshd[375517]: debug1: userauth-request for user
>  ouruser service ssh-connection method keyboard-interactive [preauth]
>  Mar 11 15:39:27 ourdomain sshd[375517]: debug1: attempt 2 failures 1
>  [preauth]
>  Mar 11 15:39:27 ourdomain sshd[375517]: debug1: keyboard-interactive devs
>   [preauth]
>  Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Mar 11 15:39:27 ourdomain sshd[375517]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Mar 11 15:39:27 ourdomain sshd[375517]: Postponed keyboard-interactive for
>  ouruser from x.x.x.x port 44136 ssh2 [preauth]
>  Mar 11 15:39:30 ourdomain sshd[375524]: pam_unix(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
>   user=ouruser
>  Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth): authentication
>  failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
>  Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth): received for
>  user ouruser: 9 (Authentication service cannot retrieve authentication info)
>
>
> > > Does this have to do with your comment above about SSSD not working?
> >
> > Yes. A keytab is a password and this is effectively a "bad password"
> > error.
>
>
>  Do I have to restart any services?
>

[Robert Kudyba]

I'd like to provide an update. I can get ssh -k to work but here's what I
had to do:
1. I had to run ipa-client-install on another server/computer
2. I ran kinit ouruser(a)OURDOMAIN.EDU
3. I could then run ssh -k ouruser(a)ourdomain.edu and automatically logged
in without needing to enter a password.

My question is, how does this scale to users, i.e., in our case, students,
who are all over the world using their own laptops? Does every user client,
i.e., computer, need to run ipa-client-install? Am I missing something?

On Thu, Mar 11, 2021 at 5:43 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:

>  Still seeing:
>  preauth (spake) verify failure: Preauthentication failed
>
>  kvno ldap/ourdomain
>  kvno = 2
>  kvno http/ourdomain
>  kvno = 1
>
>  klist -k
>  Keytab name: FILE:/etc/krb5.keytab
>  KVNO Principal
>  ----
>  --------------------------------------------------------------------------
>     2 host/ourdomain.edu(a)OURDOMAIN.EDU
>     2 host/ourdomain.edu(a)OURDOMAIN.EDU
>     2 host/ourdomain.edu(a)OURDOMAIN.EDU
>     2 host/ourdomain.edu(a)OURDOMAIN.EDU
>     2 host/ourdomain.edu(a)OURDOMAIN.EDU
>     2 host/ourdomain.edu(a)OURDOMAIN.EDU
>     4 host/ourclient.edu(a)OURCLIENT.EDU
>     4 host/ourclient.edu(a)OURCLIENT.EDU
>     5 host/ourclient.edu(a)OURCLIENT.EDU
>     5 host/ourclient.edu(a)OURCLIENT.EDU
>     6 host/ourclient.edu(a)OURCLIENT.EDU
>     6 host/ourclient.edu(a)OURCLIENT.EDU
>     7 host/ourclient.edu(a)OURCLIENT.EDU
>     7 host/ourclient.edu(a)OURCLIENT.EDU
>
>
>  On Thu, Mar 11, 2021 at 3:47 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote:
>
> > >     > Keytab successfully retrieved and stored in: /tmp/client.keytab
> >> >
> >> >     This is why SSSD isn't working. SSSD uses the host keytab in
> >> >     /etc/krb5.keytab and you invalidated it with the above command.
> >> >
> >> >
> >> > OK what do I need to do to fix this? I got this
> >> > from
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o...
> >> > (which I realize is old),
> >>
> >> Please do not use these documents. We'd remove them if it were in our
> >> control.
> >>
> >
> > Understood.
> >
> >
> > You can re-run your getkeytab command using /etc/krb5.keytab instead to
> >> sync up the keys.
> >>
> >
> > OK I ran:
> >
> > ipa-getkeytab -s ourdomain.edu -p host/client.ourdomain.edu -k
> > /etc/krb5.keytab
> > Keytab successfully retrieved and stored in: /etc/krb5.keytab
> >
> > Then I scp the keytab to the client /etc directory. Now in the krb5.log I
> > see:
> >
> > Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): TGS_REQ (6 etypes
> > {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20),
> > camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17),
> > aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25)}) 150.108.64.55:
> > LOOKING_UP_SERVER: authtime 0, etypes {rep=UNSUPPORTED:(0)}
> > host/client.ourdomain edu(a)OURDOMAIN.EDU for DNS/
> > ds-dc31.ds.ourdomain.edu(a)OURDOMAIN.EDU, Server not found in Kerberos
> > database
> > Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): closing down fd 11
> >
> > ssh logs:
> > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: userauth-request for user
> > ouruser service ssh-connection method keyboard-interactive [preauth]
> > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: attempt 2 failures 1
> > [preauth]
> > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: keyboard-interactive devs
> >  [preauth]
> > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge:
> > user=ouruser devs= [preauth]
> > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: kbdint_alloc: devices
> > 'pam' [preauth]
> > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge_start:
> > trying authentication method 'pam' [preauth]
> > Mar 11 15:39:27 ourdomain sshd[375517]: Postponed keyboard-interactive
> > for ouruser from x.x.x.x port 44136 ssh2 [preauth]
> > Mar 11 15:39:30 ourdomain sshd[375524]: pam_unix(sshd:auth):
> > authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> >  user=ouruser
> > Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth):
> > authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x
> > user=ouruser
> > Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth): received for
> > user ouruser: 9 (Authentication service cannot retrieve authentication info)
> >
> >
> >> > Does this have to do with your comment above about SSSD not working?
> >>
> >> Yes. A keytab is a password and this is effectively a "bad password"
> >> error.
> >
> >
> > Do I have to restart any services?
> >
>

[Rob Crittenden]

Robert Kudyba wrote:
>  I'd like to provide an update. I can get ssh -k to work but
> here's what
>  I had to do:
>  1. I had to run ipa-client-install on another server/computer
>  2. I ran kinit ouruser(a)OURDOMAIN.EDU <mailto:[e-mail]>
>  3. I could then run ssh -k ouruser(a)ourdomain.edu
>  <mailto:[e-mail]> and automatically logged in without
>  needing to enter a password.
>  
>  My question is, how does this scale to users, i.e., in our case,
>  students, who are all over the world using their own laptops? Does every
>  user client, i.e., computer, need to run ipa-client-install? Am I
>  missing something? 
It depends on what the expectations are for these user-owned machines.

If you don't need IPA identities and IPA users won't log into them, then
they only need a working krb5.conf and DNS configured on them.

So your students would log into their own controlled machine using their
own local account, kinit student123(a)univ.edu and ssh using their
credentials.

The krb5.conf will tell the student machine how to contact the KDC.
That's all that is necessary (beyond working DNS).

rob

>  
>  On Thu, Mar 11, 2021 at 5:43 PM Robert Kudyba <rkudyba(a)fordham.edu
>  <mailto:[e-mail]>> wrote:
>  
>      Still seeing:
>      preauth (spake) verify failure: Preauthentication failed
>  
>      kvno ldap/ourdomain
>      kvno = 2
>      kvno http/ourdomain
>      kvno = 1
>  
>      klist -k
>      Keytab name: FILE:/etc/krb5.keytab
>      KVNO Principal
>      ----
>      --------------------------------------------------------------------------
>         2 host/ourdomain.edu(a)OURDOMAIN.EDU
>      <mailto:[e-mail]>
>         2 host/ourdomain.edu(a)OURDOMAIN.EDU
>      <mailto:[e-mail]>
>         2 host/ourdomain.edu(a)OURDOMAIN.EDU
>      <mailto:[e-mail]>
>         2 host/ourdomain.edu(a)OURDOMAIN.EDU
>      <mailto:[e-mail]>
>         2 host/ourdomain.edu(a)OURDOMAIN.EDU
>      <mailto:[e-mail]>
>         2 host/ourdomain.edu(a)OURDOMAIN.EDU
>      <mailto:[e-mail]>
>         4 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>         4 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>         5 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>         5 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>         6 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>         6 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>         7 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>         7 host/ourclient.edu(a)OURCLIENT.EDU
>      <mailto:[e-mail]>
>  
>  
>      On Thu, Mar 11, 2021 at 3:47 PM Robert Kudyba <rkudyba(a)fordham.edu
>      <mailto:[e-mail]>> wrote:
>  
>              >     > Keytab successfully retrieved and stored in:
>              /tmp/client.keytab
>              >
>              >     This is why SSSD isn't working. SSSD uses the host
>              keytab in
>              >     /etc/krb5.keytab and you invalidated it with the above
>              command.
>              >
>              >
>              > OK what do I need to do to fix this? I got this
>              >
>             
> from https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproj...
>  
>              > (which I realize is old),
>  
>              Please do not use these documents. We'd remove them if it
>              were in our
>              control.
>  
>  
>          Understood. 
>  
>  
>              You can re-run your getkeytab command using /etc/krb5.keytab
>              instead to
>              sync up the keys.
>  
>  
>          OK I ran: 
>  
>          ipa-getkeytab -s ourdomain.edu <http://ourdomain.edu> -p
>          host/client.ourdomain.edu <http://client.ourdomain.edu> -k
>          /etc/krb5.keytab
>          Keytab successfully retrieved and stored in: /etc/krb5.keytab
>  
>          Then I scp the keytab to the client /etc directory. Now in the
>          krb5.log I see:
>  
>          Mar 11 15:39:29 ourdomain.edu <http://ourdomain.edu>
>          krb5kdc[369141](info): TGS_REQ (6 etypes
>          {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20),
>          camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17),
>          aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25)})
>          150.108.64.55 <http://150.108.64.55>;: LOOKING_UP_SERVER:
>          authtime 0, etypes {rep=UNSUPPORTED:(0)}
>          host/client.ourdomain edu(a)OURDOMAIN.EDU
>          <mailto:[e-mail]> for
>          DNS/ds-dc31.ds.ourdomain.edu(a)OURDOMAIN.EDU
>          <mailto:[e-mail]>, Server not
>          found in Kerberos database
>          Mar 11 15:39:29 ourdomain.edu <http://ourdomain.edu>
>          krb5kdc[369141](info): closing down fd 11
>  
>          ssh logs:
>          Mar 11 15:39:27 ourdomain sshd[375517]: debug1: userauth-request
>          for user ouruser service ssh-connection method
>          keyboard-interactive [preauth]
>          Mar 11 15:39:27 ourdomain sshd[375517]: debug1: attempt 2
>          failures 1 [preauth]
>          Mar 11 15:39:27 ourdomain sshd[375517]: debug1:
>          keyboard-interactive devs  [preauth]
>          Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge:
>          user=ouruser devs= [preauth]
>          Mar 11 15:39:27 ourdomain sshd[375517]: debug1: kbdint_alloc:
>          devices 'pam' [preauth]
>          Mar 11 15:39:27 ourdomain sshd[375517]: debug1:
>          auth2_challenge_start: trying authentication method 'pam' [preauth]
>          Mar 11 15:39:27 ourdomain sshd[375517]: Postponed
>          keyboard-interactive for ouruser from x.x.x.x port 44136 ssh2
>          [preauth]
>          Mar 11 15:39:30 ourdomain sshd[375524]: pam_unix(sshd:auth):
>          authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>          rhost=x.x.x.x  user=ouruser
>          Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth):
>          authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>          rhost=x.x.x.x user=ouruser
>          Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth):
>          received for user ouruser: 9 (Authentication service cannot
>          retrieve authentication info)
>           
>  
>              > Does this have to do with your comment above about SSSD
>              not working?
>  
>              Yes. A keytab is a password and this is effectively a "bad
>              password" error.
>  
>           
>          Do I have to restart any services?
>

[Robert Kudyba]

On Mon, Mar 15, 2021 at 4:31 PM Rob Crittenden <rcritten(a)redhat.com&gt; wrote:

>  Robert Kudyba wrote:
>  > I'd like to provide an update. I can get ssh -k to work but here's what
>  > I had to do:
>  > 1. I had to run ipa-client-install on another server/computer
>  > 2. I ran kinit ouruser(a)OURDOMAIN.EDU <mailto:[e-mail]>
>  > 3. I could then run ssh -k ouruser(a)ourdomain.edu
>  > <mailto:[e-mail]> and automatically logged in without
>  > needing to enter a password.
>  >
>  > My question is, how does this scale to users, i.e., in our case,
>  > students, who are all over the world using their own laptops? Does every
>  > user client, i.e., computer, need to run ipa-client-install? Am I
>  > missing something?
>
>  It depends on what the expectations are for these user-owned machines.
>  
Only expectation is to be able to log in to a server, get access to their
home directory and be able to do their assignments, e.g., C++, Java or
Python programming.


>  If you don't need IPA identities and IPA users won't log into
> them, then
>  they only need a working krb5.conf and DNS configured on them.
>  
So each device needs to drop in the krb5.conf file from the FreeIPA server?
How does this work on a Windows client?


>  So your students would log into their own controlled machine using
> their
>  own local account, kinit student123(a)univ.edu and ssh using their
>  credentials.
>
>  The krb5.conf will tell the student machine how to contact the KDC.
>  That's all that is necessary (beyond working DNS). 

I just tried this on another Fedora 33 workstation, dropped in the
/etc/krb5.conf file and all I get is:
kinit: No KCM server found while getting default ccache

I'm puzzled as to what we'd need to tell/provide to a student, who is
enrolled remotely and can't come on campus, to be able to connect to our
server via their Windows or Mac laptop.

[Rob Crittenden]

Robert Kudyba wrote:
>  
>  
>  On Mon, Mar 15, 2021 at 4:31 PM Rob Crittenden <rcritten(a)redhat.com
>  <mailto:[e-mail]>> wrote:
>  
>      Robert Kudyba wrote:
>      > I'd like to provide an update. I can get ssh -k to work but here's
>      what
>      > I had to do:
>      > 1. I had to run ipa-client-install on another server/computer
>      > 2. I ran kinit ouruser(a)OURDOMAIN.EDU
>      <mailto:[e-mail]> <mailto:[e-mail]
>      <mailto:[e-mail]>>
>      > 3. I could then run ssh -k ouruser(a)ourdomain.edu
>      <mailto:[e-mail]>
>      > <mailto:[e-mail] <mailto:[e-mail]>> and
>      automatically logged in without
>      > needing to enter a password.
>      >
>      > My question is, how does this scale to users, i.e., in our case,
>      > students, who are all over the world using their own laptops? Does
>      every
>      > user client, i.e., computer, need to run ipa-client-install? Am I
>      > missing something?
>  
>      It depends on what the expectations are for these user-owned machines.
>  
>  
>  Only expectation is to be able to log in to a server, get access to
>  their home directory and be able to do their assignments, e.g., C++,
>  Java or Python programming.
>   
>  
>      If you don't need IPA identities and IPA users won't log into them, then
>      they only need a working krb5.conf and DNS configured on them.
>  
>  
>  So each device needs to drop in the krb5.conf file from the FreeIPA
>  server? How does this work on a Windows client? 
From the server? I wouldn't. It is likely going to need some hand-tuning
depending on your configuration. For example the server is going to have
a hardcoded KDC in it. You may or may not want that.
  
>  
>      So your students would log into their own controlled machine using their
>      own local account, kinit student123(a)univ.edu
>      <mailto:[e-mail]> and ssh using their
>      credentials.
>  
>      The krb5.conf will tell the student machine how to contact the KDC.
>      That's all that is necessary (beyond working DNS).
>  
>  
>  I just tried this on another Fedora 33 workstation, dropped in the
>  /etc/krb5.conf file and all I get is:
>  kinit: No KCM server found while getting default ccache 
You can comment the values out in /etc/krb5.conf.d/kcm_default_ccache to
change the default ccache type, or comment out the includes in krb5.conf
(probably easier).

>  I'm puzzled as to what we'd need to tell/provide to a
> student, who is
>  enrolled remotely and can't come on campus, to be able to connect to our
>  server via their Windows or Mac laptop.  
I don't know about Windows. I used the Windows MIT Kerberos packages a
decade or more ago and they worked fine with PuTTY (and IPA with
discovery) but whether that applies now or not I have no idea.

Mac I think should work similar to Linux: provide a krb5.conf and things
should just work. Again, you'll likely have to tweak the configuration
depending on what version of MIT Mac ships these days.

We mostly got out of the "here is how you configure all possible
clients" game quite a long time ago because it was just unsupportable.
We couldn't keep up with all the different *nix versions much less Mac
and Windows.

You're talking pure Kerberos from some client to an IPA server. The fact
that it is IPA should be irrelevant other than maybe some server-side
configuration. So the documentation for that given OS should provide
reasonable guidance.

rob

[Robert Kudyba]

On Tue, Mar 16, 2021 at 3:40 PM Rob Crittenden <rcritten(a)redhat.com&gt; wrote:

>  >     It depends on what the expectations are for these
> user-owned
>  machines.
>  >
>  >
>  > Only expectation is to be able to log in to a server, get access to
>  > their home directory and be able to do their assignments, e.g., C++,
>  > Java or Python programming.
>  >
>  >
>  >     If you don't need IPA identities and IPA users won't log into them,
>  then
>  >     they only need a working krb5.conf and DNS configured on them.
>  >
>  >
>  > So each device needs to drop in the krb5.conf file from the FreeIPA
>  > server? How does this work on a Windows client?
>
>  From the server? I wouldn't. It is likely going to need some hand-tuning
>  depending on your configuration. For example the server is going to have
>  a hardcoded KDC in it. You may or may not want that.
>  
So we have to customized the /etc/krb5.conf file that exists on the server
for any student devices.


>  >
>  >     So your students would log into their own controlled machine using
>  their
>  >     own local account, kinit student123(a)univ.edu
>  >     <mailto:[e-mail]> and ssh using their
>  >     credentials.
>  >
>  >     The krb5.conf will tell the student machine how to contact the KDC.
>  >     That's all that is necessary (beyond working DNS).
>  >
>  >
>  > I just tried this on another Fedora 33 workstation, dropped in the
>  > /etc/krb5.conf file and all I get is:
>  > kinit: No KCM server found while getting default ccache
>
>  You can comment the values out in /etc/krb5.conf.d/kcm_default_ccache to
>  change the default ccache type, or comment out the includes in krb5.conf
>  (probably easier).
>  
OK now I can get any Fedora client to kinit and then ssh.


>  > I'm puzzled as to what we'd need to tell/provide to a
> student, who is
>  > enrolled remotely and can't come on campus, to be able to connect to our
>  > server via their Windows or Mac laptop.
>
>  I don't know about Windows. I used the Windows MIT Kerberos packages a
>  decade or more ago and they worked fine with PuTTY (and IPA with
>  discovery) but whether that applies now or not I have no idea.
>
>  Mac I think should work similar to Linux: provide a krb5.conf and things
>  should just work. Again, you'll likely have to tweak the configuration
>  depending on what version of MIT Mac ships these days.
>  
kinit --version

kinit (Heimdal 1.5.1apple1)

So my first test with the server krb5.conf file copied into /etc:

kinit: krb5_get_init_creds: unable to reach any KDC in realm OURDOMAIN.EDU,
tried 0 KDCs

So the first suggestion <https://apple.stackexchange.com/a/273064> I found
was to preface kdc = tcp

Then I made sure the firewall on the Mac was disabled. I also added the
test IPA server & IP into /etc/hosts. I can ping it successfully.

What else needs to change?

[Rob Crittenden]

Robert Kudyba via FreeIPA-users wrote:
>  
>  
>  On Tue, Mar 16, 2021 at 3:40 PM Rob Crittenden <rcritten(a)redhat.com
>  <mailto:[e-mail]>> wrote:
>  
>      >     It depends on what the expectations are for these user-owned
>      machines.
>      >
>      >
>      > Only expectation is to be able to log in to a server, get access to
>      > their home directory and be able to do their assignments, e.g., C++,
>      > Java or Python programming.
>      >  
>      >
>      >     If you don't need IPA identities and IPA users won't log into
>      them, then
>      >     they only need a working krb5.conf and DNS configured on them.
>      >
>      >
>      > So each device needs to drop in the krb5.conf file from the FreeIPA
>      > server? How does this work on a Windows client?
>  
>      From the server? I wouldn't. It is likely going to need some hand-tuning
>      depending on your configuration. For example the server is going to have
>      a hardcoded KDC in it. You may or may not want that.
>  
>  
>  So we have to customized the /etc/krb5.conf file that exists on the
>  server for any student devices.  
I mean, you don't want to use ipa-client-install which would do all of
this for you, and I understand the reasons, but it does mean some
additional work on your part.

I don't know your network so at most I can make general suggestions, not
provide you a full configuration.

In retrospect the default krb5.conf that ships on Fedora provides for
includes. I think this is probably your best bet: provide an IPA
configuration that resides there and it should co-exist pretty easily
with any other configuration.

I'm not completely sure about the order of loading and which
configuration "wins" when there is conflict. The man page is the place
to look.

And kcm_default_ccache has instructions on how to enable/run sssd-kcm so
that this should work out-of-the-box. That is probably better than
having students comment it out, unless you can control the order of what
"wins" when there is conflicting configuration.

>   
>  
>      >
>      >     So your students would log into their own controlled machine
>      using their
>      >     own local account, kinit student123(a)univ.edu
>      <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>> and
>      ssh using their
>      >     credentials.
>      >
>      >     The krb5.conf will tell the student machine how to contact the
>      KDC.
>      >     That's all that is necessary (beyond working DNS).
>      >
>      >
>      > I just tried this on another Fedora 33 workstation, dropped in the
>      > /etc/krb5.conf file and all I get is:
>      > kinit: No KCM server found while getting default ccache
>  
>      You can comment the values out in /etc/krb5.conf.d/kcm_default_ccache to
>      change the default ccache type, or comment out the includes in krb5.conf
>      (probably easier).
>  
>  
>  OK now I can get any Fedora client to kinit and then ssh. 
See about for perhaps a less hacky approach than I originally suggested.
  
>  
>      > I'm puzzled as to what we'd need to tell/provide to a student, who is
>      > enrolled remotely and can't come on campus, to be able to connect
>      to our
>      > server via their Windows or Mac laptop. 
>  
>      I don't know about Windows. I used the Windows MIT Kerberos packages a
>      decade or more ago and they worked fine with PuTTY (and IPA with
>      discovery) but whether that applies now or not I have no idea.
>  
>      Mac I think should work similar to Linux: provide a krb5.conf and things
>      should just work. Again, you'll likely have to tweak the configuration
>      depending on what version of MIT Mac ships these days.
>  
>  
>  kinit --version
>  
>  kinit (Heimdal 1.5.1apple1)
>  
>   
>  So my first test with the server krb5.conf file copied into /etc:
>  
>  kinit: krb5_get_init_creds: unable to reach any KDC in realm
>  OURDOMAIN.EDU <http://OURDOMAIN.EDU>;, tried 0 KDCs
>  
>  
>  So the first suggestion <https://apple.stackexchange.com/a/273064> I
>  found was to preface kdc = tcp
>  
>  Then I made sure the firewall on the Mac was disabled. I also added the
>  test IPA server & IP into /etc/hosts. I can ping it successfully.
>  
>  What else needs to change? 
It's difficult to troubleshoot in a void. I don't know your network
configuration nor what krb5.conf you're using. It sure looks like
discovery of the KDC over DNS failed.

rob

[Robert Kudyba]

On Wed, Mar 17, 2021 at 9:27 AM Rob Crittenden <rcritten(a)redhat.com&gt; wrote:

>  Robert Kudyba via FreeIPA-users wrote:
>  >
>  >
>  > On Tue, Mar 16, 2021 at 3:40 PM Rob Crittenden <rcritten(a)redhat.com
>  > <mailto:[e-mail]>> wrote:
>  >
>  >     >     It depends on what the expectations are for these user-owned
>  >     machines.
>  >     >
>  >     >
>  >     > Only expectation is to be able to log in to a server, get access to
>  >     > their home directory and be able to do their assignments, e.g.,
>  C++,
>  >     > Java or Python programming.
>  >     >
>  >     >
>  >     >     If you don't need IPA identities and IPA users won't log into
>  >     them, then
>  >     >     they only need a working krb5.conf and DNS configured on them.
>  >     >
>  >     >
>  >     > So each device needs to drop in the krb5.conf file from the FreeIPA
>  >     > server? How does this work on a Windows client?
>  >
>  >     From the server? I wouldn't. It is likely going to need some
>  hand-tuning
>  >     depending on your configuration. For example the server is going to
>  have
>  >     a hardcoded KDC in it. You may or may not want that.
>  >
>  >
>  > So we have to customized the /etc/krb5.conf file that exists on the
>  > server for any student devices.
>
>  I mean, you don't want to use ipa-client-install which would do all of
>  this for you, and I understand the reasons, but it does mean some
>  additional work on your part.
>
>  I don't know your network so at most I can make general suggestions, not
>  provide you a full configuration.
>  
Since it's a test server DNS is not fully configured on the server to
resolve properly, so I now set the krb5.conf file to ignore DNS (see below)



>  In retrospect the default krb5.conf that ships on Fedora provides
> for
>  includes. I think this is probably your best bet: provide an IPA
>  configuration that resides there and it should co-exist pretty easily
>  with any other configuration.
>
>  I'm not completely sure about the order of loading and which
>  configuration "wins" when there is conflict. The man page is the place
>  to look.
>
>  And kcm_default_ccache has instructions on how to enable/run sssd-kcm so
>  that this should work out-of-the-box. That is probably better than
>  having students comment it out, unless you can control the order of what
>  "wins" when there is conflicting configuration.
>  
Thanks I'll also look into this.

>      >     So your students would log into their own controlled
> machine
>  >     using their
>  >     >     own local account, kinit student123(a)univ.edu
>  >     <mailto:[e-mail]>
>  >     >     <mailto:[e-mail] <mailto:[e-mail]>>
> and
>  >     ssh using their
>  >     >     credentials.
>  >     >
>  >     >     The krb5.conf will tell the student machine how to contact the
>  >     KDC.
>  >     >     That's all that is necessary (beyond working DNS).
>  >     >
>  >     >
>  >     > I just tried this on another Fedora 33 workstation, dropped in the
>  >     > /etc/krb5.conf file and all I get is:
>  >     > kinit: No KCM server found while getting default ccache
>  >
>  >     You can comment the values out in
>  /etc/krb5.conf.d/kcm_default_ccache to
>  >     change the default ccache type, or comment out the includes in
>  krb5.conf
>  >     (probably easier).
>  >
>  >
>  > OK now I can get any Fedora client to kinit and then ssh.
>
>  See about for perhaps a less hacky approach than I originally suggested.
>  
What "about" are you referring to?


>  >     > I'm puzzled as to what we'd need to
> tell/provide to a student, who
>  is
>  >     > enrolled remotely and can't come on campus, to be able to connect
>  >     to our
>  >     > server via their Windows or Mac laptop.
>  >
>  >     I don't know about Windows. I used the Windows MIT Kerberos packages
>  a
>  >     decade or more ago and they worked fine with PuTTY (and IPA with
>  >     discovery) but whether that applies now or not I have no idea.
>  >
>  >     Mac I think should work similar to Linux: provide a krb5.conf and
>  things
>  >     should just work. Again, you'll likely have to tweak the
>  configuration
>  >     depending on what version of MIT Mac ships these days.
>  >
>  >
>  > kinit --version
>  >
>  > kinit (Heimdal 1.5.1apple1)
>  >
>  >
>  > So my first test with the server krb5.conf file copied into /etc:
>  >
>  > kinit: krb5_get_init_creds: unable to reach any KDC in realm
>  > OURDOMAIN.EDU <
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__OURDOMAIN.EDU&d=D...
>  >, tried 0 KDCs
>  >
>  >
>  > So the first suggestion <
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__apple.stackexchange....
>  > I
>  > found was to preface kdc = tcp
>  >
>  > Then I made sure the firewall on the Mac was disabled. I also added the
>  > test IPA server & IP into /etc/hosts. I can ping it successfully.
>  >
>  > What else needs to change?
>
>  It's difficult to troubleshoot in a void. I don't know your network
>  configuration nor what krb5.conf you're using. It sure looks like
>  discovery of the KDC over DNS failed.
>  
I configured the following in krb5.conf and now at least get prompted for a
password and kinit works!:
  [libdefaults]
dns_lookup_kdc   = no
dns_lookup_realm = no

klist
Ticket cache: API:krb5cc
Default principal: ouruser(a)OURDOMAIN.EDU

Valid starting     Expires            Service principal
03/18/21 15:17:43  03/19/21 15:17:39  krbtgt/OURDOMAIN.EDU(a)OURDOMAIN.EDU

However ssh -k on both a Mac and Windows PC do NOT automatically log me in
and only the NIS password works. From ssh -vv all I see is:
debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1

And from the ssh logs:
Mar 18 15:52:48 ourserver sshd[634486]: debug1: restore_uid: 0/0
Mar 18 15:52:48 ourserver sshd[634486]: debug1: temporarily_use_uid: 99/99
(e=0/0)
Mar 18 15:52:48 ourserver sshd[634486]: debug1: restore_uid: 0/0
Mar 18 15:52:48 ourserver sshd[634486]: debug1: temporarily_use_uid: 99/99
(e=0/0)
Mar 18 15:52:48 ourserver sshd[634486]: debug1: restore_uid: 0/0
Mar 18 15:52:48 ourserver sshd[634486]: Failed publickey for ouruser from
x.x.x.x port 51827 ssh2: ED25519
SHA256:BH1fuycgWofiOBV9lPK4XB2vYK3frN2FKv208PnmENI
Mar 18 15:52:48 ourserver sshd[634486]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]
Mar 18 15:52:48 ourserver sshd[634486]: debug1: attempt 3 failures 2
[preauth]
Mar 18 15:52:48 ourserver sshd[634486]: debug1: keyboard-interactive devs
 [preauth]
Mar 18 15:52:48 ourserver sshd[634486]: debug1: auth2_challenge:
user=ouruser devs= [preauth]
Mar 18 15:52:48 ourserver sshd[634486]: debug1: kbdint_alloc: devices 'pam'
[preauth]
Mar 18 15:52:48 ourserver sshd[634486]: debug1: auth2_challenge_start:
trying authentication method 'pam' [preauth]
Mar 18 15:52:48 ourserver sshd[634486]: Postponed keyboard-interactive for
ouruser from x.x.x.x port 51827 ssh2 [preauth]
Mar 18 15:52:58 ourserver sshd[634508]: pam_unix(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser
Mar 18 15:52:58 ourserver sshd[634508]: pam_sss(sshd:auth): authentication
failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser
Mar 18 15:52:58 ourserver sshd[634508]: pam_sss(sshd:auth): received for
user ouruser: 9 (Authentication service cannot retrieve authentication info)
Mar 18 15:53:00 ourserver sshd[634486]: error: PAM: Authentication failure
for ouruser from x.x.x.x
Mar 18 15:53:00 ourserver sshd[634486]: Failed keyboard-interactive/pam for
ouruser from x.x.x.x port 51827 ssh2
Mar 18 15:53:00 ourserver sshd[634486]: debug1: userauth-request for user
ouruser service ssh-connection method keyboard-interactive [preauth]

So is there some other configuration that needs to be set to pass
on/through from kinit/ticket to ssh, on Windows and Mac? Perhaps something
in krb5.conf?

[Rob Crittenden]

Robert Kudyba via FreeIPA-users wrote:
>  On Wed, Mar 17, 2021 at 9:27 AM Rob Crittenden
> <rcritten(a)redhat.com
>  <mailto:[e-mail]>> wrote:
>  
>      Robert Kudyba via FreeIPA-users wrote:
>      >
>      >
>      > On Tue, Mar 16, 2021 at 3:40 PM Rob Crittenden
>      <rcritten(a)redhat.com <mailto:[e-mail]>
>      > <mailto:[e-mail] <mailto:[e-mail]>>>
> wrote:
>      >
>      >     >     It depends on what the expectations are for these user-owned
>      >     machines.
>      >     >
>      >     >
>      >     > Only expectation is to be able to log in to a server, get
>      access to
>      >     > their home directory and be able to do their assignments,
>      e.g., C++,
>      >     > Java or Python programming.
>      >     >  
>      >     >
>      >     >     If you don't need IPA identities and IPA users won't log
>      into
>      >     them, then
>      >     >     they only need a working krb5.conf and DNS configured on
>      them.
>      >     >
>      >     >
>      >     > So each device needs to drop in the krb5.conf file from the
>      FreeIPA
>      >     > server? How does this work on a Windows client?
>      >
>      >     From the server? I wouldn't. It is likely going to need some
>      hand-tuning
>      >     depending on your configuration. For example the server is
>      going to have
>      >     a hardcoded KDC in it. You may or may not want that.
>      >
>      >
>      > So we have to customized the /etc/krb5.conf file that exists on the
>      > server for any student devices.
>  
>      I mean, you don't want to use ipa-client-install which would do all of
>      this for you, and I understand the reasons, but it does mean some
>      additional work on your part.
>  
>      I don't know your network so at most I can make general suggestions, not
>      provide you a full configuration.
>  
>  
>  Since it's a test server DNS is not fully configured on the server to
>  resolve properly, so I now set the krb5.conf file to ignore DNS (see below)
>  
>   
>  
>      In retrospect the default krb5.conf that ships on Fedora provides for
>      includes. I think this is probably your best bet: provide an IPA
>      configuration that resides there and it should co-exist pretty easily
>      with any other configuration.
>  
>      I'm not completely sure about the order of loading and which
>      configuration "wins" when there is conflict. The man page is the place
>      to look.
>  
>      And kcm_default_ccache has instructions on how to enable/run sssd-kcm so
>      that this should work out-of-the-box. That is probably better than
>      having students comment it out, unless you can control the order of what
>      "wins" when there is conflicting configuration.
>  
>  
>  Thanks I'll also look into this. 
>  
>      >     >     So your students would log into their own controlled machine
>      >     using their
>      >     >     own local account, kinit student123(a)univ.edu
>      <mailto:[e-mail]>
>      >     <mailto:[e-mail] <mailto:[e-mail]>>
>      >     >     <mailto:[e-mail] <mailto:[e-mail]>
>      <mailto:[e-mail] <mailto:[e-mail]>>> and
>      >     ssh using their
>      >     >     credentials.
>      >     >
>      >     >     The krb5.conf will tell the student machine how to
>      contact the
>      >     KDC.
>      >     >     That's all that is necessary (beyond working DNS).
>      >     >
>      >     >
>      >     > I just tried this on another Fedora 33 workstation, dropped
>      in the
>      >     > /etc/krb5.conf file and all I get is:
>      >     > kinit: No KCM server found while getting default ccache
>      >
>      >     You can comment the values out in
>      /etc/krb5.conf.d/kcm_default_ccache to
>      >     change the default ccache type, or comment out the includes in
>      krb5.conf
>      >     (probably easier).
>      >
>      >
>      > OK now I can get any Fedora client to kinit and then ssh.
>  
>      See about for perhaps a less hacky approach than I originally suggested.
>  
>  
>  What "about" are you referring to? 
Typo. Above.
  
>  
>      >     > I'm puzzled as to what we'd need to tell/provide to a
>      student, who is
>      >     > enrolled remotely and can't come on campus, to be able to
>      connect
>      >     to our
>      >     > server via their Windows or Mac laptop. 
>      >
>      >     I don't know about Windows. I used the Windows MIT Kerberos
>      packages a
>      >     decade or more ago and they worked fine with PuTTY (and IPA with
>      >     discovery) but whether that applies now or not I have no idea.
>      >
>      >     Mac I think should work similar to Linux: provide a krb5.conf
>      and things
>      >     should just work. Again, you'll likely have to tweak the
>      configuration
>      >     depending on what version of MIT Mac ships these days.
>      >
>      >
>      > kinit --version
>      >
>      > kinit (Heimdal 1.5.1apple1)
>      >
>      >  
>      > So my first test with the server krb5.conf file copied into /etc:
>      >
>      > kinit: krb5_get_init_creds: unable to reach any KDC in realm
>      > OURDOMAIN.EDU <http://OURDOMAIN.EDU>
>     
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__OURDOMAIN.EDU&d=D...
>      >, tried 0 KDCs
>      >
>      >
>      > So the first suggestion
>     
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__apple.stackexchange....
>      > I
>      > found was to preface kdc = tcp
>      >
>      > Then I made sure the firewall on the Mac was disabled. I also
>      added the
>      > test IPA server & IP into /etc/hosts. I can ping it successfully.
>      >
>      > What else needs to change?
>  
>      It's difficult to troubleshoot in a void. I don't know your network
>      configuration nor what krb5.conf you're using. It sure looks like
>      discovery of the KDC over DNS failed.
>  
>  
>  I configured the following in krb5.conf and now at least get prompted
>  for a password and kinit works!:
>    [libdefaults]
>  dns_lookup_kdc   = no
>  dns_lookup_realm = no 
>  
>  klist
>  Ticket cache: API:krb5cc
>  Default principal: ouruser(a)OURDOMAIN.EDU <mailto:[e-mail]>
>  
>  Valid starting     Expires            Service principal
>  03/18/21 15:17:43  03/19/21 15:17:39  krbtgt/OURDOMAIN.EDU(a)OURDOMAIN.EDU
>  <mailto:[e-mail]> 
I don't know why mac/Windows isn't working. It doesn't look like it is
even trying GSSAPI.

rob

>  
>  However ssh -k on both a Mac and Windows PC do NOT automatically log me
>  in and only the NIS password works. From ssh -vv all I see is:
>  debug1: Authentications that can continue:
>  publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive
>  debug2: userauth_kbdint
>  debug2: we sent a keyboard-interactive packet, wait for reply
>  debug2: input_userauth_info_req
>  debug2: input_userauth_info_req: num_prompts 1  
>  
>  And from the ssh logs:
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: restore_uid: 0/0
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: temporarily_use_uid:
>  99/99 (e=0/0)
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: restore_uid: 0/0
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: temporarily_use_uid:
>  99/99 (e=0/0)
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: restore_uid: 0/0
>  Mar 18 15:52:48 ourserver sshd[634486]: Failed publickey for ouruser
>  from x.x.x.x port 51827 ssh2: ED25519
>  SHA256:BH1fuycgWofiOBV9lPK4XB2vYK3frN2FKv208PnmENI
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: userauth-request for
>  user ouruser service ssh-connection method keyboard-interactive [preauth]
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: attempt 3 failures 2
>  [preauth]
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: keyboard-interactive
>  devs  [preauth]
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: auth2_challenge:
>  user=ouruser devs= [preauth]
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: kbdint_alloc: devices
>  'pam' [preauth]
>  Mar 18 15:52:48 ourserver sshd[634486]: debug1: auth2_challenge_start:
>  trying authentication method 'pam' [preauth]
>  Mar 18 15:52:48 ourserver sshd[634486]: Postponed keyboard-interactive
>  for ouruser from x.x.x.x port 51827 ssh2 [preauth]
>  Mar 18 15:52:58 ourserver sshd[634508]: pam_unix(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>  rhost=x.x.x.x  user=ouruser
>  Mar 18 15:52:58 ourserver sshd[634508]: pam_sss(sshd:auth):
>  authentication failure; logname= uid=0 euid=0 tty=ssh ruser=
>  rhost=x.x.x.x user=ouruser
>  Mar 18 15:52:58 ourserver sshd[634508]: pam_sss(sshd:auth): received for
>  user ouruser: 9 (Authentication service cannot retrieve authentication info)
>  Mar 18 15:53:00 ourserver sshd[634486]: error: PAM: Authentication
>  failure for ouruser from x.x.x.x
>  Mar 18 15:53:00 ourserver sshd[634486]: Failed keyboard-interactive/pam
>  for ouruser from x.x.x.x port 51827 ssh2
>  Mar 18 15:53:00 ourserver sshd[634486]: debug1: userauth-request for
>  user ouruser service ssh-connection method keyboard-interactive [preauth]
>  
>  So is there some other configuration that needs to be set to pass
>  on/through from kinit/ticket to ssh, on Windows and Mac? Perhaps
>  something in krb5.conf?
>  
>  
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>

[Robert Kudyba]

>  > I configured the following in krb5.conf and now at least get
> prompted
>  > for a password and kinit works!:
>  >   [libdefaults]
>  > dns_lookup_kdc   = no
>  > dns_lookup_realm = no
>  >
>  > klist
>  > Ticket cache: API:krb5cc
>  > Default principal: ouruser(a)OURDOMAIN.EDU <mailto:[e-mail]>
>  >
>  > Valid starting     Expires            Service principal
>  > 03/18/21 15:17:43  03/19/21 15:17:39  krbtgt/OURDOMAIN.EDU(a)OURDOMAIN.EDU
>  > <mailto:[e-mail]>
>
>  I don't know why mac/Windows isn't working. It doesn't look like it is
>  even trying GSSAPI.
>  
Some progress. On Windows if I download and install MIT Kerberos and log in
with a user, I can use Putty for password-less login. Note Putty needs to
be a recent version, e.g.,  0.74.

I also got a Mac client to work. After adding the krb5.conf file into
/etc/krb5.conf, and running kinit I can run ssh -K and voila.

Using another Windows SSH client, e.g., MobaXterm (version 20 or later with
GSSAPI Kerberos checked in SSH settings), I also had to fill in the Domain:
field and use the "Native Windows" option:
[image: image.png]

However when testing, the various kinit attempts from a client generates
the below tickets that appear to be expired.
klist
Ticket cache: KCM:0:59081
Default principal: host/ouserver.edu(a)OURSERVER.EDU

Valid starting     Expires            Service principal
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
12/31/69 19:00:00  12/31/69 19:00:00  Encrypted/Credentials/v1@X-GSSPROXY:

We are on freeipa-server-4.9.2-4.fc33.x86_64 with kernel-5.11.11-200
ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful

The logs that I see related to this user in krb5.conf, with debug enabled
are:
Apr 07 14:03:39 ourdomain.edu krb5kdc[1350528](info): AS_REQ (8 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17),
aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19),
UNSUPPORTED:des3-hmac-sha1(16), DEPRECATED:arcfour-hmac(23),
camellia128-cts-cmac(25), camellia256-cts-cmac(26)}) 150.108.68.55: ISSUE:
authtime 1617818619, etypes {rep=aes256-cts-hmac-sha1-96(18),
tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)},
ouruser(a)ourdomain.EDU for rbtgt/ourdomain.EDU(a)ourdomain.EDU

Apr 07 14:03:39 ourdomain.edu krb5kdc[1350528](info): closing down fd 11

Apr 07 14:03:44 ourdomain.edu krb5kdc[1350531](info): TGS_REQ (8 etypes
{aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17),
aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19),
UNSUPPORTED:des3-hmac-sha1(16), DEPRECATED:arcfour-hmac(23),
camellia128-cts-cmac(25), camellia256-cts-cmac(26)}) 150.108.68.55: ISSUE:
authtime 1617818619, etypes {rep=aes256-cts-hmac-sha1-96(18),
tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)},
ouruser(a)ourdomain.EDU for host/ourdomain.edu(a)ourdomain.EDU

Apr 07 14:03:44 ourdomain.edu krb5kdc[1350531](info): closing down fd 11

Apr 07 14:03:44 ourdomain.edu krb5kdc[1350531](info): TGS_REQ (1 etypes
{aes256-cts-hmac-sha1-96(18)}) 150.108.68.55: ISSUE: authtime 1617818619,
etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18),
ses=aes256-cts-hmac-sha1-96(18)}, ouruser(a)ourdomain.EDU for
krbtgt/ourdomain.EDU(a)ourdomain.EDU

So what causes those "expired" tickets? These "clients" I'm
testing are a
Windows desktop and a Mac, so their hostnames aren't added a la a
freeipa-client.