[Freeipa-users] Samba update can't read NT Hash

[Randy Morgan]

Yesterday we updated our fileserver to bring it up to the newest 
kernel.  At the same time it update the ipa-client and samba. After the 
update was finished our ability to access the shared resources on the 
fileserver disappeared.  After some very careful troubleshooting we have 
been able to narrow it down to a problem with Samba, but we have been 
unable to find where in the configuration the problem is.  I am 
including several logs, config files, etc with this, we need this 
restored ASAP, but can't seem to isolate the issue.

logs:

Log.192.168.105.237

[2017/08/17 07:59:38.684827,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[homes]"

[2017/08/17 07:59:38.684939,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[stockroom]"

[2017/08/17 07:59:38.685049,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[inorgstk]"

[2017/08/17 07:59:38.685144,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[netlogon]"

[2017/08/17 07:59:38.685211,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[deptchair]"

[2017/08/17 07:59:38.685333,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[deptfinance]"

[2017/08/17 07:59:38.685448,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[facultysearch]"

[2017/08/17 07:59:38.685523,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[research]"

[2017/08/17 07:59:38.685610,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[working]"

[2017/08/17 07:59:38.685713,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[csradmin]"

[2017/08/17 07:59:38.685802,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[install]"

[2017/08/17 07:59:38.685933,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[coffice]"

[2017/08/17 07:59:38.686097,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[grants]"

[2017/08/17 07:59:38.686202,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[deptoffice]"

[2017/08/17 07:59:38.686330,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[gradadmissions]"

[2017/08/17 07:59:38.686411,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[mainoffice]"

[2017/08/17 07:59:38.686525,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[busoffice]"

[2017/08/17 07:59:38.686607,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[entropy]"

[2017/08/17 07:59:38.686718,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[ltarch]"

[2017/08/17 07:59:38.686807,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[netlogon-n175]"

[2017/08/17 07:59:38.686963,3] ../source3/param/loadparm.c:1592(lp_add_ipc)

adding IPC service

[2017/08/17 07:59:38.687257,2] ../source3/lib/interface.c:345(add_interface)

added interface eth0 ip=192.168.105.99 bcast=192.168.105.99 
netmask=255.255.255.255

[2017/08/17 07:59:38.687362,3] ../source3/smbd/oplock.c:1322(init_oplocks)

init_oplocks: initializing messages.

[2017/08/17 07:59:38.687511,3] ../source3/smbd/process.c:1957(process_smb)

Transaction 0 of length 159 (0 toread)

[2017/08/17 07:59:38.687557,3] 
../source3/smbd/process.c:1538(switch_message)

switch message SMBnegprot (pid 22349) conn 0x0

[2017/08/17 07:59:38.688383,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [PC NETWORK PROGRAM 1.0]

[2017/08/17 07:59:38.688408,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [LANMAN1.0]

[2017/08/17 07:59:38.688418,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [Windows for Workgroups 3.1a]

[2017/08/17 07:59:38.688423,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [LM1.2X002]

[2017/08/17 07:59:38.688429,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [LANMAN2.1]

[2017/08/17 07:59:38.688434,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [NT LM 0.12]

[2017/08/17 07:59:38.688439,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [SMB 2.002]

[2017/08/17 07:59:38.688444,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [SMB 2.???]

[2017/08/17 07:59:38.688548,3] 
../source3/smbd/smb2_negprot.c:290(smbd_smb2_request_process_negprot)

Selected protocol SMB2_FF

[2017/08/17 07:59:38.689133,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'gssapi_spnego' registered

[2017/08/17 07:59:38.689159,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'gssapi_krb5' registered

[2017/08/17 07:59:38.689171,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'gssapi_krb5_sasl' registered

[2017/08/17 07:59:38.689181,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'spnego' registered

[2017/08/17 07:59:38.689191,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'schannel' registered

[2017/08/17 07:59:38.689203,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'naclrpc_as_system' registered

[2017/08/17 07:59:38.689221,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'sasl-EXTERNAL' registered

[2017/08/17 07:59:38.689249,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'ntlmssp' registered

[2017/08/17 07:59:38.689265,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'ntlmssp_resume_ccache' registered

[2017/08/17 07:59:38.689283,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'http_basic' registered

[2017/08/17 07:59:38.689334,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'http_ntlm' registered

[2017/08/17 07:59:38.690888,3] ../source3/smbd/negprot.c:730(reply_negprot)

Selected protocol SMB 2.???

[2017/08/17 07:59:38.691535,3] 
../source3/smbd/smb2_negprot.c:290(smbd_smb2_request_process_negprot)

Selected protocol SMB3_11

[2017/08/17 07:59:46.501902,3] 
../source3/smbd/smb2_server.c:3097(smbd_smb2_request_error_ex)

smbd_smb2_request_error_ex: smbd_smb2_request_error_ex: idx[1] 
status[NT_STATUS_LOGON_FAILURE] || at ../source3/smbd/smb2_sesssetup.c:134

[2017/08/17 07:59:46.503583,3] 
../source3/smbd/server_exit.c:246(exit_server_common)

Server exit (NT_STATUS_CONNECTION_RESET)

[2017/08/17 07:59:59.462220,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[homes]"

[2017/08/17 07:59:59.462329,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[stockroom]"

[2017/08/17 07:59:59.462456,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[inorgstk]"

[2017/08/17 07:59:59.462530,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[netlogon]"

[2017/08/17 07:59:59.462577,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[deptchair]"

[2017/08/17 07:59:59.462630,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[deptfinance]"

[2017/08/17 07:59:59.462711,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[facultysearch]"

[2017/08/17 07:59:59.462761,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[research]"

[2017/08/17 07:59:59.462839,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[working]"

[2017/08/17 07:59:59.462896,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[csradmin]"

[2017/08/17 07:59:59.462962,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[install]"

[2017/08/17 07:59:59.463032,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[coffice]"

[2017/08/17 07:59:59.463098,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[grants]"

[2017/08/17 07:59:59.463161, 2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[deptoffice]"

[2017/08/17 07:59:59.463238,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[gradadmissions]"

[2017/08/17 07:59:59.463289,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[mainoffice]"

[2017/08/17 07:59:59.463355,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[busoffice]"

[2017/08/17 07:59:59.463418,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[entropy]"

[2017/08/17 07:59:59.463478,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[ltarch]"

[2017/08/17 07:59:59.463540,2] 
../source3/param/loadparm.c:2769(lp_do_section)

Processing section "[netlogon-n175]"

[2017/08/17 07:59:59.463623,3] ../source3/param/loadparm.c:1592(lp_add_ipc)

adding IPC service

[2017/08/17 07:59:59.463828,2] ../source3/lib/interface.c:345(add_interface)

added interface eth0 ip=192.168.105.99 bcast=192.168.105.99 
netmask=255.255.255.255

[2017/08/17 07:59:59.463902,3] ../source3/smbd/oplock.c:1322(init_oplocks)

init_oplocks: initializing messages.

[2017/08/17 07:59:59.464003,3] ../source3/smbd/process.c:1957(process_smb)

Transaction 0 of length 159 (0 toread)

[2017/08/17 07:59:59.464038,3] 
../source3/smbd/process.c:1538(switch_message)

switch message SMBnegprot (pid 22371) conn 0x0

[2017/08/17 07:59:59.464721,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [PC NETWORK PROGRAM 1.0]

[2017/08/17 07:59:59.464747,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [LANMAN1.0]

[2017/08/17 07:59:59.464760,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [Windows for Workgroups 3.1a]

[2017/08/17 07:59:59.464786,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [LM1.2X002]

[2017/08/17 07:59:59.464795,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [LANMAN2.1]

[2017/08/17 07:59:59.464817,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [NT LM 0.12]

[2017/08/17 07:59:59.464876,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [SMB 2.002]

[2017/08/17 07:59:59.464893,3] ../source3/smbd/negprot.c:603(reply_negprot)

Requested protocol [SMB 2.???]

[2017/08/17 07:59:59.465013,3] 
../source3/smbd/smb2_negprot.c:290(smbd_smb2_request_process_negprot)

Selected protocol SMB2_FF

[2017/08/17 07:59:59.465821,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'gssapi_spnego' registered

[2017/08/17 07:59:59.465869,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'gssapi_krb5' registered

[2017/08/17 07:59:59.465879,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'gssapi_krb5_sasl' registered

[2017/08/17 07:59:59.465888,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'spnego' registered

[2017/08/17 07:59:59.465910,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'schannel' registered

[2017/08/17 07:59:59.465930,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'naclrpc_as_system' registered

[2017/08/17 07:59:59.465941,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'sasl-EXTERNAL' registered

[2017/08/17 07:59:59.465949,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'ntlmssp' registered

[2017/08/17 07:59:59.465957,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'ntlmssp_resume_ccache' registered

[2017/08/17 07:59:59.465972,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'http_basic' registered

[2017/08/17 07:59:59.465982,3] 
../auth/gensec/gensec_start.c:918(gensec_register)

GENSEC backend 'http_ntlm' registered

[2017/08/17 07:59:59.467516,3] ../source3/smbd/negprot.c:730(reply_negprot)

Selected protocol SMB 2.???

[2017/08/17 07:59:59.468111,3] 
../source3/smbd/smb2_negprot.c:290(smbd_smb2_request_process_negprot)

Selected protocol SMB3_11

[2017/08/17 08:00:06.151513,3] 
../source3/smbd/smb2_server.c:3097(smbd_smb2_request_error_ex)

smbd_smb2_request_error_ex: smbd_smb2_request_error_ex: idx[1] 
status[NT_STATUS_LOGON_FAILURE] || at ../source3/smbd/smb2_sesssetup.c:134

[2017/08/17 08:00:06.153192,3] 
../source3/smbd/server_exit.c:246(exit_server_common)

Server exit (NT_STATUS_CONNECTION_RESET)

Log.smbd

[2017/08/17 02:27:26.578214,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 18077 -- ignoring

[2017/08/17 02:42:26.580707,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 19278 -- ignoring

[2017/08/17 02:57:26.585133,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 20546 -- ignoring

[2017/08/17 03:12:26.588487,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 21704 -- ignoring

[2017/08/17 03:27:26.592306,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 22935 -- ignoring

[2017/08/17 03:42:26.594330,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 2162 -- ignoring

[2017/08/17 03:57:26.598090,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 4040 -- ignoring

[2017/08/17 04:12:26.602245,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 5113 -- ignoring

[2017/08/17 04:27:26.606161,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 6269 -- ignoring

[2017/08/17 04:42:26.610297,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 7382 -- ignoring

[2017/08/17 04:57:26.612547,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 8497 -- ignoring

[2017/08/17 05:12:26.615685,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 9614 -- ignoring

[2017/08/17 05:27:26.618609,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 11133 -- ignoring

[2017/08/17 05:42:26.621232,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 12292 -- ignoring

[2017/08/17 05:57:26.625906,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 13379 -- ignoring

[2017/08/17 06:12:26.628955,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 14452 -- ignoring

[2017/08/17 06:27:26.630512,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 15536 -- ignoring

[2017/08/17 06:42:26.634709,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 16600 -- ignoring

[2017/08/17 06:57:26.638292,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 17706 -- ignoring

[2017/08/17 07:12:26.642297,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 18790 -- ignoring

[2017/08/17 07:27:26.644817,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 19881 -- ignoring

[2017/08/17 07:42:26.649127,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 21035 -- ignoring

[2017/08/17 07:57:26.653799,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 22175 -- ignoring

[2017/08/17 08:12:26.656684,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 23303 -- ignoring

[2017/08/17 08:27:26.660355,2] 
../source3/smbd/server.c:794(remove_child_pid)

Could not find child 24397 -- ignoring

smb.conf (global portion only)

[global]
         #debug level = 2
         debug level = 3
         workgroup = RESEARCH
         realm = CHEM.BYU.EDU
         netbios name = CHEM
         kerberos method = dedicated keytab
         dedicated keytab file = FILE:/etc/samba/samba.keytab
         create krb5 conf = no
         log file = /var/log/samba/log.%m

         security = user
         passdb backend = ipasam:ldaps://ipa1.chem.byu.edu
         ldapsam:trusted = yes
         ldap ssl = no
         ldap suffix = dc=chem,dc=byu,dc=edu
         ldap user suffix = cn=users,cn=accounts
         ldap group suffix = cn=groups,cn=accounts

         load printers = no
         cups options = raw
         printcap name = /dev/null

running an ldapsearch yields correct results:

[root@fs-ipa-rhel7 samba]# ldapsearch -Y GSSAPI uid=randym ipaNTHash
SASL/GSSAPI authentication started
SASL username: randym(a)CHEM.BYU.EDU
SASL SSF: 56
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base <dc=chem,dc=byu,dc=edu> (default) with scope subtree
# filter: uid=randym
# requesting: ipaNTHash
#

# randym, users, compat, chem.byu.edu
dn: uid=randym,cn=users,cn=compat,dc=chem,dc=byu,dc=edu

# randym, users, compat, chem.byu.edu
dn: uid=randym,cn=users,cn=compat,dc=chem,dc=byu,dc=edu

# randym, users, accounts, chem.byu.edu
dn: uid=randym,cn=users,cn=accounts,dc=chem,dc=byu,dc=edu

# search result
search: 4
result: 0 Success

# numResponses: 4
# numEntries: 3

I also tried changing my password to see if it was just an NT hash 
issue, but that had not effect either.

Any help would be greatly appreciated.

Randy

-- 
Randy Morgan
CSR
Department of Chemistry and Biochemistry
Brigham Young University
801-422-4100