Hi folks,
after the upgrade from ipa-server.x86_64 4.9.12-9 to version 4.9.12-11 my FreeIPA servers' web interfaces became inaccessible. At login time there is a message
Your session has expired. Please log in again.
I found some other threads about similar problems in this ML. However, the suggested fix to create SIDs
[root@ipa0 log]# /usr/libexec/ipa/oddjob/org.freeipa.server.config-enable-sid --netbios-name EXAMPLE --add-sids Configuring SID generation [1/8]: creating samba domain object Samba domain object already exists [2/8]: adding admin(group) SIDs Admin SID already set, nothing to do Admin group SID already set, nothing to do [3/8]: adding RID bases RID bases already set, nothing to do [4/8]: updating Kerberos config 'dns_lookup_kdc' already set to 'true', nothing to do. [5/8]: activating sidgen task Sidgen task plugin already configured, nothing to do [6/8]: restarting Directory Server to take MS PAC and LDAP plugins changes into account [7/8]: adding fallback group Fallback group already set, nothing to do [8/8]: adding SIDs to existing users and groups This step may take considerable amount of time, please wait.. Done. The ipa-enable-sid command was successful [root@ipa0 log]# echo $? 0
did not help. I still cannot login on the web interface. (Looking at the output it didn't had to do anything, anyway. AFAIR this SID thingy was already done during migration from CentOS 7 to 8, AFAIR).
[root@ipa0 ~]# ipa idrange-find --raw ---------------- 3 ranges matched ---------------- cn: EXAMPLE.DE_id_range ipabaseid: 379400000 ipaidrangesize: 200000 ipabaserid: 379400000 ipasecondarybaserid: 379600000 iparangetype: ipa-local
cn: EXAMPLE.DE_posix ipabaseid: 1000 ipaidrangesize: 99000 ipabaserid: 1000 ipasecondarybaserid: 100000 iparangetype: ipa-local
cn: EXAMPLE.DE_subid_range ipabaseid: 2147483648 ipaidrangesize: 2147352576 ipabaserid: 2147283648 ipanttrusteddomainsid: S-1-5-21-738065-838566-194929194 iparangetype: ipa-ad-trust ---------------------------- Number of entries returned 3 ----------------------------
/var/log/messages shows
Jan 23 13:50:28 ipa0 [6654]: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty) Jan 23 13:50:28 ipa0 [6653]: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty) Jan 23 13:50:31 ipa0 [6654]: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty) Jan 23 13:50:31 ipa0 [6653]: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty)
/var/log/krb5kdc.log
Jan 23 13:50:28 ipa0.example.de krb5kdc[6611](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC: authtime 1706012763, etypes {rep=UNSUPPORTED:(0)} HTTP/ipa0.example.de@EXAMPLE.DE for ldap/ipa0.example.de@EXAMPLE.DE, KDC policy rejects request Jan 23 13:50:28 ipa0.example.de krb5kdc[6611](info): ... CONSTRAINED-DELEGATION s4u-client=<unknown> Jan 23 13:50:28 ipa0.example.de krb5kdc[6611](info): closing down fd 4 Jan 23 13:50:28 ipa0.example.de krb5kdc[6611](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC: authtime 1706012763, etypes {rep=UNSUPPORTED:(0)} HTTP/ipa0.example.de@EXAMPLE.DE for ldap/ipa0.example.de@EXAMPLE.DE, KDC policy rejects request Jan 23 13:50:28 ipa0.example.de krb5kdc[6611](info): ... CONSTRAINED-DELEGATION s4u-client=<unknown> Jan 23 13:50:28 ipa0.example.de krb5kdc[6611](info): closing down fd 4 Jan 23 13:50:30 ipa0.example.de krb5kdc[6611](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: NEEDED_PREAUTH: WELLKNOWN/ANONYMOUS@EXAMPLE.DE for krbtgt/EXAMPLE.DE@EXAMPLE.DE, Additional pre-authentication required Jan 23 13:50:30 ipa0.example.de krb5kdc[6611](info): closing down fd 4 Jan 23 13:50:31 ipa0.example.de krb5kdc[6587](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: ISSUE: authtime 1706014231, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, WELLKNOWN/ANONYMOUS@EXAMPLE.DE for krbtgt/EXAMPLE.DE@EXAMPLE.DE Jan 23 13:50:31 ipa0.example.de krb5kdc[6587](info): closing down fd 4 Jan 23 13:50:31 ipa0.example.de krb5kdc[6611](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: NEEDED_PREAUTH: hdunkel@EXAMPLE.DE for krbtgt/EXAMPLE.DE@EXAMPLE.DE, Additional pre-authentication required Jan 23 13:50:31 ipa0.example.de krb5kdc[6611](info): closing down fd 4 Jan 23 13:50:31 ipa0.example.de krb5kdc[6592](info): AS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: ISSUE: authtime 1706014231, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, hdunkel@EXAMPLE.DE for krbtgt/EXAMPLE.DE@EXAMPLE.DE Jan 23 13:50:31 ipa0.example.de krb5kdc[6592](info): closing down fd 4 Jan 23 13:50:31 ipa0.example.de krb5kdc[6588](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: ISSUE: authtime 1706014231, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, hdunkel@EXAMPLE.DE for HTTP/ipa0.example.de@EXAMPLE.DE Jan 23 13:50:31 ipa0.example.de krb5kdc[6588](info): closing down fd 4 Jan 23 13:50:31 ipa0.example.de krb5kdc[6587](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC: authtime 1706014231, etypes {rep=UNSUPPORTED:(0)} HTTP/ipa0.example.de@EXAMPLE.DE for ldap/ipa0.example.de@EXAMPLE.DE, KDC policy rejects request Jan 23 13:50:31 ipa0.example.de krb5kdc[6587](info): ... CONSTRAINED-DELEGATION s4u-client=<unknown> Jan 23 13:50:31 ipa0.example.de krb5kdc[6587](info): closing down fd 4 Jan 23 13:50:31 ipa0.example.de krb5kdc[6588](info): TGS_REQ (6 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17), camellia256-cts-cmac(26), camellia128-cts-cmac(25)}) 172.19.96.2: S4U2PROXY_EVIDENCE_TKT_WITHOUT_PAC: authtime 1706014231, etypes {rep=UNSUPPORTED:(0)} HTTP/ipa0.example.de@EXAMPLE.DE for ldap/ipa0.example.de@EXAMPLE.DE, KDC policy rejects request Jan 23 13:50:31 ipa0.example.de krb5kdc[6588](info): ... CONSTRAINED-DELEGATION s4u-client=<unknown> Jan 23 13:50:31 ipa0.example.de krb5kdc[6588](info): closing down fd 4
Every helpful hint is highly appreciated.
Harri
Hi Harri,
did you check your admin user, if the attribute is set?
# ipa user-show admin --all | grep ipantsecurityidentifier
Regards, Sören
Hi Soeren,
On 2024-01-23 14:06:11, Sören R. via FreeIPA-users wrote:
Hi Harri,
did you check your admin user, if the attribute is set?
# ipa user-show admin --all | grep ipantsecurityidentifier
The admin user has this attribute set, but my own account used to access the web interface hasn't. I am still trying to find a way how to add this ipantsecurityidentifier attribute to all users, but wasn't there some kind of builtin supposed to fix this automagically?
Still trying to find the right documentation.
Regards
Harri
Does your "admin" User can access the WebUI now? If yes, the fastest way would be to delete and recreate your user. This worked for me.
On Аўт, 23 сту 2024, Harald Dunkel via FreeIPA-users wrote:
Hi Soeren,
On 2024-01-23 14:06:11, Sören R. via FreeIPA-users wrote:
Hi Harri,
did you check your admin user, if the attribute is set?
# ipa user-show admin --all | grep ipantsecurityidentifier
The admin user has this attribute set, but my own account used to access the web interface hasn't. I am still trying to find a way how to add this ipantsecurityidentifier attribute to all users, but wasn't there some kind of builtin supposed to fix this automagically?
No, not automatically because it is a task that goes through all user accounts one by one and fixes them. It also requires to have properly defined ID ranges that cover all uidNumber/gidNumber in user/group entries.
One issue we identified today together with Fedora infrastructure team is that staged users (created with 'ipa stageuser-add') will prevent sidgen plugin to generate entries.
Still trying to find the right documentation.
All documentation was mentioned already in these threads. Please see at https://access.redhat.com/articles/7027037 for more details (needs RHEL subscription, including a free developer subscription).
Hi Alex,
On 2024-01-23 14:41:30, Alexander Bokovoy wrote:
One issue we identified today together with Fedora infrastructure team is that staged users (created with 'ipa stageuser-add') will prevent sidgen plugin to generate entries.
I didn't even know this command.
[root@ipa0 ~]# ipa stageuser-find --------------- 0 users matched --------------- ---------------------------- Number of entries returned 0 ----------------------------
Still trying to find the right documentation.
All documentation was mentioned already in these threads. Please see at https://access.redhat.com/articles/7027037 for more details (needs RHEL subscription, including a free developer subscription).
Thank you for the link.
I found one problem by now: Regular UIDs start with 501 in my environment, for historical reasons. The GIDs are >=1000. When we migrated from good ol' yellow pages to FreeIPA there was no problem with small UIDs. And in the BSD and SYSV years before Linux only the UIDs <100 were reserved for system.
Do I have to migrate the existing users between 501 and 999 to new UIDs
1000? I would like to see an error message showing that this is indeed the
problem, first. Surely I would prefer to just adjust the ID ranges instead of migrating about 90 user accounts.
What would you suggest?
Regards Harri
On Аўт, 23 сту 2024, Harald Dunkel wrote:
Hi Alex,
On 2024-01-23 14:41:30, Alexander Bokovoy wrote:
One issue we identified today together with Fedora infrastructure team is that staged users (created with 'ipa stageuser-add') will prevent sidgen plugin to generate entries.
I didn't even know this command.
[root@ipa0 ~]# ipa stageuser-find
0 users matched
Number of entries returned 0
Still trying to find the right documentation.
All documentation was mentioned already in these threads. Please see at https://access.redhat.com/articles/7027037 for more details (needs RHEL subscription, including a free developer subscription).
Thank you for the link.
I found one problem by now: Regular UIDs start with 501 in my environment, for historical reasons. The GIDs are >=1000. When we migrated from good ol' yellow pages to FreeIPA there was no problem with small UIDs. And in the BSD and SYSV years before Linux only the UIDs <100 were reserved for system.
Do I have to migrate the existing users between 501 and 999 to new UIDs
1000? I would like to see an error message showing that this is indeed the
problem, first. Surely I would prefer to just adjust the ID ranges instead of migrating about 90 user accounts.
What would you suggest?
You can add a new local ID range to cover existing UIDs/GIDs. Make sure to set base RID and secondary base RID when defining a new ID range.
See https://access.redhat.com/articles/7027037 for details.
Hi Alex,
On 2024-01-24 08:13:44, Alexander Bokovoy wrote:
On Аўт, 23 сту 2024, Harald Dunkel wrote:
I found one problem by now: Regular UIDs start with 501 in my environment, for historical reasons. The GIDs are >=1000. When we migrated from good ol' yellow pages to FreeIPA there was no problem with small UIDs. And in the BSD and SYSV years before Linux only the UIDs <100 were reserved for system.
Do I have to migrate the existing users between 501 and 999 to new UIDs
1000? I would like to see an error message showing that this is indeed the
problem, first. Surely I would prefer to just adjust the ID ranges instead of migrating about 90 user accounts.
What would you suggest?
You can add a new local ID range to cover existing UIDs/GIDs. Make sure to set base RID and secondary base RID when defining a new ID range.
See https://access.redhat.com/articles/7027037 for details.
ACK. One question, though: Shouldn't the
ipa config-mod --enable-sid --add-sids
I had run set the missing ipantsecurityidentifier entries at least for the users matching the existing address range from 1000 to 99999? It didn't, AFAICT. It didn't show an error message, either.
Looking at the changes between the previously installed freeipa packages and the version I have right now I got
# rpm -q --changelog ipa-server * Fri Dec 01 2023 Julien Rische jrische@redhat.com - 4.9.12-11 - Generate Kerberos PAC as soon as server installation completed Resolves: RHEL-16532
* Thu Nov 16 2023 Julien Rische jrische@redhat.com - 4.9.12-10 - ipa-kdb: Detect and block Bronze-Bit attacks Resolves: RHEL-16532 - Fix for CVE-2023-5455 Resolves: RHEL-12577
* Wed Oct 04 2023 Julien Rische jrische@redhat.com - 4.9.12-9 : :
4.9.12-9 was the previous version. It worked fine. Fixing the CVEs was the reason for the upgrade. Kerberos authentication is fine in the new setup, too. How comes these changes triggered the problem about missing ipantsecurityidentifier entries? Is this as intended?
Regards
Harri
On Срд, 24 сту 2024, Harald Dunkel wrote:
Hi Alex,
On 2024-01-24 08:13:44, Alexander Bokovoy wrote:
On Аўт, 23 сту 2024, Harald Dunkel wrote:
I found one problem by now: Regular UIDs start with 501 in my environment, for historical reasons. The GIDs are >=1000. When we migrated from good ol' yellow pages to FreeIPA there was no problem with small UIDs. And in the BSD and SYSV years before Linux only the UIDs <100 were reserved for system.
Do I have to migrate the existing users between 501 and 999 to new UIDs
1000? I would like to see an error message showing that this is indeed the
problem, first. Surely I would prefer to just adjust the ID ranges instead of migrating about 90 user accounts.
What would you suggest?
You can add a new local ID range to cover existing UIDs/GIDs. Make sure to set base RID and secondary base RID when defining a new ID range.
See https://access.redhat.com/articles/7027037 for details.
ACK. One question, though: Shouldn't the
ipa config-mod --enable-sid --add-sids
I had run set the missing ipantsecurityidentifier entries at least for the users matching the existing address range from 1000 to 99999? It didn't, AFAICT. It didn't show an error message, either.
Looking at the changes between the previously installed freeipa packages and the version I have right now I got
# rpm -q --changelog ipa-server
- Fri Dec 01 2023 Julien Rische jrische@redhat.com - 4.9.12-11
- Generate Kerberos PAC as soon as server installation completed
Resolves: RHEL-16532
- Thu Nov 16 2023 Julien Rische jrische@redhat.com - 4.9.12-10
- ipa-kdb: Detect and block Bronze-Bit attacks
Resolves: RHEL-16532
- Fix for CVE-2023-5455
Resolves: RHEL-12577
- Wed Oct 04 2023 Julien Rische jrische@redhat.com - 4.9.12-9
: :
4.9.12-9 was the previous version. It worked fine. Fixing the CVEs was the reason for the upgrade. Kerberos authentication is fine in the new setup, too. How comes these changes triggered the problem about missing ipantsecurityidentifier entries? Is this as intended?
Yes, it was intended. Bronze-bit (CVE-2020-17049) is about Kerberos constrained delegation (S4U extensions). S4U Kerberos extensions require presence of MS-PAC structures in Kerberos tickets and to generate MS-PAC we have to have SIDs. This was already enforced in MIT Kerberos 1.20+ in RHEL 9 for quite some time.
New RHEL IdM deployments do have SIDs associated with user/group accounts since RHEL 8.5. If you have older installations, you might not have them. If you have earlier established trust to Active Directory, your IPA configuration might already permit generating SIDs for new users since ~2013. Still, some user and group accounts from 'legacy' configurations might not have them and those will be denied accessing any Kerberos resources protected with constrained delegation. IPA API is one such resource.
As discussions on this mailing list show, there are plenty of edge cases, mostly around 'legacy' UID/GIDs and missing ID ranges that would have covered those IDs. Or ID ranges missing SID-specific attributes (base RID and secondary base RID) that prevent use of those ranges to generate SIDs.
KCS https://access.redhat.com/articles/7027037 describes a lot of those details, so I would recommend reading through it and investigating your ID range configuration based on those details.
We also have upstream document that outlines ID mapping usage in FreeIPA: https://freeipa.readthedocs.io/en/latest/designs/id-mapping.html
Hi Alex,
On 2024-01-24 10:00:42, Alexander Bokovoy via FreeIPA-users wrote:
KCS https://access.redhat.com/articles/7027037 describes a lot of those details, so I would recommend reading through it and investigating your ID range configuration based on those details.
We also have upstream document that outlines ID mapping usage in FreeIPA: https://freeipa.readthedocs.io/en/latest/designs/id-mapping.html
I finally found the error messages: They are in the LDAP log, in /var/log/dirsrv/slapd-EXAMPLE-DE/errors. There were a few errors to fix (a GID 100 outside of the defined idranges, a duplicate GID, ...), but my FreeIPA seems to be running again.
Thank you very much for your support
Regards
Harri
Alexander Bokovoy via FreeIPA-users freeipa-users@lists.fedorahosted.org writes:
As discussions on this mailing list show, there are plenty of edge cases, mostly around 'legacy' UID/GIDs and missing ID ranges that would have covered those IDs. Or ID ranges missing SID-specific attributes (base RID and secondary base RID) that prevent use of those ranges to generate SIDs. KCS https://access.redhat.com/articles/7027037 describes a lot of those details, so I would recommend reading through it and investigating your ID range configuration based on those details.
Would it be helpful to have ipa-healthcheck or checkipaconsistency warn about that? During ipa-server-upgrade is too late and it runs most of the time in the background...
Jochen "I also needed to fix my id ranges"
freeipa-users@lists.fedorahosted.org
-
Alexander Bokovoy -
Harald Dunkel -
Jochen Kellner -
Sören R.