migrating NIS passwords to FreeIPA in Fedora 33 with {CRYPT} and RH sample nis-users.sh script

[Robert Kudyba]

Running FreeIPS 4.8.10-6,  5.10.10-200.fc33.x86_64

I'm using the nis-users.sh script from
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/...

First note that the part (inside 'Now create this entry') that has
--gecos='$gecos' actually inserts $gecos into the FreeIPA record. Also a
simple fix to insert a first and last name would be:
first=$(echo $gecos | sed -e 's/\(.*\), \(.*$\)/\2 \1/' | awk '{print
$1}')
last=$(echo $gecos | sed -e 's/\(.*\), \(.*$\)/\2 \1/' | awk '{print
$NF}')

and adding in the "Now create this entry section":
--first=$first --last=$last

I'm trying to migrate the passwords from NIS so that they are merged in the
/etc/passwd file. (this is a test server). I followed Rob C's previous tips
from here
<https://freeipa-users.redhat.narkive.com/vTJsopZ5/problem-migrating-passw...
 and here
<https://www.redhat.com/archives/freeipa-users/2013-April/msg00058.html>;.

Not sure it matters but in  /etc/libuser.conf, crypt_style = sha512

In the script I added:
password1=$(echo $line | cut -f2 -d:)
and in the Now create this entry section:
--setattr "userpassword='{CRYPT}$password1'"

Here's what gets logged when debug is turned on:
[Tue Feb 02 22:08:52.541857 2021] [wsgi:error] [pid 16097:tid 16365]
[remote x.x.x.x:59726 <http://150.108.64.156:59726/>] ipa: INFO:
[jsonserver_session] admin(a)OURDOMAIN.EDU <admin(a)OLDDSM.DSM.FORDHAM.EDU&gt;:
user_add/1('john', givenname='John', sn='Smith',
homedirectory='/home/smith', gecos="'John Smith'",
loginshell='/bin/tcsh',
uidnumber=5319, gidnumber=150,
setattr=("userpassword='{CRYPT}the-actual-hash-of-the-password'",),
version='2.239'): SUCCESS

So does that appear that {CRYPT} is not being interpreted? I also added
some debug:
echo "Password hash value is $password1"

And what prints is the original hash, sans {CRYPT}.

So to test this outside of the script I added a test user:
ipa user-add --first=test --last=user --setattr userpassword='{CRYPT}
the-actual-hash-of-the-password' testuser

Then I ran the following and the password worked:
ldapsearch -x -D 'uid=testuser,cn=users,cn=accountsdc=ourdomain,dc=edu' -W

# testuser, users, accounts, ourdomain.edu <http://olddsm.dsm.fordham.edu/>
dn: uid=testuser,cn=users,cn=accounts,dc=ourdomain,dc=edu
givenName: test
sn: user
uid: testuser
cn: test user
displayName: test user
initials: tu
gecos: test user
krbPrincipalName: testuser(a)OURDOMAIN.EDU <testuser(a)OLDDSM.DSM.FORDHAM.EDU&gt;
objectClass: top
objectClass: person
objectClass: organizationalperson
objectClass: inetorgperson
objectClass: inetuser
objectClass: posixaccount
objectClass: krbprincipalaux
objectClass: krbticketpolicyaux
objectClass: ipaobject
objectClass: ipasshuser
objectClass: fasuser
objectClass: ipaSshGroupOfPubKeys
objectClass: mepOriginEntry
loginShell: /bin/sh
homeDirectory: /home/testuser
mail: testuser(a)ourdomain.edu <testuser(a)olddsm.dsm.fordham.edu&gt;
krbCanonicalName: testuser(a)OURDOMAIN.EDU <testuser(a)OLDDSM.DSM.FORDHAM.EDU&gt;
ipaUniqueID: 34ee1f48-65d2-11eb-8c33-001ec9ab7ef0
uidNumber: 1520800007
gidNumber: 1520800007
memberOf: cn=ipausers,cn=groups,cn=accounts,dc=ourdomain,dc=edu
krbLastPwdChange: 20210203034524Z
krbPasswordExpiration: 20210504034524Z

# testuser, groups, accounts, ourdomain.edu <http://olddsm.dsm.fordham.edu/>
dn: cn=testuser,cn=groups,cn=accounts,dc=ourdomain,dc=edu
objectClass: posixgroup
objectClass: ipaobject
objectClass: mepManagedEntry
objectClass: top
cn: testuser
gidNumber: 1520800007
description: User private group for testuser
mepManagedBy: uid=testuser,cn=users,cn=accounts,dc=ourdomain,dc
 =edu
ipaUniqueID: 34f39b4e-65d2-11eb-8c33-001ec9ab7ef0

# search result
search: 2
result: 0 Success

Is it still possible to do this in the current versions?

Thanks,

Rob

[Rob Crittenden]

Robert Kudyba via FreeIPA-users wrote:
>  Running FreeIPS 4.8.10-6,  5.10.10-200.fc33.x86_64
>  
>  I'm using the nis-users.sh script
>
> from https://access.redhat.com/documentation/en-us/red_hat_enterprise_lin...
>  
>  First note that the part (inside 'Now create this entry') that
>  has --gecos='$gecos' actually inserts$gecos into the FreeIPA record.
>  Also a simple fix to insert a first and last name would be:
>  first=$(echo $gecos | sed -e 's/\(.*\), \(.*$\)/\2 \1/' | awk '{print
> $1}')
>  last=$(echo $gecos | sed -e 's/\(.*\), \(.*$\)/\2 \1/' | awk '{print
> $NF}')
>  
>  and adding in the "Now create this entry section":
>  --first=$first --last=$last
>  
>  I'm trying to migrate the passwords from NIS so that they are merged in
>  the /etc/passwd file. (this is a test server). I followed Rob C's
>  previous tips from here
>
> <https://freeipa-users.redhat.narkive.com/vTJsopZ5/problem-migrating-passw...
>  <https://www.redhat.com/archives/freeipa-users/2013-April/msg00058.html>. 
>  
>  Not sure it matters but in  /etc/libuser.conf, crypt_style = sha512
>  
>  In the script I added:
>  password1=$(echo $line | cut -f2 -d:)
>  and in the Now create this entry section:
>  --setattr "userpassword='{CRYPT}$password1'"
>   
>  Here's what gets logged when debug is turned on:
>  [Tue Feb 02 22:08:52.541857 2021] [wsgi:error] [pid 16097:tid 16365]
>  [remote x.x.x.x:59726 <http://150.108.64.156:59726/>] ipa: INFO:
>  [jsonserver_session] admin(a)OURDOMAIN.EDU
>  <mailto:[e-mail]>: user_add/1('john',
>  givenname='John', sn='Smith', homedirectory='/home/smith',
> gecos="'John
>  Smith'", loginshell='/bin/tcsh', uidnumber=5319, gidnumber=150,
>  setattr=("userpassword='{CRYPT}the-actual-hash-of-the-password'",),
>  version='2.239'): SUCCESS
>  
>  So does that appear that {CRYPT} is not being interpreted? I also added
>  some debug:
>  echo "Password hash value is $password1"
>  
>  And what prints is the original hash, sans {CRYPT}.
>  
>  So to test this outside of the script I added a test user:
>  ipa user-add --first=test --last=user --setattr userpassword='{CRYPT}
>  the-actual-hash-of-the-password' testuser
>  
>  Then I ran the following and the password worked:
>  ldapsearch -x -D 'uid=testuser,cn=users,cn=accountsdc=ourdomain,dc=edu' -W
>  
>  # testuser, users, accounts, ourdomain.edu <http://olddsm.dsm.fordham.edu/>
>  dn: uid=testuser,cn=users,cn=accounts,dc=ourdomain,dc=edu
>  givenName: test
>  sn: user
>  uid: testuser
>  cn: test user
>  displayName: test user
>  initials: tu
>  gecos: test user
>  krbPrincipalName: [e-mail]
>  <mailto:[e-mail]>
>  objectClass: top
>  objectClass: person
>  objectClass: organizationalperson
>  objectClass: inetorgperson
>  objectClass: inetuser
>  objectClass: posixaccount
>  objectClass: krbprincipalaux
>  objectClass: krbticketpolicyaux
>  objectClass: ipaobject
>  objectClass: ipasshuser
>  objectClass: fasuser
>  objectClass: ipaSshGroupOfPubKeys
>  objectClass: mepOriginEntry
>  loginShell: /bin/sh
>  homeDirectory: /home/testuser
>  mail: [e-mail] <mailto:[e-mail]>
>  krbCanonicalName: [e-mail]
>  <mailto:[e-mail]>
>  ipaUniqueID: 34ee1f48-65d2-11eb-8c33-001ec9ab7ef0
>  uidNumber: 1520800007
>  gidNumber: 1520800007
>  memberOf: cn=ipausers,cn=groups,cn=accounts,dc=ourdomain,dc=edu
>  krbLastPwdChange: 20210203034524Z
>  krbPasswordExpiration: 20210504034524Z
>  
>  # testuser, groups, accounts, ourdomain.edu <http://olddsm.dsm.fordham.edu/>
>  dn: cn=testuser,cn=groups,cn=accounts,dc=ourdomain,dc=edu
>  objectClass: posixgroup
>  objectClass: ipaobject
>  objectClass: mepManagedEntry
>  objectClass: top
>  cn: testuser
>  gidNumber: 1520800007
>  description: User private group for testuser
>  mepManagedBy: uid=testuser,cn=users,cn=accounts,dc=ourdomain,dc
>   =edu
>  ipaUniqueID: 34f39b4e-65d2-11eb-8c33-001ec9ab7ef0
>  
>  # search result
>  search: 2
>  result: 0 Success
>  
>  Is it still possible to do this in the current versions? 
I think the problem is your double-quoting around the setattr value.
Drop that and it works for me.

I'm not entirely sure how that is messing things up in the API but yeah
it looks like it isn't seeing the {CRYPT} and encoding the whole thing
as a raw password (probably including the {CRYPT}).

rob

[Robert Kudyba]

>  I think the problem is your double-quoting around the setattr value.
>  Drop that and it works for me.
>
>  I'm not entirely sure how that is messing things up in the API but yeah
>  it looks like it isn't seeing the {CRYPT} and encoding the whole thing
>  as a raw password (probably including the {CRYPT}).
>  
So now I put:
ipa user-add $username --first=$first --last=$last \
             --setattr userpassword='{CRYPT}$password1' --gidnumber=$gid
--uid=$uid --gecos="'$gecos'" --homedir=$homedir \
             --shell=$shell

But now the debug log has:
 raw: user_add('jmist', givenname='John', sn='Smith',
homedirectory='/home/jsmith', gecos="'John Smith'",
loginshell='/bin/tcsh',
uidnumber=5278, gidnumber=200, *setattr=*('*userpassword={CRYPT}$password*1',),
version='2.239')
[Wed Feb 03 12:02:59.129692 2021] [wsgi:error] [pid 182762:tid 183116]
[remote x.x.x.x:38886] ipa: DEBUG: user_add('jsmith', givenname='John',
sn='Smith', cn='John Smith', displayname='John Smith',
initials='JS',
homedirectory='/home/jsmith', gecos="'John Smith'",
loginshell='/bin/tcsh',
krbprincipalname=(ipapython.kerberos.Principal('jsmith(a)MYDOMAIN.EDU'),),
random=False, uidnumber=5278, gidnumber=200,*
setattr=('userpassword={CRYPT}$password1',)*, noprivate=False, all=False,
raw=False, version='2.239', no_members=False)

So how do I format the --setattr userpassword without quotes?

[Jochen Kellner]

Robert Kudyba via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org&gt;
writes:

>  So now I put:
>  ipa user-add $username --first=$first --last=$last \
>               --setattr userpassword='{CRYPT}$password1' --gidnumber=$gid 
Try:
              --setattr "userpassword={CRYPT}$password1" --gidnumber=$gid


Jochen

-- 
This space is intentionally left blank.

[Robert Kudyba]

On Wed, Feb 3, 2021 at 12:18 PM Jochen Kellner <jochen(a)jochen.org&gt; wrote:

>  Robert Kudyba via FreeIPA-users
> <freeipa-users(a)lists.fedorahosted.org&gt;
>  writes:
>
>  > So now I put:
>  > ipa user-add $username --first=$first --last=$last \
>  >              --setattr userpassword='{CRYPT}$password1' --gidnumber=$gid
>
>  Try:
>                --setattr "userpassword={CRYPT}$password1" --gidnumber=$gid
>  
Nice, that worked as well as:
   --setattr userpassword="{CRYPT}$password1" --gidnumber=$gid

Now any idea why the original  '$gecos' inserts the actual string  $gecos
into FreeIPA/LDAP?

Logs also spit out this warning after every user is added:
Failed to set perms (3140) on file (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU)!,
referer: https://oudomain.edu/ipa/xml

[Simo Sorce]

On Wed, 2021-02-03 at 12:34 -0500, Robert Kudyba via FreeIPA-users
wrote:
>  On Wed, Feb 3, 2021 at 12:18 PM Jochen Kellner
> <jochen(a)jochen.org&gt; wrote:
>  
>  > Robert Kudyba via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org&gt;
>  > writes:
>  > 
>  > > So now I put:
>  > > ipa user-add $username --first=$first --last=$last \
>  > >              --setattr userpassword='{CRYPT}$password1'
> --gidnumber=$gid
>  > 
>  > Try:
>  >               --setattr "userpassword={CRYPT}$password1" --gidnumber=$gid
>  > 
>  
>  Nice, that worked as well as:
>     --setattr userpassword="{CRYPT}$password1" --gidnumber=$gid
>  
>  Now any idea why the original  '$gecos' inserts the actual string  $gecos
>  into FreeIPA/LDAP? 
It's a shell issue, single quotes prevents any argument expansion, use
double quotes.

>  Logs also spit out this warning after every user is added:
>  Failed to set perms (3140) on file (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU)!,
>  referer: https://oudomain.edu/ipa/xml
>  _______________________________________________
>  FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>  To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]... 
-- 
Simo Sorce
RHEL Crypto Team
Red Hat, Inc

[Robert Kudyba]

>  > Now any idea why the original  '$gecos' inserts the actual string  $gecos
>  > into FreeIPA/LDAP?
>
>  It's a shell issue, single quotes prevents any argument expansion, use
>  double quotes.
>  
Sure but using just double quotes $gecos is still added to the FreeIPA
record. I can at least import it with "'$gecos'" but the script then
imports the single quotes as a part of the record.

We also noticed if a NIS user has a middle initial the script errors with:
ipa: ERROR: command 'user_add' takes at most 1 argument

So is there some escape character or something that works around this that
isn't documented? I don't see anything with ip help user-add


>  Logs also spit out this warning after every user is added: 
>  Failed to set perms (3140) on file
> (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU
>  )!,
>  
I added a comment to issue 7032
https://pagure.io/freeipa/issue/7032#comment-713784 not sure if this is
still a "won't fix" issue?

[Rob Crittenden]

Robert Kudyba wrote:
>  
>      > Now any idea why the original  '$gecos' inserts the actual string 
>      $gecos
>      > into FreeIPA/LDAP?
>  
>      It's a shell issue, single quotes prevents any argument expansion, use
>      double quotes.
>  
>  
>  Sure but using just double quotes $gecos is still added to the FreeIPA
>  record. I can at least import it with "'$gecos'" but the script then
>  imports the single quotes as a part of the record. 
I assume you tried "$gecos"?

>  We also noticed if a NIS user has a middle initial the script errors
> with:
>  ipa: ERROR: command 'user_add' takes at most 1 argument 
There should be a way to file a doc bug within the docs themselves.

The script was never meant to be bullet-proof. It was provided for
guidance purposes. There are *lots* of corner cases with names.

>  So is there some escape character or something that works around
> this
>  that isn't documented? I don't see anything withip help user-add
>  
>  
>      > Logs also spit out this warning after every user is added: 
>  
>      > Failed to set perms (3140) on file
>      (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU <mailto:[e-mail]>)!,
>  
>  
>  I added a comment to issue
>  7032 https://pagure.io/freeipa/issue/7032#comment-713784 not sure if
>  this is still a "won't fix" issue?  
It's not won't fix, it's just extremely low in priority since it doesn't
affect operation.

rob

[Robert Kudyba]

On Thu, Feb 4, 2021 at 11:47 AM Rob Crittenden <rcritten(a)redhat.com&gt; wrote:

>  Robert Kudyba wrote:
>  >
>  >     > Now any idea why the original  '$gecos' inserts the actual string
>  >     $gecos
>  >     > into FreeIPA/LDAP?
>  >
>  >     It's a shell issue, single quotes prevents any argument expansion,
>  use
>  >     double quotes.
>  >
>  >
>  > Sure but using just double quotes $gecos is still added to the FreeIPA
>  > record. I can at least import it with "'$gecos'" but the script
> then
>  > imports the single quotes as a part of the record.
>
>  I assume you tried "$gecos"?
>  
Yes and I found a fix. All that is needed is to surround the echo command
with double quotes at the top of the script where username is set:
username="$(echo $line | cut -f1 -d:)"

>  We also noticed if a NIS user has a middle initial the script errors
> with:
>  > ipa: ERROR: command 'user_add' takes at most 1 argument
>
>  There should be a way to file a doc bug within the docs themselves.
>  
Indeed perhaps to add comments, but then those are never spam-proof.

The script was never meant to be bullet-proof. It was provided for
>  guidance purposes. There are *lots* of corner cases with names.
>  
Understood but it's a great launching point. I can share what my colleague
added with a couple awks to handle names in NIS that have middle initial,
"last name, first name" and other permutations:

    # Change Last, First to First Last.  (Fill in dummy for empty gecos.)
    if [ -z "$gecos" ]; then
        firstlast='First Last'
    else
        firstlast=$(echo $gecos | sed -e 's/\(.*\), \(.*$\)/\2 \1/')
    fi

    # Extract First and Last into separate variables
        first=$(echo $firstlast | awk '/^(\w|[-'\''])+ \w\. / { print $1,
$2; } \
                                   /^(\w|[-'\''])+ (\w|[-'\''])+(
|$)/ {
print $1; }' )
#echo this dollar 1 $1 this is dollar 2 $2
        last=$(echo $firstlast  | awk 'BEGIN {ORS=" ";} \
                                   /^(\w|[-'\''])+ \w\. / { for (i=3;
i<=NF; i++) print $i; } \
                                   /^(\w|[-'\''])+ (\w|[-'\''])+(
|$)/ {
for (i=2; i<=NF; i++) print $i; }' \
                                | sed 's/ *$//' )

>  >     > Logs also spit out this warning after every user is added:
>  >
>  >     > Failed to set perms (3140) on file
>  >     (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU <mailto:[e-mail]
>  >)!,
>  >
>  >
>  > I added a comment to issue
>  > 7032
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__pagure.io_freeipa_is...
>  not sure if
>  > this is still a "won't fix" issue?
>
>  It's not won't fix, it's just extremely low in priority since it doesn't
>  affect operation.
>  
Thanks for acknowledging. Hope I added some value to future users that
might find this thread.

[Jochen Kellner]

Hi,

Robert Kudyba via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org&gt;
writes:

>  Yes and I found a fix. All that is needed is to surround the echo
> command
>  with double quotes at the top of the script where username is set:
>  username="$(echo $line | cut -f1 -d:)" 
For some of these errors using shellcheck might help. Not bulletproof,
but helpful.

Jochen

-- 
This space is intentionally left blank.