I am running an F23 machine (paxos) with a C6.7 dhcp/ddns server (maui) - both fully updated I have the ability to "RESET" the dhcp/ddns to a default condition on the server(maui)
Aside I have another F23 client (naxos) that has NOT been fully updated and works correctly I also have an Android smart phone that works correctly and updates the DNS whenever it is reconnected or powered up.
Following a "RESET" of the server and a clean boot of an F23 client (paxos) things are fine - the client (paxos) shows dhclient running with the lease file listed and server (maui) messages are as shown
First boot from clean - hdclient is running on paxos /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eno1.pid \ -lf /var/lib/NetworkManager/dhclient-37107b25-12bf-4de0-935e-12b9b6062fb4-eno1.lease \ -cf /var/lib/NetworkManager/dhclient-eno1.conf eno1
[root@paxos:~]$ ls -l /var/lib/NetworkManager/dhclient-37107b25-12bf-4de0-935e-12b9b6062fb4-eno1.lease -rw-r--r--. 1 root root 521 Jan 25 09:54 /var/lib/NetworkManager/dhclient-37107b25-12bf-4de0-935e-12b9b6062fb4-eno1.lease [root@paxos:~]$ cat /var/lib/NetworkManager/dhclient-37107b25-12bf-4de0-935e-12b9b6062fb4-eno1.lease default-duid "\000\001\000\001\0368\255;x$\257:~:"; lease { interface "eno1"; fixed-address 148.197.29.130; option subnet-mask 255.255.255.0; option routers 148.197.29.254; option dhcp-lease-time 86400; option dhcp-message-type 5; option domain-name-servers 148.197.29.5,212.104.130.9; option dhcp-server-identifier 148.197.29.5; option broadcast-address 148.197.29.255; option domain-name "jaa.org.uk"; renew 1 2016/01/25 21:27:07; rebind 2 2016/01/26 06:54:04; expire 2 2016/01/26 09:54:04; }
Associated messages file on server maui Jan 25 09:54:04 maui dhcpd: DHCPDISCOVER from 78:24:af:3a:7e:3a via eth0 Jan 25 09:54:05 maui dhcpd: DHCPOFFER on 148.197.29.130 to 78:24:af:3a:7e:3a (paxos) via eth0 Jan 25 09:54:05 maui named[29523]: client 148.197.29.5#39074: updating zone 'jaa.org.uk/IN': adding an RR at 'paxos.jaa.org.uk' A Jan 25 09:54:05 maui named[29523]: client 148.197.29.5#39074: updating zone 'jaa.org.uk/IN': adding an RR at 'paxos.jaa.org.uk' TXT Jan 25 09:54:05 maui dhcpd: Added new forward map from paxos.jaa.org.uk to 148.197.29.130 Jan 25 09:54:05 maui named[29523]: client 148.197.29.5#55776: updating zone '29.197.148.in-addr.arpa/IN': deleting rrset at '130.29.197.148.in-addr.arpa' PTR Jan 25 09:54:05 maui named[29523]: client 148.197.29.5#55776: updating zone '29.197.148.in-addr.arpa/IN': adding an RR at '130.29.197.148.in-addr.arpa' PTR Jan 25 09:54:05 maui dhcpd: added reverse map from 130.29.197.148.in-addr.arpa. to paxos.jaa.org.uk Jan 25 09:54:05 maui dhcpd: DHCPREQUEST for 148.197.29.130 (148.197.29.5) from 78:24:af:3a:7e:3a (paxos) via eth0 Jan 25 09:54:05 maui dhcpd: DHCPACK on 148.197.29.130 to 78:24:af:3a:7e:3a (paxos) via eth0 #################################################################################################### Reboot here
/sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eno1.pid -lf \ /var/lib/NetworkManager/dhclient-81ded380-936a-405c-83f2-bfd5b980f69d-eno1.lease \ -cf /var/lib/NetworkManager/dhclient-eno1.conf eno1
[root@paxos:~]$ ls -l /var/lib/NetworkManager/dhclient-81ded380-936a-405c-83f2-bfd5b980f69d-eno1.lease -rw-r--r--. 1 root root 524 Jan 25 10:26 /var/lib/NetworkManager/dhclient-81ded380-936a-405c-83f2-bfd5b980f69d-eno1.lease [root@paxos:~]$ cat /var/lib/NetworkManager/dhclient-81ded380-936a-405c-83f2-bfd5b980f69d-eno1.lease default-duid "\000\001\000\001\0368\264\335x$\257:~:"; lease { interface "eno1"; fixed-address 148.197.29.131; option subnet-mask 255.255.255.0; option routers 148.197.29.254; option dhcp-lease-time 86400; option dhcp-message-type 5; option domain-name-servers 148.197.29.5,212.104.130.9; option dhcp-server-identifier 148.197.29.5; option broadcast-address 148.197.29.255; option domain-name "jaa.org.uk"; renew 1 2016/01/25 22:15:10; rebind 2 2016/01/26 07:26:38; expire 2 2016/01/26 10:26:38; }
Associated messages file on maui
Jan 25 10:26:37 maui dhcpd: DHCPDISCOVER from 78:24:af:3a:7e:3a via eth0 Jan 25 10:26:38 maui dhcpd: DHCPOFFER on 148.197.29.131 to 78:24:af:3a:7e:3a (paxos) via eth0 Jan 25 10:26:38 maui named[29523]: client 148.197.29.5#56622: updating zone 'jaa.org.uk/IN': update unsuccessful: paxos.jaa.org.uk: 'name not in use' prerequisite not satisfied (YXDOMAIN) Jan 25 10:26:38 maui named[29523]: client 148.197.29.5#40701: updating zone 'jaa.org.uk/IN': update unsuccessful: paxos.jaa.org.uk/TXT: 'RRset exists (value dependent)' prerequisite not satisfied (NXRRSET) Jan 25 10:26:38 maui dhcpd: Forward map from paxos.jaa.org.uk to 148.197.29.131 FAILED: Has an address record but no DHCID, not mine. Jan 25 10:26:38 maui dhcpd: DHCPREQUEST for 148.197.29.131 (148.197.29.5) from 78:24:af:3a:7e:3a (paxos) via eth0 Jan 25 10:26:38 maui dhcpd: DHCPACK on 148.197.29.131 to 78:24:af:3a:7e:3a (paxos) via eth0 Jan 25 10:27:29 maui rpc.mountd[1815]: refused mount request from 148.197.29.131 for /home/.hidden (/): not exported Jan 25 10:29:30 maui rpc.idmapd[1860]: nss_getpwnam: name '202' does not map into domain 'jaa.org.uk' ####################################################################################################
This is very bad news as the IP address of the client has been changed but DNS is still pointing to the old IP address!
Things that may be important
1. The name of the lease file is different between the first boot and the second dhclient-37107b25-12bf-4de0-935e-12b9b6062fb4-eno1.lease dhclient-81ded380-936a-405c-83f2-bfd5b980f69d-eno1.lease
2. The default-duid value in these files is different Hence giving the "no DHCID, not mine" error above I assume default-duid "\000\001\000\001\0368\255;x$\257:~:"; default-duid "\000\001\000\001\0368\264\335x$\257:~:";
In the case of the working client (naxos) then the name of the lease file remains the same between boots and the default-duid is the same within that file
A typical reboot request from the working F23 machine (naxos) is shown here Jan 25 12:14:50 maui named[25341]: client 148.197.29.5#47180: updating zone 'jaa.org.uk/IN': update unsuccessful: naxos.jaa.org.uk: 'name not in use' prerequisite not satisfied (YXDOMAIN) Jan 25 12:14:50 maui named[25341]: client 148.197.29.5#53787: updating zone 'jaa.org.uk/IN': deleting rrset at 'naxos.jaa.org.uk' A Jan 25 12:14:50 maui named[25341]: client 148.197.29.5#53787: updating zone 'jaa.org.uk/IN': adding an RR at 'naxos.jaa.org.uk' A Jan 25 12:14:50 maui dhcpd: Added new forward map from naxos.jaa.org.uk to 148.197.29.231 Jan 25 12:14:50 maui named[25341]: client 148.197.29.5#41058: updating zone '29.197.148.in-addr.arpa/IN': deleting rrset at '231.29.197.148.in-addr.arpa' PTR Jan 25 12:14:50 maui named[25341]: client 148.197.29.5#41058: updating zone '29.197.148.in-addr.arpa/IN': adding an RR at '231.29.197.148.in-addr.arpa' PTR Jan 25 12:14:50 maui dhcpd: added reverse map from 231.29.197.148.in-addr.arpa. to naxos.jaa.org.uk Jan 25 12:14:50 maui dhcpd: DHCPREQUEST for 148.197.29.231 from 78:24:af:9a:6a:72 (naxos) via eth0 Jan 25 12:14:50 maui dhcpd: DHCPACK on 148.197.29.231 to 78:24:af:9a:6a:72 (naxos) via eth0
3. No DHCPDISCOVER is issued by the client on the working machine naxos
A typical request from a smart phone is shown here Jan 25 12:20:08 maui named[25341]: client 148.197.29.5#57657: updating zone 'jaa.org.uk/IN': adding an RR at 'android-????????????.jaa.org.uk' A Jan 25 12:20:08 maui named[25341]: client 148.197.29.5#57657: updating zone 'jaa.org.uk/IN': adding an RR at 'android-????????????.jaa.org.uk' TXT Jan 25 12:20:08 maui dhcpd: Added new forward map from android-95490d78aa645882.jaa.org.uk to 148.197.29.175 Jan 25 12:20:08 maui named[25341]: client 148.197.29.5#60372: updating zone '29.197.148.in-addr.arpa/IN': deleting rrset at '175.29.197.148.in-addr.arpa' PTR Jan 25 12:20:08 maui named[25341]: client 148.197.29.5#60372: updating zone '29.197.148.in-addr.arpa/IN': adding an RR at '175.29.197.148.in-addr.arpa' PTR Jan 25 12:20:08 maui dhcpd: added reverse map from 175.29.197.148.in-addr.arpa. to android-???????????????.jaa.org.uk Jan 25 12:20:08 maui dhcpd: DHCPREQUEST for 148.197.29.175 from 00:ae:fa:41:eb:3d via eth0 Jan 25 12:20:08 maui dhcpd: DHCPACK on 148.197.29.175 to 00:ae:fa:41:eb:3d (android-?????????????) via eth0
3. Again no DHCPDISCOVER is issued by the smart phone client
I have tried downgrading the following on the failing client to no avail NetworkManager, dhcp-client, nm-connection-editor I have tried booting with an older kernel
4. I am using ddns-update-style interim; on the server as I believe that dhcp-4.1.1-49.P1.el6.centos.x86_6 does not support "standard" interim "uses TXT RRs instead of DHCID RRs" However the error message does refer to "Has an address record but no DHCID, not mine."
I may have somehow screwed up the configuration on the failing machine or the server but I don't think so.
Any help/advice gratefully received!
John
On Mon, Jan 25, 2016 at 5:39 AM, Dr J Austin ja@jaa.org.uk wrote:
Jan 25 10:26:38 maui dhcpd: Forward map from paxos.jaa.org.uk to 148.197.29.131 FAILED: Has an address record but no DHCID, not mine.
This message means that the DHCP server checks DNS and finds there is already an A record for paxos.jaa.org.uk, but there is no matching TXT record to show that this A record was made by this DHCP server. For example, my workstation at work:
# dig snowcrash.scd.ucar.edu any [comments deleted] snowcrash.scd.ucar.edu. 3600 IN A 128.117.10.112 snowcrash.scd.ucar.edu. 3600 IN TXT "<long ugly hex string>"
When a workstation comes online and requests an address, the DHCP server will update the DNS, assuming that the TXT record exists and has a value that shows it was made by the same DHCP server that is trying to do an update. The "no DHCID, not mine" message means the A record is there but the TXT record is not, therefore the DHCP server doesn't believe it has the authority to delete the A record and install a new one.
The only way I've found to fix this is to use something like nsupdate(8) to delete the offending A record, then reconnect the workstation. That should cause the correct A and TXT records to be added to DNS. If you have a small zone that doesn't change often, another option might be to get the DNS server to dump the current zone, edit that dumped zone to remove the offending A record, move the ".jnl" file out of the way, put the edited zone file in to the place declared in the "file" parameter for this zone in named.conf, then reload the DNS server. (I can't remember how to get a dump of the zone, but I remember doing it in the past. Since we have a very busy DHCP server at work, I always just use nsupdate(8) to modify the active zone).
--Greg
Allegedly, on or about 25 January 2016, Greg Woods sent:
(I can't remember how to get a dump of the zone, but I remember doing it in the past.
Simply stopping the nameserver ought to cause it to reconcile its records on file. That's what I do when I've struck a DNS/DHCP foul-up. Stop DHCP server, Stop BIND, edit records, increment the serial number, restart BIND, restart DHCP server.
That sort of thing tends to happen when you're experimenting, and haven't invented a new IP outside of the DHCP pool.
e.g. If your LAN uses 192.168.0.0, then set aside a range to be dynamic, such as 192.168.0.1 to 192.168.0.100, and configure your DHCP server to only dole out those addresses to dynamic clients. For static addresses, whether individually set on the client equipment, or handed out as fixed DHCP addresses, use IPs outside of that range.
Other things that confuse DHCP/DNS server combinations are dual-boot PCs, where on one OS the DHCP client sends out prefix codes that the other does not, and the DHCP server looks at those codes as well as MAC addresses, and doesn't give the same PC the same address for each OS. Which is logical enough, except that it doesn't properly reset the one it'd previously doled out to the same PC. It gets truly messy if the PC uses the same hostname on both OSs. I've seen reverse IP records stay stale, and forward ones updated.
On Tue, 2016-01-26 at 14:05 +1030, Tim wrote:
Allegedly, on or about 25 January 2016, Greg Woods sent:
(I can't remember how to get a dump of the zone, but I remember doing it in the past.
Simply stopping the nameserver ought to cause it to reconcile its records on file. That's what I do when I've struck a DNS/DHCP foul-up. Stop DHCP server, Stop BIND, edit records, increment the serial number, restart BIND, restart DHCP server.
That sort of thing tends to happen when you're experimenting, and haven't invented a new IP outside of the DHCP pool.
e.g. If your LAN uses 192.168.0.0, then set aside a range to be dynamic, such as 192.168.0.1 to 192.168.0.100, and configure your DHCP server to only dole out those addresses to dynamic clients. For static addresses, whether individually set on the client equipment, or handed out as fixed DHCP addresses, use IPs outside of that range.
Other things that confuse DHCP/DNS server combinations are dual-boot PCs, where on one OS the DHCP client sends out prefix codes that the other does not, and the DHCP server looks at those codes as well as MAC addresses, and doesn't give the same PC the same address for each OS. Which is logical enough, except that it doesn't properly reset the one it'd previously doled out to the same PC. It gets truly messy if the PC uses the same hostname on both OSs. I've seen reverse IP records stay stale, and forward ones updated.
-- [tim@localhost ~]$ uname -rsvp Linux 3.9.10-100.fc17.x86_64 #1 SMP Sun Jul 14 01:31:27 UTC 2013 x86_64
Boilerplate: All mail to my mailbox is automatically deleted, there is no point trying to privately email me, I only get to see the messages posted to the mailing list.
I'd just like to say that vinyl record crackles and pops are far less annoying than digigigigital mu-u-u-u-usic hiccicicicups and yooo-----------------u tu-----be ....... pauses.
Thank you for the replies
I am working on a small network and hence I can reset the dhcp/ddns to "Zero" - I have a script that does this.
I added my experiences to https://bugzilla.redhat.com/show_bug.cgi?id=1269093 and received a very prompt reply from Dan Williams
I have tested out the use of the two commands nmcli con mod eno1 connection.autoconnect no nmcli con mod eno1 connection.autoconnect yes and by magic the client no longer issues DHCPDISCOVER every time NetworkManager is restarted and hence does not request and obtain a new IP address.
I also checked what happened when the leases ran out naturally by setting "default-lease-time 300;" in dhcpd.conf on the server
Again all was well
I have not yet worked out how often the magic incantation is required - once after a new installation or once before the lease runs out or ...
Aside: I suspect that originally the client may not have been deleting the old IP address from the interface but was adding the new IP as a "secondary". At one stage during testing "ip a s" gave multiple entries of the form inet 148.197.29.133/24 brd 148.197.29.255 scope global dynamic eno1
This might explain my original symptoms - after several days the client and server would both freeze with the network busily flashing away. The only way out was to press the button on all machines. I originally thought it was a hardware problem but now suspect that NetworkManager may have been the cause of the crashes. At least my machines, router, switches, cables, ... have all been cleaned and dusted!
A nasty little problem as I have been happily running my arrangement for 10 years or more!
On 1/25/2016 10:35 PM, Tim wrote:
Allegedly, on or about 25 January 2016, Greg Woods sent:
(I can't remember how to get a dump of the zone, but I remember doing it in the past.
Simply stopping the nameserver ought to cause it to reconcile its records on file. That's what I do when I've struck a DNS/DHCP foul-up. Stop DHCP server, Stop BIND, edit records, increment the serial number, restart BIND, restart DHCP server.
Try using: rndc freeze example.com edit the zone rndc thaw example.com
If you are using DNS views: rndc freeze example.com in <view name> edit the zone rndc thaw example.com in <view name>
Don't forget to increment the serial no.
You don't have to stop DNS or DHCP.
Bill
On Thu, 2016-01-28 at 15:18 -0500, Bill Shirley wrote:
On 1/25/2016 10:35 PM, Tim wrote:
Allegedly, on or about 25 January 2016, Greg Woods sent:
(I can't remember how to get a dump of the zone, but I remember doing it in the past.
Simply stopping the nameserver ought to cause it to reconcile its records on file. That's what I do when I've struck a DNS/DHCP foul-up. Stop DHCP server, Stop BIND, edit records, increment the serial number, restart BIND, restart DHCP server.
Try using: rndc freeze example.com edit the zone rndc thaw example.com
If you are using DNS views: rndc freeze example.com in <view name> edit the zone rndc thaw example.com in <view name>
Don't forget to increment the serial no.
You don't have to stop DNS or DHCP.
Bill --
Thanks for the feedback
I've hacked together a script to do as you suggest whilst I'm experimenting to try and find out how/why my server became/becomes corrupted.
I now think it is to do with the way I cloned these particular client machines.
I used a variation of Create a valid F23 master machine/SSD and dump it whilst unmounted ... dump 0f /mnt/zip/F22_dump_2015_06_22 /dev/sdb5 ... Restore to the disk/SSD of another machine restore -rf /mnt/zip/F22_dump_2015_06_22 (This took less than 120s) ... mount /dev/sda6 /mnt/zip mount -o bind /dev /mnt/zip/dev mount -o bind /proc /mnt/zip/proc mount -o bind /sys /mnt/zip/sys chroot /mnt/zip gedit /etc/fstab gedit /etc/hostname Sort out grub2
I used to gedit /etc/sysconfig/network-scripts/ifcfg-eno1 but recently I have been lazy and let NM "sort it out" - for F23 I think it/I didn't get it quite right!
Adding an extra "mount -o bind /run /mnt/zip/run" to the list above lets me use nmcli in the chroot environment but it is not quite clear to me exactly what to do at that stage!
John
-
Bill Shirley -
Dr J Austin -
Greg Woods -
Tim