Sync fails to render DNS files following a replicate

[Greg Chavez]

Just before I saw the announcement for cobbler 2.4.0-1, I upgraded my
beta RHEL5 installation to cobbler-2.4.0-beta6.el5. I synced my master
and everything looked good. Next, I upgrade my slave and ran a
replicate:

cobbler replicate --master=$master --distros="*" --profiles="*"
--systems="*" --repos="*" --prune

The replication portion went well, but the ensuing sync failed at this
point (from cobbler.log):

Thu Jun 20 18:37:51 2013 - INFO | rendering DNS files

The exception:

rendering DNS files
Exception occured: exceptions.KeyError
Exception value: 'ipv6_address'
Exception Info:
File "/usr/lib/python2.4/site-packages/cobbler/remote.py", line 89, in run
rc = self._run(self)
File "/usr/lib/python2.4/site-packages/cobbler/remote.py", line 188, in runner
return
self.remote.api.sync(self.options.get("verbose",False),logger=self.logger)
File "/usr/lib/python2.4/site-packages/cobbler/api.py", line 767, in sync
return sync.run()
File "/usr/lib/python2.4/site-packages/cobbler/action_sync.py", line 137, in
run
self.dns.write_dns_files()
File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
line 587, in write_dns_files
self.__write_named_conf()
File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
line 259, in write_named_conf
metadata = {'forward_zones': self.forward_zones().keys(),
File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
line 131, in __forward_zones
ipv6 = interface["ipv6_address"]

!!! TASK FAILED !!!

The error also occurs after doing a "cobbler sync", although the
cobbler.log messages are different:

There are no errors if I change manage_dns from 1 to 0 in
/etc/cobbler/settings. The only thing I haven't done is check all my
500+ system entries for a weird ipv6_address entry (we don't use it).

Maybe this get fixed with the production 2.4.0-1 release? Thanks.

--
\*..+.-
--Greg Chavez
+//..;};

P.S. Also submitted a bug with this info.

[James Cammarata]

Is the system you're replicating to a different version of cobbler?
 On Jun 20, 2013 8:13 PM, "Greg Chavez" <greg.chavez(a)gmail.com&gt; wrote:

>  Just before I saw the announcement for cobbler 2.4.0-1, I upgraded
> my
>  beta RHEL5 installation to cobbler-2.4.0-beta6.el5. I synced my master
>  and everything looked good. Next, I upgrade my slave and ran a
>  replicate:
>
>  cobbler replicate --master=$master --distros="*" --profiles="*"
>  --systems="*" --repos="*" --prune
>
>  The replication portion went well, but the ensuing sync failed at this
>  point (from cobbler.log):
>
>  Thu Jun 20 18:37:51 2013 - INFO | rendering DNS files
>
>  The exception:
>
>  rendering DNS files
>  Exception occured: exceptions.KeyError
>  Exception value: 'ipv6_address'
>  Exception Info:
>  File "/usr/lib/python2.4/site-packages/cobbler/remote.py", line 89, in run
>  rc = self._run(self)
>  File "/usr/lib/python2.4/site-packages/cobbler/remote.py", line 188, in
>  runner
>  return
>  self.remote.api.sync(self.options.get("verbose",False),logger=self.logger)
>  File "/usr/lib/python2.4/site-packages/cobbler/api.py", line 767, in sync
>  return sync.run()
>  File "/usr/lib/python2.4/site-packages/cobbler/action_sync.py", line 137,
>  in run
>  self.dns.write_dns_files()
>  File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
>  line 587, in write_dns_files
>  self.__write_named_conf()
>  File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
>  line 259, in write_named_conf
>  metadata = {'forward_zones': self.forward_zones().keys(),
>  File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
>  line 131, in __forward_zones
>  ipv6 = interface["ipv6_address"]
>
>  !!! TASK FAILED !!!
>
>  The error also occurs after doing a "cobbler sync", although the
>  cobbler.log messages are different:
>
>  There are no errors if I change manage_dns from 1 to 0 in
>  /etc/cobbler/settings. The only thing I haven't done is check all my
>  500+ system entries for a weird ipv6_address entry (we don't use it).
>
>  Maybe this get fixed with the production 2.4.0-1 release? Thanks.
>
>  --
>  \*..+.-
>  --Greg Chavez
>  +//..;};
>
>  P.S. Also submitted a bug with this info.
>  _______________________________________________
>  cobbler mailing list
>  cobbler(a)lists.fedorahosted.org
>  https://lists.fedorahosted.org/mailman/listinfo/cobbler
>

[Greg Chavez]

No sir.

MASTER: cobbler-2.4.0-beta6.el5
SLAVE: cobbler-2.4.0-beta6.el5

First thing I checked actually :)

On Thu, Jun 20, 2013 at 9:22 PM, James Cammarata <jimi(a)sngx.net&gt; wrote:
>  Is the system you're replicating to a different version of
> cobbler?
>
>  On Jun 20, 2013 8:13 PM, "Greg Chavez" <greg.chavez(a)gmail.com&gt; wrote:
> >
> > Just before I saw the announcement for cobbler 2.4.0-1, I upgraded my
> > beta RHEL5 installation to cobbler-2.4.0-beta6.el5. I synced my master
> > and everything looked good. Next, I upgrade my slave and ran a
> > replicate:
> >
> > cobbler replicate --master=$master --distros="*" --profiles="*"
> > --systems="*" --repos="*" --prune
> >
> > The replication portion went well, but the ensuing sync failed at this
> > point (from cobbler.log):
> >
> > Thu Jun 20 18:37:51 2013 - INFO | rendering DNS files
> >
> > The exception:
> >
> > rendering DNS files
> > Exception occured: exceptions.KeyError
> > Exception value: 'ipv6_address'
> > Exception Info:
> > File "/usr/lib/python2.4/site-packages/cobbler/remote.py", line 89, in run
> > rc = self._run(self)
> > File "/usr/lib/python2.4/site-packages/cobbler/remote.py", line 188, in
> > runner
> > return
> > self.remote.api.sync(self.options.get("verbose",False),logger=self.logger)
> > File "/usr/lib/python2.4/site-packages/cobbler/api.py", line 767, in sync
> > return sync.run()
> > File "/usr/lib/python2.4/site-packages/cobbler/action_sync.py", line 137,
> > in run
> > self.dns.write_dns_files()
> > File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
> > line 587, in write_dns_files
> > self.__write_named_conf()
> > File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
> > line 259, in write_named_conf
> > metadata = {'forward_zones': self.forward_zones().keys(),
> > File "/usr/lib/python2.4/site-packages/cobbler/modules/manage_bind.py",
> > line 131, in __forward_zones
> > ipv6 = interface["ipv6_address"]
> >
> > !!! TASK FAILED !!!
> >
> > The error also occurs after doing a "cobbler sync", although the
> > cobbler.log messages are different:
> >
> > There are no errors if I change manage_dns from 1 to 0 in
> > /etc/cobbler/settings. The only thing I haven't done is check all my
> > 500+ system entries for a weird ipv6_address entry (we don't use it).
> >
> > Maybe this get fixed with the production 2.4.0-1 release? Thanks.
> >
> > --
> > \*..+.-
> > --Greg Chavez
> > +//..;};
> >
> > P.S. Also submitted a bug with this info.
> > _______________________________________________
> > cobbler mailing list
> > cobbler(a)lists.fedorahosted.org
> > https://lists.fedorahosted.org/mailman/listinfo/cobbler
>
>
>  _______________________________________________
>  cobbler mailing list
>  cobbler(a)lists.fedorahosted.org
>  https://lists.fedorahosted.org/mailman/listinfo/cobbler
>  


-- 
\*..+.-
--Greg Chavez
+//..;};

[James Cammarata]

On Thu, Jun 20, 2013 at 8:57 PM, Greg Chavez <greg.chavez(a)gmail.com&gt; wrote:

>  No sir.
>
>  MASTER: cobbler-2.4.0-beta6.el5
>  SLAVE: cobbler-2.4.0-beta6.el5
>
>  First thing I checked actually :)
>  
It would seem the .json for one (or more) of your systems is missing the
ipv6_address field. You might want to scan through them for the problem,
and or just write a script that just does "cobbler system edit
--name=<system>" for all of your systems (on both sides of the replication)
to force it to rewrite the json.

Definitely a bug though, missing that field shouldn't cause this problem.

[Greg Chavez]

So I here's what I did.

1) I ran this on the master

root@io-ns-03:/var/lib/cobbler/config/systems.d> grep -hc ipv6_address
* | sort | uniq -c
    146 0
    417 1

146 systems missing any reference, key or value, to ipv6_address.  You
were right.

2) I ran the "cobbler system edit" that you suggested, but it did not
result in these keys getting initialized.  Same error.

3) Next, I ran this on the master:

for X in `cobbler system list` ; do
    ifc=$( cb-search.pl -l --no-prefix $X | awk '{print $1}' )  # just
some stupid script I wrote
    cobbler system edit --interface=$ifc --ipv6-address=""
done

Then I replicated to the slave, but the sync failed again, only this
time the troublesome key was "ipv6_secondaries".

4) I ran the same loop as above to init all the ipv6_secondaries keys.
 When the sync ran on the slave, it was complaining about a missing
ipv6_address key again.

MY next move is to re-run that loop with a sub-loop that cycles
through each system's interfaces and on each one, init ipv6_address
and ipv6_secondaries.

I also updated my bug report: https://github.com/cobbler/cobbler/issues/509

Thanks
--Greg

On Thu, Jun 20, 2013 at 11:26 PM, James Cammarata <jimi(a)sngx.net&gt; wrote:
>  On Thu, Jun 20, 2013 at 8:57 PM, Greg Chavez
> <greg.chavez(a)gmail.com&gt; wrote:
> >
> > No sir.
> >
> > MASTER: cobbler-2.4.0-beta6.el5
> > SLAVE: cobbler-2.4.0-beta6.el5
> >
> > First thing I checked actually :)
>
>
>  It would seem the .json for one (or more) of your systems is missing the
>  ipv6_address field. You might want to scan through them for the problem, and
>  or just write a script that just does "cobbler system edit
> --name=<system>"
>  for all of your systems (on both sides of the replication) to force it to
>  rewrite the json.
>
>  Definitely a bug though, missing that field shouldn't cause this problem.
>
>  _______________________________________________
>  cobbler mailing list
>  cobbler(a)lists.fedorahosted.org
>  https://lists.fedorahosted.org/mailman/listinfo/cobbler