Change in vdsm[master]: vdsm-store-net-config: make use of oVirt node bindings

Antoni Segura Puimedon has uploaded a new change for review.

Change subject: vdsm-store-net-config: make use of oVirt node bindings
......................................................................

vdsm-store-net-config: make use of oVirt node bindings

Up until now, vdsm-store-net-config was a bash script that relied on
oVirt node's bash bindings for their python scripts. This patch
avoids the extra step to have a more fine grained access to the
new style ovirt.node.utils.fs package.

This patch depends on the oVirt node patches that make oVirt node
support symlink persistence.

Change-Id: Iee19cb9f6e1ba007631e7e6d315726d6efa03d09
Signed-off-by: Antoni S. Puimedon <asegurap(a)redhat.com&gt;
---
M .gitignore
M vdsm/Makefile.am
A vdsm/vdsm-store-net-config
D vdsm/vdsm-store-net-config.in
4 files changed, 170 insertions(+), 85 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/29355/1

diff --git a/.gitignore b/.gitignore
index 5e8b53f..24ddb94 100644
--- a/.gitignore
+++ b/.gitignore
@@ -65,7 +65,6 @@
 vdsm/vdsm-gencerts.sh
 vdsm/vdsm-logrotate.conf
 vdsm/vdsm-sosplugin.py
-vdsm/vdsm-store-net-config
 vdsm/vdsm.rwtab
 vdsm/vdsmd.8
 vdsm/gluster/hostname.py
diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am
index 763ef2a..66b7a69 100644
--- a/vdsm/Makefile.am
+++ b/vdsm/Makefile.am
@@ -60,7 +60,6 @@
 
 nodist_vdsm_SCRIPTS = \
 	mk_sysprep_floppy \
-	vdsm-store-net-config
 
 nodist_noinst_DATA = \
 	logger.conf \
@@ -80,6 +79,7 @@
 	supervdsmServer \
 	vdsm \
 	vdsm-restore-net-config \
+	vdsm-store-net-config \
 	$(NULL)
 
 nodist_man8_MANS = \
@@ -118,7 +118,6 @@
 	vdsm-logrotate \
 	vdsm-logrotate.conf.in \
 	vdsm-modules-load.d.conf \
-	vdsm-store-net-config.in \
 	vdsm-sysctl.conf \
 	vdsm.rwtab.in \
 	vdsmd.8.in \
diff --git a/vdsm/vdsm-store-net-config b/vdsm/vdsm-store-net-config
new file mode 100644
index 0000000..a54da13
--- /dev/null
+++ b/vdsm/vdsm-store-net-config
@@ -0,0 +1,169 @@
+#!/usr/bin/env python2
+#
+# Copyright 2009-2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+"""
+Stores all the vdsm configured networks persistently
+"""
+import glob
+import logging
+import logging.config
+import os
+import time
+import shutil
+
+from vdsm import config, constants, utils
+
+# ifcfg persistence directories
+NET_CONF_DIR = '/etc/sysconfig/network-scripts/'
+NET_CONF_BACK_DIR = constants.P_VDSM_LIB + '/netconfback'
+
+# Unified persistence directories
+RUN_CONF_DIR = constants.P_VDSM_RUN + '/netconf'
+PERSISTENT_SYMLINK_PATH = constants.P_VDSM_LIB + 'persistence/netconf'
+
+
+def _ifcfg_non_node_persist():
+    """Removes the backed up configuration files, thus implicitly marking
those
+    under /etc/sysconfig/network-scripts as safe"""
+    logging.debug('Persistence: Marking /etc/sysconfig/network-scripts as safe'
+                  )
+    _rm_dir_contents(NET_CONF_BACK_DIR)
+
+
+def _ifcfg_node_persist():
+    """Removes the backed up configuration files and leverages oVirt
node's
+    facilities to make /etc/sysconfig/netwrok-scripts persistent"""
+    node_conf = fs.Config()
+    for path in os.listdir(NET_CONF_BACK_DIR):
+        filename = os.path.basename(path)
+        sysconf_ifcfg_path = NET_CONF_DIR + filename
+        if os.path.exists(sysconf_ifcfg_path):
+            # If we currently have a running configuration for the file, we
+            # should make sure that it is persisted by the node
+            logging.debug('Persistence: oVirt node persisting "%s"' %
+                          sysconf_ifcfg_path)
+            node_conf.persist(sysconf_ifcfg_path)
+        else:
+            # If there is no current file, we should make sure that there is
+            # no persistent version either
+            logging.debug('Persistence: oVirt node deleting "%s"' %
+                          sysconf_ifcfg_path)
+            node_conf.delete(sysconf_ifcfg_path)
+        # remove the backup in case there's restore nets before reboot (on
+        # reboot the file would be gone anyway)
+        utils.rmFile(path)
+
+
+def _unified_non_node_persist():
+    """Atomically set RUN_CONF_DIR configuration as the new persistent
vdsm
+    network configuration"""
+    timestamp = _nano_timestamp()
+    new_dir = PERSISTENT_SYMLINK_PATH + '.' + timestamp
+    new_symlink = PERSISTENT_SYMLINK_PATH + '.link.' + timestamp
+
+    logging.debug('Persistence: persisting "%s"' % new_dir)
+    shutil.copytree(RUN_CONF_DIR, new_dir, symlinks=True)
+    os.symlink(new_dir, new_symlink)
+    logging.debug('Persistence: atomically making "%s" point to the new
'
+                  'configuration directory "%s' % (PERSISTENT_SYMLINK_PATH,
+                                                   new_dir))
+    os.rename(new_symlink, PERSISTENT_SYMLINK_PATH)
+
+    for path in glob.iglob(PERSISTENT_SYMLINK_PATH + '.*'):
+        if path == new_dir:
+            continue
+        logging.debug('Persistence: removing old config "%s"' % path)
+        if os.path.islink(path) or os.path.isfile(path):
+            utils.rmFile(path)
+        else:  # Directory
+            utils.rmTree(path)
+
+
+def _unified_node_persist():
+    node_conf = fs.Config()
+    timestamp = _nano_timestamp()
+    new_dir = PERSISTENT_SYMLINK_PATH + '.' + timestamp
+    new_symlink = PERSISTENT_SYMLINK_PATH + '.link.' + timestamp
+
+    shutil.copytree(RUN_CONF_DIR, new_dir, symlinks=True)
+    logging.debug('Persistence: oVirt node persisting "%s"' % new_dir)
+    node_conf.persist(new_dir)
+
+    os.symlink(new_dir, new_symlink)
+    logging.debug('Persistence: atomically making "%s" point to the new
'
+                  'configuration directory "%s' % (PERSISTENT_SYMLINK_PATH,
+                                                   new_dir))
+    os.rename(new_symlink, PERSISTENT_SYMLINK_PATH)
+
+    logging.debug('Persistence: oVirt node persisting symlink "%s", which
now '
+                  'points to the new configuration directory "%s' %
+                  (PERSISTENT_SYMLINK_PATH, new_dir))
+    node_conf.persist(PERSISTENT_SYMLINK_PATH)
+
+    # Now that the oVirt node has properly persisted the new directory and
+    # the new version of the symlinks, let's cleanup the old network
+    # configurations
+    for path in glob.iglob(PERSISTENT_SYMLINK_PATH + '.*'):
+        if path == new_dir:
+            continue
+        logging.debug('Persistence: oVirt node shredding old config
"%s"' %
+                      path)
+        node_conf.delete(path)
+
+
+def _nano_timestamp():
+    return ('%.20f' % time.time()).replace('.', '')[:19]
+
+
+def _rm_dir_contents(dirpath):
+    """Empties a directory"""
+    for path in os.listdir(dirpath):
+        if os.path.islink(path) or os.path.isfile(path):
+            utils.rmFile(path)
+        else:  # Directory
+            utils.rmTree(path)
+
+
+_unified = config.config.get('vars', 'net_persistence') ==
'unified'
+if utils.isOvirtNode():
+    from ovirt.node.utils import fs
+    if _unified:
+        persist = _unified_node_persist
+    else:
+        persist = _ifcfg_node_persist
+else:
+    if _unified:
+        persist = _unified_non_node_persist
+    else:
+        persist = _ifcfg_non_node_persist
+
+if __name__ == '__main__':
+    try:
+        logging.config.fileConfig('/etc/vdsm/svdsm.logger.conf',
+                                  disable_existing_loggers=False)
+    except:
+        logging.basicConfig(filename='/dev/stdout', filemode='w+',
+                            level=logging.DEBUG)
+        logging.error('Could not init proper logging', exc_info=True)
+
+    try:
+        persist()
+    except Exception:
+        logging.error('Failed to store vdsm network configuration.',
+                      sys_exc=True)
diff --git a/vdsm/vdsm-store-net-config.in b/vdsm/vdsm-store-net-config.in
deleted file mode 100755
index 94e658b..0000000
--- a/vdsm/vdsm-store-net-config.in
+++ /dev/null
@@ -1,82 +0,0 @@
-#!/bin/bash
-#
-# vdsm-store-net-config: store network configuration files persistently
-#
-
-. @LIBEXECDIR(a)/ovirt_functions.sh
-
-# ifcfg persistence directories
-NET_CONF_DIR='/etc/sysconfig/network-scripts/'
-NET_CONF_BACK_DIR='@VDSMLIBDIR@/netconfback'
-
-# Unified persistence directories
-RUN_CONF_DIR='@VDSMRUNDIR@/netconf'
-PERS_CONF_PATH='@VDSMLIBDIR@/persistence'
-PERS_NET_CONF_PATH="$PERS_CONF_PATH/netconf"
-
-PERSISTENCE=$1
-
-ifcfg_node_persist() {
-    for f in "$NET_CONF_BACK_DIR"/*;
-    do
-        [ ! -f "$f" ] && continue
-        bf=`basename "$f"`
-        if [ -f "$NET_CONF_DIR/$bf" ];
-        then
-            ovirt_store_config "$NET_CONF_DIR/$bf"
-        else
-            ovirt_safe_delete_config "$NET_CONF_DIR/$bf"
-        fi
-        rm "$NET_CONF_BACK_DIR/$bf"
-    done
-}
-
-ifcfg_nonnode_persist() {
-    # Remove the backed up configuration files thus marking the ones under
-    # /etc/sysconfig as "safe".
-    rm -rf "$NET_CONF_BACK_DIR"/*
-}
-
-unified_node_persist() {
-    unified_nonnode_persist
-
-    # oVirt node ovirt_store_config puts the dir in persistent storage and
-    # bind mounts it in the original place. So that's all we really need to do.
-    ovirt_store_config "$PERS_CONF_PATH"
-}
-
-unified_nonnode_persist() {
-    # Atomic directory copy by using the atomicity of overwriting a link
-    # (rename syscall).
-    TIMESTAMP=$(date +%s%N)
-    PERS_CONF_SYMLINK=$PERS_NET_CONF_PATH
-    PERS_CONF_DIR_ROOTNAME="$PERS_CONF_SYMLINK."
-    PERS_CONF_NEW_DIR="$PERS_CONF_DIR_ROOTNAME$TIMESTAMP"
-    PERS_CONF_NEW_SYMLINK="$PERS_CONF_SYMLINK.link.$TIMESTAMP"
-
-    cp -r "$RUN_CONF_DIR" "$PERS_CONF_NEW_DIR"
-    ln -s "$PERS_CONF_NEW_DIR" "$PERS_CONF_NEW_SYMLINK"
-    mv -fT "$PERS_CONF_NEW_SYMLINK" "$PERS_CONF_SYMLINK"
-    find "$PERS_CONF_PATH" -type d -path "$PERS_CONF_DIR_ROOTNAME*" |
\
-        grep -v "$PERS_CONF_NEW_DIR" | xargs rm -fr
-}
-
-
-if isOvirtNode
-then
-    # for node, persist the changed configuration files
-
-    . /usr/libexec/ovirt-functions
-
-    if [ "$PERSISTENCE" == "unified" ]; then
-        unified_node_persist
-    else
-        ifcfg_node_persist
-    fi
-else
-    if [ "$PERSISTENCE" == "unified" ]; then
-        unified_nonnode_persist
-    else
-        ifcfg_nonnode_persist
-    fi
-fi


-- 
To view, visit http://gerrit.ovirt.org/29355
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iee19cb9f6e1ba007631e7e6d315726d6efa03d09
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Antoni Segura Puimedon <asegurap(a)redhat.com&gt;

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: vdsm-store-net-config: make use of oVirt node bindings
......................................................................


Patch Set 1:

Build Failed 

http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/9722/ : FAILURE

http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/10507/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/10664/ : FAILURE

http://jenkins.ovirt.org/job/vdsm_master_network_functional_tests_gerrit/... : There was
an infra issue, please contact infra(a)ovirt.org

-- 
To view, visit http://gerrit.ovirt.org/29355
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iee19cb9f6e1ba007631e7e6d315726d6efa03d09
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Antoni Segura Puimedon <asegurap(a)redhat.com&gt;
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com&gt;
Gerrit-Reviewer: Douglas Schilling Landgraf <dougsland(a)redhat.com&gt;
Gerrit-Reviewer: Ondřej Svoboda <osvoboda(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No

automation(a)ovirt.org has posted comments on this change.

Change subject: vdsm-store-net-config: make use of oVirt node bindings
......................................................................


Patch Set 1:

* Update tracker::IGNORE, no Bug-Url found

-- 
To view, visit https://gerrit.ovirt.org/29355
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iee19cb9f6e1ba007631e7e6d315726d6efa03d09
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Antoni Segura Puimedon <toni+ovirt(a)midokura.com&gt;
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com&gt;
Gerrit-Reviewer: Fabian Deutsch <fabiand(a)redhat.com&gt;
Gerrit-Reviewer: Ido Barkan <ibarkan(a)redhat.com&gt;
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Ondřej Svoboda <osvoboda(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-HasComments: No

[Dan Kenigsberg]

Dan Kenigsberg has abandoned this change.

Change subject: vdsm-store-net-config: make use of oVirt node bindings
......................................................................


Abandoned

too bad, but no one is owning this now.

-- 
To view, visit https://gerrit.ovirt.org/29355
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Iee19cb9f6e1ba007631e7e6d315726d6efa03d09
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Antoni Segura Puimedon <toni+ovirt(a)midokura.com&gt;
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com&gt;
Gerrit-Reviewer: Fabian Deutsch <fabiand(a)redhat.com&gt;
Gerrit-Reviewer: Ido Barkan <ibarkan(a)redhat.com&gt;
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Ondřej Svoboda <osvoboda(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org