vdsm-patches April 2015

vdsm-patches@lists.fedorahosted.org

51 participants
798 discussions

Change in vdsm[master]: To avoid vdsm upgrade bugs all modules configure should run
by ybronhei＠redhat.com 06 Jun '15

06 Jun '15

Yaniv Bronhaim has uploaded a new change for review. Change subject: To avoid vdsm upgrade bugs all modules configure should run ...................................................................... To avoid vdsm upgrade bugs all modules configure should run Currently we run only libvirt and multipath configure. The problem is that we add those call to the new version and on upgrade the %postun script of the updated version is ran. Adding the new multipath configure won't help between 3.5 to 3.6 because it does not exist in 3.5. We need to find another solution for specific modules configuration change between 3.5 to 3.6 but with this patch from 3.6 and above we won't need to worry about other modules - all configurators will run on upgrade. If between 3.5 to 3.6 multipath.conf must update we need to do it in vdsmd_init_script as deprecated code. This patch also removes the calls to '/usr/lib/systemd/systemd-vdsmd' '/etc/init.d/vdsmd' which related to 3.3 version and does not need to be supported anymore. Change-Id: If6ab88c2dc6321864d9d934355e618f70e95ca10 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm.spec.in 1 file changed, 2 insertions(+), 18 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/92/39492/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index ec0b377..1a29382 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -937,25 +937,9 @@ supervdsmd_start_required='yes' fi - if ! %{_bindir}/vdsm-tool is-configured --module multipath >/dev/null 2>&1; + if ! %{_bindir}/vdsm-tool is-configured >/dev/null 2>&1; then - %{_bindir}/vdsm-tool configure --module multipath --force \ - >/dev/null 2>&1; - fi - - if ! %{_bindir}/vdsm-tool is-configured --module libvirt >/dev/null 2>&1; - then - if ! %{_bindir}/vdsm-tool configure --module libvirt --force \ - >/dev/null 2>&1; then - # fallback to vdsmd reconfigure api - This change may be removed - # when vdsm won't support any downgrade\upgrade to versions that - # don't include vdsm-tool configure api (vdsm <= 3.3) - for f in '/usr/lib/systemd/systemd-vdsmd' '/etc/init.d/vdsmd'; do - if [ -f "${f}" ]; then - "${f}" reconfigure >/dev/null 2>&1 || : - fi - done - fi + %{_bindir}/vdsm-tool configure --force >/dev/null 2>&1; then fi if [ "${supervdsmd_start_required}" = 'yes' ]; then -- To view, visit https://gerrit.ovirt.org/39492 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If6ab88c2dc6321864d9d934355e618f70e95ca10 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

7 51

Change in vdsm[master]: lvm: Remove duplication in udev rules
by Nir Soffer 05 Jun '15

05 Jun '15

Nir Soffer has uploaded a new change for review. Change subject: lvm: Remove duplication in udev rules ...................................................................... lvm: Remove duplication in udev rules The horrible uuid pattern was repeated 5 times, making the source unreadable. Now we build the udev patterns once, and use the built patterns to format the final output. Change-Id: Id259071c8c2c8e059308377823081e6b307ccbe0 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/vdsm_lvm_rules.template.in 1 file changed, 21 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/09/39309/1 diff --git a/vdsm/storage/vdsm_lvm_rules.template.in b/vdsm/storage/vdsm_lvm_rules.template.in index d73c0c6..045d523 100644 --- a/vdsm/storage/vdsm_lvm_rules.template.in +++ b/vdsm/storage/vdsm_lvm_rules.template.in @@ -11,8 +11,21 @@ """ import sys -write = sys.stdout.write enable_chown_hack = "@CHOWN_HACK@" + +env = { + "uuid": "-".join(("[a-f0-9]" * 8, + "[a-f0-9]" * 4, + "[a-f0-9]" * 4, + "[a-f0-9]" * 4, + "[a-f0-9]" * 12)), + "temp": "[a-zA-Z0-9]" * 8, +} + + +def write(fmt): + sys.stdout.write(fmt % env) + write("""\ # @@ -30,7 +43,7 @@ ACTION!="add|change", GOTO="lvm_end" # Filter out vgs which do not look like a vdsm vg -ENV{DM_VG_NAME}!="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", GOTO="lvm_end" +ENV{DM_VG_NAME}!="%(uuid)s", GOTO="lvm_end" # Volumes used as vdsm images """) @@ -40,18 +53,20 @@ # WARNING: we cannot use OWNER, GROUP and MODE since using any of them will # change the selinux label to the default, causing vms to pause after extending # disks. https://bugzilla.redhat.com/1147910 -ENV{DM_LV_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", RUN+="@CHOWN_PATH@ @VDSMUSER@:@QEMUGROUP@ $env{DEVNAME}", GOTO="lvm_end" +ENV{DM_LV_NAME}=="%(uuid)s", RUN+="@CHOWN_PATH@ @VDSMUSER@:@QEMUGROUP@ $env{DEVNAME}", GOTO="lvm_end" """) else: write("""\ -ENV{DM_LV_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" +ENV{DM_LV_NAME}=="%(uuid)s", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" """) write("""\ # Temporary volumes - not accessed by libvirt/qemu -ENV{DM_LV_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]_MERGE", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -ENV{DM_LV_NAME}=="_remove_me_[a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9]_[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" +ENV{DM_LV_NAME}=="%(uuid)s_MERGE", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" + +# Deleted volumes +ENV{DM_LV_NAME}=="_remove_me_%(temp)s_%(uuid)s", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" # Special volumes used by vdsm ENV{DM_LV_NAME}=="metadata|inbox|outbox", MODE:="0600", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -- To view, visit https://gerrit.ovirt.org/39309 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id259071c8c2c8e059308377823081e6b307ccbe0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 9

Change in vdsm[master]: lvm: Cleanup lvm udev rules
by Nir Soffer 05 Jun '15

05 Jun '15

Nir Soffer has uploaded a new change for review. Change subject: lvm: Cleanup lvm udev rules ...................................................................... lvm: Cleanup lvm udev rules Make the vdsm lvm rules suck less: - Check the vg name once, instead repating the horrible uuid pattern on every rule - Group rules with same settings (e.g. sanlock rules) - Document each group of rules - Remove irelevant documentation, copied from dm-lvm.rules Change-Id: Ib185a7d9a5da54036b8b0163a83ff6376b511533 Releates-To: https://bugzilla.redhat.com/1149883 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/vdsm-lvm.rules.tpl.in 1 file changed, 17 insertions(+), 20 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/74/33874/1 diff --git a/vdsm/storage/vdsm-lvm.rules.tpl.in b/vdsm/storage/vdsm-lvm.rules.tpl.in index fb6c87a..bdb33f1 100644 --- a/vdsm/storage/vdsm-lvm.rules.tpl.in +++ b/vdsm/storage/vdsm-lvm.rules.tpl.in @@ -1,5 +1,5 @@ # -# Copyright 2010 Red Hat, Inc. and/or its affiliates. +# Copyright 2010-2014 Red Hat, Inc. and/or its affiliates. # # Licensed to you under the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or @@ -7,17 +7,8 @@ # LICENSE_GPL_v2 which accompany this distribution. # -# Udev rules for LVM. -# -# These rules create symlinks for LVM logical volumes in -# /dev/VG directory (VG is an actual VG name). Some udev -# environment variables are set (they can be used in later -# rules as well): -# DM_LV_NAME - logical volume name -# DM_VG_NAME - volume group name -# DM_LV_LAYER - logical volume layer (blank if not set) +# Vdsm rules for lvm logical volumes. {{if chcon_hack}} -# # The libvirt image label is required to allow qemu to access volumes. Libvirt # sets this label on volumes when starting a vm, but on EL7 and Fedora the # label is lost after refreshing a logical volume, and vm get paused. This rule @@ -28,15 +19,21 @@ # "add" event is processed on coldplug only, so we need "change", too. ACTION!="add|change", GOTO="lvm_end" -# Fix ownership for RHEV volumes -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@"{{if chcon_hack}}, RUN+="vdsm-chcon"{{endif}}, GOTO="lvm_end" -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]_MERGE", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="_remove_me_[a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9]_[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="metadata", MODE:="0600", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="ids", MODE:="0660", OWNER:="@VDSMUSER@", GROUP:="@SNLKGROUP@", GOTO="lvm_end" -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="inbox", MODE:="0600", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="outbox", MODE:="0600", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -ENV{DM_VG_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", ENV{DM_LV_NAME}=="leases", MODE:="0660", OWNER:="@VDSMUSER@", GROUP:="@SNLKGROUP@", GOTO="lvm_end" +# Filter out vgs which do not look like a vdsm vg +ENV{DM_VG_NAME}!="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", GOTO="lvm_end" + +# Volumes used as vdsm images +ENV{DM_LV_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@"{{if chcon_hack}}, RUN+="vdsm-chcon"{{endif}}, GOTO="lvm_end" + +# Temprory volumes - not accessed by libvirt/qemu +ENV{DM_LV_NAME}=="[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]_MERGE", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" +ENV{DM_LV_NAME}=="_remove_me_[a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9]_[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9]-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" + +# Special storage domain volumes used by vdsm +ENV{DM_LV_NAME}=="metadata|inbox|outbox", MODE:="0600", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" + +# Special storage domain volumes used by sanlock +ENV{DM_LV_NAME}=="ids|leases", MODE:="0660", OWNER:="@VDSMUSER@", GROUP:="@SNLKGROUP@", GOTO="lvm_end" # FIXME: make special lvs vdsm-only readable, MODE doesn't work on rhel6 -- To view, visit http://gerrit.ovirt.org/33874 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib185a7d9a5da54036b8b0163a83ff6376b511533 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 24

Change in vdsm[master]: stompReactor: fix naming of default destination
by Piotr Kliczewski 04 Jun '15

04 Jun '15

Piotr Kliczewski has uploaded a new change for review. Change subject: stompReactor: fix naming of default destination ...................................................................... stompReactor: fix naming of default destination Change-Id: I1fb12dc725930076c44ea9720605f6283df0ec46 Signed-off-by: pkliczew <piotr.kliczewski(a)gmail.com> --- M lib/yajsonrpc/stompReactor.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/45/40345/1 diff --git a/lib/yajsonrpc/stompReactor.py b/lib/yajsonrpc/stompReactor.py index 2d3e70e..76271a4 100644 --- a/lib/yajsonrpc/stompReactor.py +++ b/lib/yajsonrpc/stompReactor.py @@ -29,7 +29,7 @@ _STATE_MSG = "Waiting for message" -_DEFAULT_RESPONSE_DESTINATIOM = "jms.topic.vdsm_legacy_responses" +_DEFAULT_RESPONSE_DESTINATION = "jms.topic.vdsm_legacy_responses" _DEFAULT_REQUEST_DESTINATION = "jms.topic.vdsm_legacy_requests" @@ -273,7 +273,7 @@ def send(self, message): self.log.debug("Sending response") - destination = _DEFAULT_RESPONSE_DESTINATIOM + destination = _DEFAULT_RESPONSE_DESTINATION try: resp = json.loads(message) destination = self._req_dest[resp.get("id")] -- To view, visit https://gerrit.ovirt.org/40345 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1fb12dc725930076c44ea9720605f6283df0ec46 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>

6 34

Change in vdsm[master]: subscriptions
by Piotr Kliczewski 04 Jun '15

04 Jun '15

Piotr Kliczewski has uploaded a new change for review. Change subject: subscriptions ...................................................................... subscriptions Change-Id: I1493070f2ba66ca9d39a6661876c82c4727cad62 Signed-off-by: pkliczewski <piotr.kliczewski(a)gmail.com> --- M lib/stompClient.py M lib/yajsonrpc/__init__.py M lib/yajsonrpc/stomp.py M lib/yajsonrpc/stompReactor.py M tests/jsonRpcHelper.py M vdsm/rpc/BindingJsonRpc.py 6 files changed, 177 insertions(+), 80 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/51/38451/1 diff --git a/lib/stompClient.py b/lib/stompClient.py index 5a6d7f5..db6f3d0 100644 --- a/lib/stompClient.py +++ b/lib/stompClient.py @@ -10,6 +10,7 @@ import socket from threading import Thread, Lock import time +import uuid _reactor = None _reactorLock = Lock() @@ -61,7 +62,7 @@ stomp_client = reactor.createClient(sock) subscription = stomp_client.subscribe( _DEFAULT_RESPONSE_DESTINATION, - sub_id="__vdsm_fake_broker__", + sub_id=str(uuid.uuid4()), ) client = yjrpc.JsonRpcClient( ClientRpcTransportAdapter( diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index 0bf3a54..f12d19b 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -205,9 +205,10 @@ class _JsonRpcServeRequestContext(object): - def __init__(self, client): + def __init__(self, client, conn): self._requests = [] self.client = client + self.conn = conn self._counter = 0 self._requests = {} self._responses = [] @@ -441,7 +442,7 @@ try: params = req.params - server_address = ctx.client.get_local_address() + server_address = ctx.conn.get_local_address() self._bridge.register_server_address(server_address) if isinstance(req.params, list): res = method(*params) @@ -469,11 +470,11 @@ if obj is None: break - client, msg = obj - self._parseMessage(client, msg) + client, conn, msg = obj + self._parseMessage(client, conn, msg) - def _parseMessage(self, client, msg): - ctx = _JsonRpcServeRequestContext(client) + def _parseMessage(self, client, conn, msg): + ctx = _JsonRpcServeRequestContext(client, conn) try: rawRequests = json.loads(msg) diff --git a/lib/yajsonrpc/stomp.py b/lib/yajsonrpc/stomp.py index 0dd6617..8b0a296 100644 --- a/lib/yajsonrpc/stomp.py +++ b/lib/yajsonrpc/stomp.py @@ -21,8 +21,9 @@ from vdsm.utils import monotonic_time import re +LEGACY_SUB_ID_REQ = "/queue/_local/vdsm/requests" +LEGACY_SUB_ID_RES = "/queue/_local/vdsm/reponses" -FAKE_SUB_ID = "__vdsm_fake_broker__" _RE_ESCAPE_SEQUENCE = re.compile(r"\\(.)") @@ -296,8 +297,9 @@ class AsyncDispatcher(object): log = logging.getLogger("stomp.AsyncDispatcher") - def __init__(self, frame_handler, bufferSize=4096): + def __init__(self, connection, frame_handler, bufferSize=4096): self._frame_handler = frame_handler + self.connection = connection self._bufferSize = bufferSize self._parser = Parser() self._outbuf = None @@ -445,11 +447,6 @@ def _process_message(self, frame, dispatcher): sub_id = frame.headers.get(Headers.SUBSCRIPTION) - if sub_id is None: - # This is actually an error, but because of the fake broker we - # injects a fake sub id - sub_id = "__vdsm_fake_broker__" - sub = self._subscriptions.get(sub_id) if sub is None: self.log.warning( @@ -458,7 +455,7 @@ ) return - sub._handle_message(frame) + sub.handle_message(frame) def _process_receipt(self, frame, dispatcher): self.log.warning("Receipt frame received and ignored") @@ -529,7 +526,7 @@ self._message_handler = message_handler self._destination = destination - def _handle_message(self, frame): + def handle_message(self, frame): self._message_handler(self, frame) def set_message_handler(self, handler): diff --git a/lib/yajsonrpc/stompReactor.py b/lib/yajsonrpc/stompReactor.py index 9ea45c0..c94fdc7 100644 --- a/lib/yajsonrpc/stompReactor.py +++ b/lib/yajsonrpc/stompReactor.py @@ -15,9 +15,11 @@ import logging from collections import deque +from uuid import uuid4 +import json import stomp - +from vdsm import utils from betterAsyncore import Dispatcher, Reactor from vdsm.sslutils import SSLSocket from . import ( @@ -55,10 +57,12 @@ class StompAdapterImpl(object): log = logging.getLogger("Broker.StompAdapter") - def __init__(self, reactor, messageHandler): + def __init__(self, reactor, sub_map, req_dest): self._reactor = reactor self._outbox = deque() - self._messageHandler = messageHandler + self._sub_dests = sub_map + self._req_dest = req_dest + self._sub_ids = {} self._commands = { stomp.Command.CONNECT: self._cmd_connect, stomp.Command.SEND: self._cmd_send, @@ -106,13 +110,99 @@ self._reactor.wakeup() def _cmd_subscribe(self, dispatcher, frame): - self.log.debug("Subscribe command ignored") + self.log.debug("Subscribe command received") + destination = frame.headers.get("destination", None) + sub_id = frame.headers.get("id", None) + + if not destination or not sub_id: + self._send_error("Missing required header", + dispatcher.connection) + return + + ack = frame.headers.get("ack", stomp.AckMode.AUTO) + subscription = stomp.Subscription(dispatcher.connection, destination, + sub_id, ack, None) + + # TODO make sure to cleanup when connection lost + try: + subs = self._sub_dests[destination] + except KeyError: + self._sub_dests[destination] = [subscription] + else: + subs.append(subscription) + + self._sub_ids[sub_id] = subscription + + def _send_error(self, msg, connection): + res = stomp.Frame( + stomp.Command.ERROR, + None, + msg + ) + connection.send_raw(res) def _cmd_unsubscribe(self, dispatcher, frame): - self.log.debug("Unsubscribe command ignored") + self.log.debug("Unsubscribe command received") + sub_id = frame.headers.get("id", None) + + if not sub_id: + self._send_error("Missing required header", + dispatcher.connection) + return + + subscription = self.subs_ids[sub_id] + if not subscription: + # ignore + return + + del self.subs_ids[sub_id] + subs = self._sub_dests[subscription.destination] + if len(subs) == 1: + del self._sub_dests[subscription.destination] + else: + if subscription in subs: + subs.remove(subscription) def _cmd_send(self, dispatcher, frame): - self._messageHandler(self, frame.body) + destination = frame.headers.get(stomp.Headers.DESTINATION, None) + if _DEFAULT_REQUEST_DESTINATION == destination: + # default subscription + try: + reqs = json.loads(frame.body) + self._req_dest[reqs[0].get("id")] = frame.headers.get( + stomp.Headers.REPLY_TO + ) + except: + # let the jsonrpc server to reply + pass + + dispatcher.connection.handleMessage(frame.body) + return + elif stomp.LEGACY_SUB_ID_REQ == destination: + try: + reqs = json.loads(frame.body) + self._req_dest[reqs[0].get("id")] = stomp.LEGACY_SUB_ID_RES + except: + # let the jsonrpc server to reply + pass + dispatcher.connection.handleMessage(frame.body) + return + + subs = self._sub_dests[destination] + if len(subs) == 0: + self._send_error("Subscription not available", + dispatcher.connection) + return + + for subscription in subs: + headers = utils.picklecopy(frame.headers) + headers[stomp.Headers.SUBSCRIPTION] = subscription.id + res = stomp.Frame( + stomp.Command.MESSAGE, + headers, + frame.body + ) + subscription.client.send_raw(res) def handle_frame(self, dispatcher, frame): self.log.debug("Handling message %s", frame) @@ -124,13 +214,15 @@ class _StompConnection(object): - def __init__(self, aclient, sock, reactor): + log = logging.getLogger("yajsonrpc.StompServer") + + def __init__(self, server, aclient, sock, reactor): self._socket = sock self._reactor = reactor - self._messageHandler = None + self._server = server self._aclient = aclient - adisp = self._adisp = stomp.AsyncDispatcher(aclient) + adisp = self._adisp = stomp.AsyncDispatcher(self, aclient) self._dispatcher = Dispatcher(adisp, sock=sock, map=reactor._map) def send_raw(self, msg): @@ -146,38 +238,32 @@ def close(self): self._dispatcher.close() + def get_local_address(self): + return self._socket.getsockname()[0] + + def setMessageHandler(self, msgHandler): + self._messageHandler = msgHandler + self._dispatcher.handle_read_event() + + def handleMessage(self, data): + if self._messageHandler is not None: + self._messageHandler((self._server, self, data)) + class StompServer(object): log = logging.getLogger("yajsonrpc.StompServer") - def __init__(self, sock, reactor): + def __init__(self, reactor): self._reactor = reactor self._messageHandler = None - self._socket = sock + self._sub_map = {} + self._req_dest = {} - adapter = StompAdapterImpl(reactor, self._handleMessage) - self._stompConn = _StompConnection( - adapter, - sock, - reactor, - ) - - def setTimeout(self, timeout): - self._stompConn.setTimeout(timeout) - - def connect(self): - self._stompConn.connect() - - def _handleMessage(self, impl, data): - if self._messageHandler is not None: - self._messageHandler((self, data)) - - def setMessageHandler(self, msgHandler): - self._messageHandler = msgHandler - self.check_read() - - def check_read(self): - self._stompConn._dispatcher.handle_read_event() + def add_client(self, sock): + adapter = StompAdapterImpl(self._reactor, self._sub_map, + self._req_dest) + return _StompConnection(self, adapter, sock, + self._reactor) def send( self, @@ -185,21 +271,31 @@ destination=_DEFAULT_RESPONSE_DESTINATION, ): self.log.debug("Sending response") - res = stomp.Frame( - stomp.Command.MESSAGE, - { - stomp.Headers.DESTINATION: destination, - stomp.Headers.CONTENT_TYPE: "application/json", - }, - message - ) - self._stompConn.send_raw(res) + + resp = json.loads(message) + try: + id = resp.get("id") + destination = self._req_dest[id] + del self._req_dest[id] + except: + # we could have no reply-to + pass + + for connection in self._sub_map[destination]: + res = stomp.Frame( + stomp.Command.MESSAGE, + { + stomp.Headers.DESTINATION: destination, + stomp.Headers.CONTENT_TYPE: "application/json", + stomp.Headers.SUBSCRIPTION: connection.id + }, + message + ) + connection.client.send_raw(res) def close(self): - self._stompConn.close() - - def get_local_address(self): - return self._socket.getsockname()[0] + for connection in self._sub_map.values(): + connection.close() class StompClient(object): @@ -212,6 +308,7 @@ self._aclient = stomp.AsyncClient("vdsm") self._stompConn = _StompConnection( + self, self._aclient, sock, reactor @@ -226,7 +323,7 @@ def handle_message(self, sub, frame): if self._messageHandler is not None: - self._messageHandler((self, frame.body)) + self._messageHandler((self, self, frame.body)) def setMessageHandler(self, msgHandler): self._messageHandler = msgHandler @@ -241,7 +338,9 @@ *args, **kwargs ): - return self._aclient.subscribe(*args, **kwargs) + sub = self._aclient.subscribe(*args, **kwargs) + self._reactor.wakeup() + return sub def send( self, @@ -264,8 +363,8 @@ self._stompConn.close() -def StompListener(reactor, acceptHandler, connected_socket): - impl = StompListenerImpl(reactor, acceptHandler, connected_socket) +def StompListener(reactor, server, acceptHandler, connected_socket): + impl = StompListenerImpl(server, acceptHandler, connected_socket) return Dispatcher(impl, connected_socket, map=reactor._map) @@ -275,16 +374,16 @@ class StompListenerImpl(object): log = logging.getLogger("jsonrpc.StompListener") - def __init__(self, reactor, acceptHandler, connected_socket): - self._reactor = reactor + def __init__(self, server, acceptHandler, connected_socket): self._socket = connected_socket self._acceptHandler = acceptHandler + self._server = server def init(self, dispatcher): dispatcher.set_reuse_addr() - client = StompServer(self._socket, self._reactor) - self._acceptHandler(self, client) + conn = self._server.add_client(self._socket) + self._acceptHandler(conn) def writable(self, dispatcher): return False @@ -293,10 +392,12 @@ class StompReactor(object): def __init__(self): self._reactor = Reactor() + self._server = StompServer(self._reactor) def createListener(self, connected_socket, acceptHandler): listener = StompListener( self._reactor, + self._server, acceptHandler, connected_socket ) @@ -399,13 +500,9 @@ request_queue, response_queue ): - sub_id = None - if request_queue == stomp.FAKE_SUB_ID: - sub_id = stomp.FAKE_SUB_ID - return JsonRpcClient( ClientRpcTransportAdapter( - stomp_client.subscribe(response_queue, sub_id=sub_id), + stomp_client.subscribe(response_queue, sub_id=str(uuid4())), request_queue, stomp_client, ) diff --git a/tests/jsonRpcHelper.py b/tests/jsonRpcHelper.py index 2fb7cfb..ad9c00a 100644 --- a/tests/jsonRpcHelper.py +++ b/tests/jsonRpcHelper.py @@ -35,7 +35,8 @@ StompRpcClient, ) from yajsonrpc.stomp import ( - FAKE_SUB_ID, + LEGACY_SUB_ID_REQ, + LEGACY_SUB_ID_RES, ) from protocoldetector import MultiProtocolAcceptor from rpc.BindingJsonRpc import BindingJsonRpc @@ -122,8 +123,8 @@ def client(client_socket): return StompRpcClient( reactor.createClient(client_socket), - FAKE_SUB_ID, - FAKE_SUB_ID, + LEGACY_SUB_ID_REQ, + LEGACY_SUB_ID_RES, ) def clientFactory(): diff --git a/vdsm/rpc/BindingJsonRpc.py b/vdsm/rpc/BindingJsonRpc.py index eeea8ba..dc2d97e 100644 --- a/vdsm/rpc/BindingJsonRpc.py +++ b/vdsm/rpc/BindingJsonRpc.py @@ -41,7 +41,7 @@ def add_socket(self, reactor, client_socket): reactor.createListener(client_socket, self._onAccept) - def _onAccept(self, listener, client): + def _onAccept(self, client): if self._clients is not None: self._clients.append(client) client.setMessageHandler(self._server.queueRequest) -- To view, visit https://gerrit.ovirt.org/38451 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1493070f2ba66ca9d39a6661876c82c4727cad62 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>

9 114

Change in vdsm[master]: jsonrpc: remove not needed param
by Piotr Kliczewski 03 Jun '15

03 Jun '15

Piotr Kliczewski has uploaded a new change for review. Change subject: jsonrpc: remove not needed param ...................................................................... jsonrpc: remove not needed param Change-Id: I24a9e44d272bb4d12c0816c7d168397ffa2758d2 Signed-off-by: pkliczew <piotr.kliczewski(a)gmail.com> --- M lib/yajsonrpc/stompReactor.py M vdsm/rpc/BindingJsonRpc.py 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/40171/1 diff --git a/lib/yajsonrpc/stompReactor.py b/lib/yajsonrpc/stompReactor.py index 47a14b0..6479698 100644 --- a/lib/yajsonrpc/stompReactor.py +++ b/lib/yajsonrpc/stompReactor.py @@ -280,7 +280,7 @@ dispatcher.set_reuse_addr() client = StompServer(self._socket, self._reactor) - self._acceptHandler(self, client) + self._acceptHandler(client) def writable(self, dispatcher): return False diff --git a/vdsm/rpc/BindingJsonRpc.py b/vdsm/rpc/BindingJsonRpc.py index a5b5b4a..2c94391 100644 --- a/vdsm/rpc/BindingJsonRpc.py +++ b/vdsm/rpc/BindingJsonRpc.py @@ -36,7 +36,7 @@ def add_socket(self, reactor, client_socket): reactor.createListener(client_socket, self._onAccept) - def _onAccept(self, listener, client): + def _onAccept(self, client): client.setMessageHandler(self._server.queueRequest) def createStompReactor(self): -- To view, visit https://gerrit.ovirt.org/40171 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I24a9e44d272bb4d12c0816c7d168397ffa2758d2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>

6 35

Change in vdsm[master]: reactor: using single instance
by Piotr Kliczewski 03 Jun '15

03 Jun '15

Piotr Kliczewski has uploaded a new change for review. Change subject: reactor: using single instance ...................................................................... reactor: using single instance Change-Id: I8f36ec36b91fcc46e1ab46eb92f9739e9e19e448 Signed-off-by: pkliczew <piotr.kliczewski(a)gmail.com> --- M tests/jsonRpcHelper.py M tests/protocoldetectorTests.py M vdsm/clientIF.py M vdsm/protocoldetector.py 4 files changed, 29 insertions(+), 16 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/70/40170/1 diff --git a/tests/jsonRpcHelper.py b/tests/jsonRpcHelper.py index de0ee12..4abca18 100644 --- a/tests/jsonRpcHelper.py +++ b/tests/jsonRpcHelper.py @@ -27,6 +27,7 @@ from itertools import product from M2Crypto import SSL from rpc.BindingXMLRPC import BindingXMLRPC, XmlDetector +from yajsonrpc.betterAsyncore import Reactor from yajsonrpc.stompReactor import StompDetector, StompRpcClient, _FAKE_SUB_ID from protocoldetector import MultiProtocolAcceptor from rpc.BindingJsonRpc import BindingJsonRpc @@ -59,7 +60,8 @@ @contextmanager def constructAcceptor(log, ssl, jsonBridge): sslctx = DEAFAULT_SSL_CONTEXT if ssl else None - acceptor = MultiProtocolAcceptor("127.0.0.1", 0, sslctx) + reactor = Reactor() + acceptor = MultiProtocolAcceptor(reactor, "127.0.0.1", 0, sslctx) cif = FakeClientIf() xml_binding = BindingXMLRPC(cif, cif.log) @@ -72,8 +74,10 @@ stompDetector = StompDetector(json_binding) acceptor.add_detector(stompDetector) - thread = threading.Thread(target=acceptor.serve_forever, - name='Detector thread') + thread = threading.Thread( + target=reactor.process_requests, + name='Detector thread', + ) thread.setDaemon(True) thread.start() diff --git a/tests/protocoldetectorTests.py b/tests/protocoldetectorTests.py index 7d300dd..695bd76 100644 --- a/tests/protocoldetectorTests.py +++ b/tests/protocoldetectorTests.py @@ -25,6 +25,7 @@ import time from contextlib import contextmanager +from yajsonrpc.betterAsyncore import Reactor from vdsm import sslutils from protocoldetector import MultiProtocolAcceptor from sslhelper import KEY_FILE, CRT_FILE @@ -199,13 +200,18 @@ # Helpers def start_acceptor(self, use_ssl): + self.reactor = Reactor() self.acceptor = MultiProtocolAcceptor( - '127.0.0.1', 0, sslctx=self.SSLCTX if use_ssl else None) + self.reactor, + '127.0.0.1', + 0, + sslctx=self.SSLCTX if use_ssl else None + ) self.acceptor.TIMEOUT = 1 self.acceptor.add_detector(Echo()) self.acceptor.add_detector(Uppercase()) self.acceptor_address = self.acceptor._acceptor.socket.getsockname() - t = threading.Thread(target=self.acceptor.serve_forever) + t = threading.Thread(target=self.reactor.process_requests) t.deamon = True t.start() diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index 2719101..0a4c12f 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -55,6 +55,8 @@ except ImportError: _glusterEnabled = False +from yajsonrpc.betterAsyncore import Reactor + class clientIF(object): """ @@ -109,6 +111,7 @@ host = config.get('addresses', 'management_ip') port = config.getint('addresses', 'management_port') + self._createReactor() self._createAcceptor(host, port) self._prepareXMLRPCBinding() self._prepareJSONRPCBinding() @@ -166,10 +169,14 @@ cls._instance = clientIF(irs, log) return cls._instance + def _createReactor(self): + self._reactor = Reactor() + def _createAcceptor(self, host, port): sslctx = self._createSSLContext() - self._acceptor = MultiProtocolAcceptor(host, port, sslctx) + self._acceptor = MultiProtocolAcceptor(self._reactor, host, + port, sslctx) def _createSSLContext(self): sslctx = None @@ -236,6 +243,8 @@ for binding in self.bindings.values(): binding.stop() + self._reactor.stop() + self._enabled = False self.channelListener.stop() self._hostStats.stop() @@ -253,8 +262,8 @@ def start(self): for binding in self.bindings.values(): binding.start() - self.thread = threading.Thread(target=self._acceptor.serve_forever, - name='Detector thread') + self.thread = threading.Thread(target=self._reactor.process_requests, + name='Reactor thread') self.thread.setDaemon(True) self.thread.start() diff --git a/vdsm/protocoldetector.py b/vdsm/protocoldetector.py index 97796fd..68972c6 100644 --- a/vdsm/protocoldetector.py +++ b/vdsm/protocoldetector.py @@ -21,9 +21,7 @@ import logging import socket -from vdsm.utils import traceback import vdsm.infra.filecontrol as filecontrol -from yajsonrpc.betterAsyncore import Reactor from vdsm.utils import monotonic_time from vdsm.sslutils import SSLHandshakeDispatcher @@ -161,13 +159,14 @@ def __init__( self, + reactor, host, port, sslctx=None, ssl_hanshake_timeout=SSLHandshakeDispatcher.SSL_HANDSHAKE_TIMEOUT, ): self._sslctx = sslctx - self._reactor = Reactor() + self._reactor = reactor sock = _create_socket(host, port) self._host, self._port = sock.getsockname() self.log.info("Listening at %s:%d", self._host, self._port) @@ -195,11 +194,6 @@ ) return dispatcher - - @traceback(on=log.name) - def serve_forever(self): - self.log.debug("Running") - self._reactor.process_requests() def add_detector(self, detector): self.log.debug("Adding detector %s", detector) -- To view, visit https://gerrit.ovirt.org/40170 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8f36ec36b91fcc46e1ab46eb92f9739e9e19e448 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>

7 47

Change in vdsm[master]: broker_support
by smizrahi＠redhat.com 03 Jun '15

03 Jun '15

Saggi Mizrahi has uploaded a new change for review. Change subject: broker_support ...................................................................... broker_support Change-Id: I8e43658f1cebd637ea3abf4396d388afa041ae71 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- A lib/stompClient.py M lib/yajsonrpc/__init__.py M lib/yajsonrpc/stomp.py M lib/yajsonrpc/stompReactor.py 4 files changed, 340 insertions(+), 215 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/68/36368/1 diff --git a/lib/stompClient.py b/lib/stompClient.py new file mode 100644 index 0000000..d3cd0ab --- /dev/null +++ b/lib/stompClient.py @@ -0,0 +1,136 @@ +#!/usr/bin/python +import yajsonrpc as yjrpc +import yajsonrpc.stompReactor as sr +import socket +from threading import Thread, Lock +import time + +_reactor = None +_reactorLock = Lock() + + +def get_reactor(): + global _reactor + if _reactor is None: + with _reactorLock: + if _reactor is None: + _reactor = sr.StompReactor() + t = Thread(target=_reactor.process_requests) + t.setDaemon(True) + t.start() + + return _reactor + + +class EchoServer(object): + def echo(self, text): + return text + + def register_server_address(self, *args, **kwargs): + pass + + def unregister_server_address(self, *args, **kwargs): + pass + + +def dummy_server(address): + sock = socket.create_connection(address) + reactor = get_reactor() + stomp_client = reactor.createClient(sock) + server = yjrpc.JsonRpcServer(EchoServer()) + t = Thread(target=server.serve_requests) + t.setDaemon(True) + t.start() + sub = stomp_client.subscribe( + sr._DEFAULT_REQUEST_DESTINATION, + message_handler=ServerRpcContextAdapter.subscription_handler(server) + ) + server._sub_ = sub + return server + + +class ServerRpcContextAdapter(object): + @classmethod + def subscription_handler(cls, server): + def handler(sub, frame): + server.queueRequest( + ( + ServerRpcContextAdapter(sub.client, frame), + frame.body + ) + ) + + return handler + + def __init__(self, client, request_frame): + self._client = client + self._reply_to = request_frame.headers.get('reply-to', None) + + def get_local_address(self, *args, **kwargs): + return "" + + def send(self, data): + if self._reply_to is None: + return + + self._client.send( + self._reply_to, + data, + { + "content-type": "application/json", + } + ) + + +class ClientRpcTransportAdapter(object): + def __init__(self, sub, destination, client): + self._sub = sub + sub.set_message_handler(self._handle_message) + self._destination = destination + self._client = client + self._message_handler = lambda arg: None + + def setMessageHandler(self, handler): + self._message_handler = handler + + def send(self, data): + headers = { + "content-type": "application/json", + "reply-to": self._sub.destination, + } + self._client.send( + data, + self._destination, + headers, + ) + + def _handle_message(self, sub, frame): + self._message_handler((self, frame.body)) + + def close(self): + self._sub.unsubscribe() + + +def connect(address): + sock = socket.create_connection(address) + reactor = get_reactor() + stomp_client = reactor.createClient(sock) + subscription = stomp_client.subscribe(sr._DEFAULT_RESPONSE_DESTINATIOM) + client = yjrpc.JsonRpcClient( + ClientRpcTransportAdapter( + subscription, + sr._DEFAULT_REQUEST_DESTINATION, + stomp_client, + ) + ) + return client + + +BROKER_ADDRESS = ("127.0.0.1", 5445) + +server = dummy_server(BROKER_ADDRESS) + +time.sleep(2) + +client = connect(BROKER_ADDRESS) +client.callMethod("echo", ["123"], 1) diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index 8120c03..6e485bf 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -50,7 +50,7 @@ log = logging.getLogger("JsonRpcInvalidRequestError") def __init__(self, object_name, msg_content): - self.log.error("Invalid message found " + msg_content) + self.log.error("Invalid message found %s", msg_content) JsonRpcError.__init__(self, -32600, "The JSON sent is not a valid Request object " "with " + object_name) @@ -100,7 +100,7 @@ raise JsonRpcInvalidRequestError("missing method header", obj) reqId = obj.get("id") - if not isinstance(reqId, (str, unicode)): + if not isinstance(reqId, (str, unicode, int)): raise JsonRpcInvalidRequestError("missing request identifier", obj) @@ -151,19 +151,7 @@ @staticmethod def decode(msg): obj = json.loads(msg, 'utf-8') - - if "result" not in obj and "error" not in obj: - raise JsonRpcInvalidRequestError("missing result or error info", - obj) - - result = obj.get('result') - error = JsonRpcError(**obj.get('error')) - - reqId = obj.get('id') - if not isinstance(reqId, (str, unicode)): - raise JsonRpcInvalidRequestError("missing response identifier", - obj) - return JsonRpcResponse(result, error, reqId) + return JsonRpcResponse.fromRawObject(obj) @staticmethod def fromRawObject(obj): @@ -178,9 +166,6 @@ error = obj.get("error") reqId = obj.get("id") - if not isinstance(reqId, (str, unicode)): - raise JsonRpcInvalidRequestError("missing response identifier", - obj) return JsonRpcResponse(result, error, reqId) @@ -297,12 +282,6 @@ self._lock = Lock() self._eventcbs = [] - def setTimeout(self, timeout): - self._transport.setTimeout(timeout) - - def connect(self): - self._transport.connect() - def callMethod(self, methodName, params=[], rid=None): resp = self.call(JsonRpcRequest(methodName, params, rid))[0] if resp.error: @@ -354,6 +333,12 @@ resp = JsonRpcResponse.fromRawObject(resp) with self._lock: + if resp.id is None: + self.log.warning( + "Got an error from server without an ID (%s)", + resp.error, + ) + ctx = self._runningRequests.pop(resp.id) ctx.addResponse(resp) diff --git a/lib/yajsonrpc/stomp.py b/lib/yajsonrpc/stomp.py index 142b22a..250b0bd 100644 --- a/lib/yajsonrpc/stomp.py +++ b/lib/yajsonrpc/stomp.py @@ -16,13 +16,10 @@ import logging import os import socket -from threading import Timer, Event +from threading import Timer from uuid import uuid4 from collections import deque -import time -from betterAsyncore import Dispatcher -import asyncore import re _RE_ESCAPE_SEQUENCE = re.compile(r"\\(.)") @@ -53,6 +50,16 @@ CONNECTED = "CONNECTED" ERROR = "ERROR" RECEIPT = "RECEIPT" + + +class Headers: + CONTENT_LENGTH = "content-length" + CONTENT_TYPE = "content-type" + SUBSCRIPTION = "subscription" + DESTINATION = "destination" + ACCEPT_VERSION = "accept-version" + REPLY_TO = "reply-to" + HOST = "host" COMMANDS = tuple([command for command in dir(Command) @@ -120,8 +127,11 @@ def decodeValue(s): # Make sure to leave this check before decoding as ':' can appear in the # value after decoding using \c - if ":" in s: - raise ValueError("Contains illigal charachter `:`") + # Disabled due to bug in hornetq: + # https://issues.jboss.org/browse/HORNETQ-1454 + + # if ":" in s: + # raise ValueError("Contains illigal charachter `:`") try: s = _RE_ESCAPE_SEQUENCE.sub( @@ -278,141 +288,15 @@ return None -class Client(object): - def __init__(self, sock=None): - """ - Initialize the client. - - The socket parameter can be an already initialized socket. Should be - used to pass specialized socket objects like SSL sockets. - """ - if sock is None: - sock = self._sock = socket.socket() - else: - self._sock = sock - - self._map = {} - # Because we don't know how large the frames are - # we have to use non bolocking IO - sock.setblocking(False) - - # We have our own timeout for operations we - # pretend to be synchronous (like connect). - self._timeout = None - self._connected = Event() - self._subscriptions = {} - - self._aclient = None - self._adisp = None - - self._inbox = deque() - - @property - def outgoing(self): - return self._adisp.outgoing - - def _registerSubscription(self, sub): - self._subscriptions[sub.id] = sub - - def _unregisterSubscription(self, sub): - del self._subscriptions[sub.id] - - @property - def connected(self): - return self._connected.isSet() - - def handle_connect(self, aclient, frame): - self._connected.set() - - def handle_message(self, aclient, frame): - self._inbox.append(frame) - - def process(self, timeout=None): - if timeout is None: - timeout = self._timeout - - asyncore.loop(use_poll=True, timeout=timeout, map=self._map, count=1) - - def connect(self, address, hostname): - sock = self._sock - - self._aclient = AsyncClient(self, hostname) - adisp = self._adisp = AsyncDispatcher(self._aclient) - disp = self._disp = Dispatcher(adisp, sock, self._map) - sock.setblocking(True) - disp.connect(address) - sock.setblocking(False) - self.recv() # wait for CONNECTED msg - - if not self._connected.isSet(): - sock.close() - raise socket.error() - - def recv(self): - timeout = self._timeout - s = time.time() - duration = 0 - while timeout is None or duration < timeout: - try: - return self._inbox.popleft() - except IndexError: - td = timeout - duration if timeout is not None else None - self.process(td) - duration = time.time() - s - - return None - - def put_subscribe(self, destination, ack=None): - subid = self._aclient.subscribe(self._adisp, destination, ack) - sub = Subscription(self, subid, ack) - self._registerSubscription(sub) - return sub - - def put_send(self, destination, data="", headers=None): - self._aclient.send(self._adisp, destination, data, headers) - - def put(self, frame): - self._adisp.send_raw(frame) - - def send(self): - disp = self._disp - timeout = self._timeout - duration = 0 - s = time.time() - while ((timeout is None or duration < timeout) and - (disp.writable() or not self._connected.isSet())): - td = timeout - duration if timeout is not None else None - self.process(td) - duration = time.time() - s - - def gettimout(self): - return self._timeout - - def settimeout(self, value): - self._timeout = value - - class AsyncDispatcher(object): log = logging.getLogger("stomp.AsyncDispatcher") - def __init__(self, frameHandler, bufferSize=4096): - self._frameHandler = frameHandler + def __init__(self, frame_handler, bufferSize=4096): + self._frame_handler = frame_handler self._bufferSize = bufferSize self._parser = Parser() - self._outbox = deque() self._outbuf = None self._outgoingHeartBeat = 0 - - def _queueFrame(self, frame): - self._outbox.append(frame) - - @property - def outgoing(self): - n = len(self._outbox) - if self._outbuf != "": - n += 1 - - return n def setHeartBeat(self, outgoing, incoming=0): if incoming != 0: @@ -422,7 +306,8 @@ self._outgoingHeartBeat = outgoing def handle_connect(self, dispatcher): - self._frameHandler.handle_connect(self) + self._outbuf = None + self._frame_handler.handle_connect(self) def handle_read(self, dispatcher): pending = self._bufferSize @@ -444,11 +329,10 @@ if data is not None: parser.parse(data) - frameHandler = self._frameHandler + frame_handler = self._frame_handler while parser.pending > 0: frame = parser.popFrame() - if hasattr(frameHandler, "handle_frame"): - frameHandler.handle_frame(self, frame) + frame_handler.handle_frame(self, frame) def popFrame(self): return self._parser.popFrame() @@ -456,7 +340,7 @@ def handle_write(self, dispatcher): if self._outbuf is None: try: - frame = self._outbox.popleft() + frame = self._frame_handler.peek_message() except IndexError: return @@ -467,14 +351,16 @@ self._lastOutgoingTimeStamp = self._milis() if numSent == len(data): self._outbuf = None + # Throw away the frame that was sent to the server + self._frame_handler.pop_message() else: self._outbuf = data[numSent:] - def send_raw(self, frame): - self._queueFrame(frame) - def writable(self, dispatcher): - if len(self._outbox) > 0 or self._outbuf is not None: + if self._frame_handler.has_outgoing_messages: + return True + + if self._outbuf is not None: return True if (self._outgoingHeartBeat > 0 @@ -493,51 +379,79 @@ class AsyncClient(object): - log = logging.getLogger("yajsonrpc.protocols.stomp.AsyncClient") + log = logging.getLogger("yajsonrpc.stomp.AsyncClient") - def __init__(self, frameHandler, hostname): + def __init__(self, hostname): self._hostname = hostname - self._frameHandler = frameHandler self._connected = False + self._outbox = deque() self._error = None + self._subscriptions = {} self._commands = { Command.CONNECTED: self._process_connected, Command.MESSAGE: self._process_message, Command.RECEIPT: self._process_receipt, - Command.ERROR: self._process_error} + Command.ERROR: self._process_error, + } @property def connected(self): return self._connected + def queue_frame(self, frame): + self._outbox.append(frame) + + @property + def has_outgoing_messages(self): + return (self._outbox.count > 0) + + def peek_message(self): + return self._outbox[0] + + def pop_message(self): + return self._outbox.popleft() + def getLastError(self): return self._error - def handle_connect(self, dispatcher): + def handle_connect(self): hostname = self._hostname - frame = Frame( + # TODO : reset subscriptions + # We use appendleft to make sure this is the first frame we send in + # case of a reconnect + self._outbox.appendleft(Frame( Command.CONNECT, - {"accept-version": "1.2", - "host": hostname}) - - dispatcher.send_raw(frame) + { + Headers.ACCEPT_VERSION: "1.2", + Headers.HOST: hostname, + } + )) def handle_frame(self, dispatcher, frame): self._commands[frame.command](frame, dispatcher) def _process_connected(self, frame, dispatcher): self._connected = True - frameHandler = self._frameHandler - if hasattr(frameHandler, "handle_connect"): - frameHandler.handle_connect(self, frame) self.log.debug("Stomp connection established") def _process_message(self, frame, dispatcher): - frameHandler = self._frameHandler + sub_id = frame.headers.get(Headers.SUBSCRIPTION) + if sub_id is None: + self.log.warning( + "Got message without a subscription" + ) + return - if hasattr(frameHandler, "handle_message"): - frameHandler.handle_message(self, frame) + sub = self._subscriptions.get(sub_id) + if sub is None: + self.log.warning( + "Got message without an unknown subscription id '%s'", + sub_id + ) + return + + sub._handle_message(frame) def _process_receipt(self, frame, dispatcher): self.log.warning("Receipt frame received and ignored") @@ -545,42 +459,86 @@ def _process_error(self, frame, dispatcher): raise StompError(frame) - def send(self, dispatcher, destination, data="", headers=None): - frame = Frame( + def send(self, destination, data="", headers=None): + final_headers = {"destination": destination} + final_headers.update(headers) + self.queue_frame(Frame( Command.SEND, - {"destination": destination}, - data) + final_headers, + data + )) - dispatcher.send_raw(frame) - - def subscribe(self, dispatcher, destination, ack=None): + def subscribe( + self, + destination, + ack=None, + sub_id=None, + message_handler=None + ): if ack is None: ack = AckMode.AUTO - subscriptionID = str(uuid4()) + if message_handler is None: + message_handler = lambda sub, frame: None - frame = Frame( + if sub_id is None: + sub_id = str(uuid4()) + + self.queue_frame(Frame( Command.SUBSCRIBE, - {"destination": destination, - "ack": ack, - "id": subscriptionID}) + { + "destination": destination, + "ack": ack, + "id": sub_id + } + )) - dispatcher.send_raw(frame) + sub = Subscription( + self, + destination, + sub_id, + ack, + message_handler, + ) + self._subscriptions[sub_id] = sub - return subscriptionID + return sub class Subscription(object): - def __init__(self, client, subid, ack): + def __init__( + self, + client, + destination, + subid, + ack, + message_handler + ): self._ack = ack self._subid = subid self._client = client self._valid = True + self._message_handler = message_handler + self._destination = destination + + def _handle_message(self, frame): + self._message_handler(self, frame) + + def set_message_handler(self, handler): + self._message_handler = handler @property def id(self): return self._subid + @property + def destination(self): + return self._destination + + @property + def client(self): + return self._client + def unsubscribe(self): client = self._client subid = self._subid diff --git a/lib/yajsonrpc/stompReactor.py b/lib/yajsonrpc/stompReactor.py index 3aa85e8..68c21b6 100644 --- a/lib/yajsonrpc/stompReactor.py +++ b/lib/yajsonrpc/stompReactor.py @@ -17,6 +17,7 @@ import os import threading import logging +from collections import deque import stomp @@ -27,8 +28,10 @@ _STATE_MSG = "Waiting for message" -_DEFAULT_RESPONSE_DESTINATIOM = "/queue/_local/vdsm/reponses" -_DEFAULT_REQUEST_DESTINATION = "/queue/_local/vdsm/requests" +_DEFAULT_RESPONSE_DESTINATIOM = "jms.topic.vdsm_legacy_responses" +_DEFAULT_REQUEST_DESTINATION = "jms.topic.vdsm_legacy_requests" + +_FAKE_SUB_ID = "__vdsm_fake_broker__" def parseHeartBeatHeader(v): @@ -55,6 +58,7 @@ def __init__(self, reactor, messageHandler): self._reactor = reactor + self._outbox = deque() self._messageHandler = messageHandler self._commands = { stomp.Command.CONNECT: self._cmd_connect, @@ -62,13 +66,30 @@ stomp.Command.SUBSCRIBE: self._cmd_subscribe, stomp.Command.UNSUBSCRIBE: self._cmd_unsubscribe} + @property + def has_outgoing_messages(self): + return (self._outbox.count > 0) + + def peek_message(self): + return self._outbox[0] + + def pop_message(self): + return self._outbox.popleft() + + def queue_frame(self, frame): + self._outbox.append(frame) + def _cmd_connect(self, dispatcher, frame): self.log.info("Processing CONNECT request") version = frame.headers.get("accept-version", None) if version != "1.2": - res = stomp.Frame(stomp.Command.ERROR, None, "Version unsupported") + resp = stomp.Frame( + stomp.Command.ERROR, + None, + "Version unsupported" + ) else: - res = stomp.Frame(stomp.Command.CONNECTED, {"version": "1.2"}) + resp = stomp.Frame(stomp.Command.CONNECTED, {"version": "1.2"}) cx, cy = parseHeartBeatHeader( frame.headers.get("heart-beat", "0,0") ) @@ -79,10 +100,10 @@ # The server can send a heart-beat every cy ms and doesn't want # to receive any heart-beat from the client. - res.headers["heart-beat"] = "%d,0" % (cy,) + resp.headers["heart-beat"] = "%d,0" % (cy,) dispatcher.setHeartBeat(cy) - dispatcher.send_raw(res) + self.queue_frame(resp) self._reactor.wakeup() def _cmd_subscribe(self, dispatcher, frame): @@ -109,11 +130,12 @@ self._reactor = reactor self._messageHandler = None + self._aclient = aclient adisp = self._adisp = stomp.AsyncDispatcher(aclient) self._dispatcher = Dispatcher(adisp, sock=sock, map=reactor._map) def send_raw(self, msg): - self._adisp.send_raw(msg) + self._aclient.queue_frame(msg) self._reactor.wakeup() def setTimeout(self, timeout): @@ -161,10 +183,15 @@ def send(self, message): self.log.debug("Sending response") - res = stomp.Frame(stomp.Command.MESSAGE, - {"destination": _DEFAULT_RESPONSE_DESTINATIOM, - "content-type": "application/json"}, - message) + res = stomp.Frame( + stomp.Command.MESSAGE, + { + stomp.Headers.DESTINATION: _DEFAULT_RESPONSE_DESTINATIOM, + stomp.Headers.SUBSCRIPTION: _FAKE_SUB_ID, + stomp.Headers.CONTENT_TYPE: "application/json", + }, + message + ) self._stompConn.send_raw(res) def close(self): @@ -182,12 +209,13 @@ self._messageHandler = None self._socket = sock - self._aclient = stomp.AsyncClient(self, "vdsm") + self._aclient = stomp.AsyncClient("vdsm") self._stompConn = _StompConnection( self._aclient, sock, reactor ) + self._aclient.handle_connect() def setTimeout(self, timeout): self._stompConn.setTimeout(timeout) @@ -195,7 +223,7 @@ def connect(self): self._stompConn.connect() - def handle_message(self, impl, frame): + def handle_message(self, sub, frame): if self._messageHandler is not None: self._messageHandler((self, frame.body)) @@ -207,11 +235,29 @@ if isinstance(self._socket, SSLSocket) and self._socket.pending() > 0: self._stompConn._dispatcher.handle_read() - def send(self, message): + def subscribe( + self, + *args, + **kwargs + ): + return self._aclient.subscribe(*args, **kwargs) + + def send( + self, + message, + destination=None, + headers=None + ): + if destination is None: + destination = _DEFAULT_REQUEST_DESTINATION + self.log.debug("Sending response") - self._aclient.send(self._stompConn, _DEFAULT_REQUEST_DESTINATION, - message, - {"content-type": "application/json"}) + self._aclient.send( + destination, + message, + headers + ) + self._reactor.wakeup() def close(self): self._stompConn.close() -- To view, visit http://gerrit.ovirt.org/36368 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8e43658f1cebd637ea3abf4396d388afa041ae71 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

8 111

Change in vdsm[master]: lvm: Remove unneeded rules for deleted volumes
by Nir Soffer 03 Jun '15

03 Jun '15

Nir Soffer has uploaded a new change for review. Change subject: lvm: Remove unneeded rules for deleted volumes ...................................................................... lvm: Remove unneeded rules for deleted volumes When deleting volumes, we mark the volumes with lvm tag instead of renaming them (as done in file-based volumes). This patch removes the unneeded rule for deleted volumes. Change-Id: I8ebdae59e569293f6eb1cff69f5e1b3122159629 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/vdsm_lvm_rules.template.in 1 file changed, 0 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/10/39310/1 diff --git a/vdsm/storage/vdsm_lvm_rules.template.in b/vdsm/storage/vdsm_lvm_rules.template.in index 045d523..4e4910a 100644 --- a/vdsm/storage/vdsm_lvm_rules.template.in +++ b/vdsm/storage/vdsm_lvm_rules.template.in @@ -19,7 +19,6 @@ "[a-f0-9]" * 4, "[a-f0-9]" * 4, "[a-f0-9]" * 12)), - "temp": "[a-zA-Z0-9]" * 8, } @@ -64,9 +63,6 @@ # Temporary volumes - not accessed by libvirt/qemu ENV{DM_LV_NAME}=="%(uuid)s_MERGE", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" - -# Deleted volumes -ENV{DM_LV_NAME}=="_remove_me_%(temp)s_%(uuid)s", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" # Special volumes used by vdsm ENV{DM_LV_NAME}=="metadata|inbox|outbox", MODE:="0600", OWNER:="@VDSMUSER@", GROUP:="@QEMUGROUP@", GOTO="lvm_end" -- To view, visit https://gerrit.ovirt.org/39310 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8ebdae59e569293f6eb1cff69f5e1b3122159629 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 9

Change in vdsm[master]: utils: add logged CPopen command execution
by Federico Simoncelli 03 Jun '15

03 Jun '15

Federico Simoncelli has uploaded a new change for review. Change subject: utils: add logged CPopen command execution ...................................................................... utils: add logged CPopen command execution Now that we're moving to a more wide use of CPopen in the code we still need consistent logging of command line executions. Change-Id: I5e04bde10be24457fe12362456e42528f9fd7196 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M lib/vdsm/utils.py 1 file changed, 15 insertions(+), 7 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/31/38831/1 diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py index d467af6..0b64d4a 100644 --- a/lib/vdsm/utils.py +++ b/lib/vdsm/utils.py @@ -641,6 +641,18 @@ return ' '.join(parts) +def loggedCPopen(args, close_fds=False, cwd=None, env=None, deathSignal=0, + childUmask=None, printable=None, logger=logging.root, + level=logging.DEBUG): + if not printable: + printable = args + + logger.log(level, "%s (cwd %s)", _list2cmdline(printable), cwd) + + return CPopen(args, close_fds=close_fds, cwd=cwd, env=env, + deathSignal=deathSignal, childUmask=childUmask) + + def execCmd(command, sudo=False, cwd=None, data=None, raw=False, printable=None, env=None, sync=True, nice=None, ioclass=None, ioclassdata=None, setsid=False, execCmdLogger=logging.root, @@ -671,13 +683,9 @@ if not callable(getattr(command, '__getitem__', None)): command = tuple(command) - if not printable: - printable = command - - execCmdLogger.debug("%s (cwd %s)", _list2cmdline(printable), cwd) - - p = CPopen(command, close_fds=True, cwd=cwd, env=env, - deathSignal=deathSignal, childUmask=childUmask) + p = loggedCPopen(command, close_fds=True, cwd=cwd, env=env, + deathSignal=deathSignal, childUmask=childUmask, + printable=printable) if not sync: p = AsyncProc(p) if data is not None: -- To view, visit https://gerrit.ovirt.org/38831 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5e04bde10be24457fe12362456e42528f9fd7196 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

6 37

← Newer
1
...
20
21
22
23
24
25
26
...
80
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches April 2015