vdsm-patches November 2014

vdsm-patches@lists.fedorahosted.org

49 participants
639 discussions

Change in vdsm[master]: vdsm: extract arch-dependent details
by ykleinbe＠redhat.com 08 Oct '15

08 Oct '15

Yoav Kleinberger has uploaded a new change for review. Change subject: vdsm: extract arch-dependent details ...................................................................... vdsm: extract arch-dependent details we now use other packages... Change-Id: I94bfc8f9d93d8f7a73e69b9390329c342f2904e7 Signed-off-by: Yoav Kleinberger <ykleinbe(a)redhat.com> --- M vdsm.spec.in M vdsm/storage/protect/Makefile.am D vdsm/storage/protect/safelease.c M vdsm/storage/protect/spmprotect.sh 4 files changed, 43 insertions(+), 766 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/97/31297/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index ceb238f..2aa8bdb 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -136,10 +136,7 @@ Requires: mom >= 0.4.1 Requires: numactl -%ifarch x86_64 -Requires: python-dmidecode -Requires: dmidecode -%endif +Requires: vdsm-arch-dependencies %if 0%{?rhel} >= 7 || 0%{?fedora} >= 18 Requires: libvirt-daemon >= 1.0.2-1 @@ -249,6 +246,8 @@ Requires(post): policycoreutils-python Requires(preun): policycoreutils-python +BuildArch: noarch + %description The VDSM service is required by a Virtualization Manager to manage the Linux hosts. VDSM manages and monitors the host's storage, memory and @@ -257,7 +256,7 @@ %package cli Summary: VDSM command line interface -BuildArch: noarch + Requires: %{name}-python = %{version}-%{release} Requires: %{name}-xmlrpc = %{version}-%{release} @@ -267,7 +266,7 @@ %package xmlrpc Summary: VDSM xmlrpc API -BuildArch: noarch + Requires: %{name}-python = %{version}-%{release} @@ -278,7 +277,7 @@ %package jsonrpc Summary: VDSM API Server -BuildArch: noarch + Requires: %{name}-python = %{version}-%{release} Requires: %{name}-yajsonrpc = %{version}-%{release} @@ -291,7 +290,7 @@ %package yajsonrpc Summary: JSON RPC server and client implementation -BuildArch: noarch + Requires: python >= 2.6 @@ -300,7 +299,7 @@ %package python-zombiereaper Summary: Collects zombie processes automatically -BuildArch: noarch + Requires: python >= 2.6 @@ -309,7 +308,7 @@ %package bootstrap Summary: VDSM bootstrapping package -BuildArch: noarch + %description bootstrap VDSM bootstrapping package. Used for delivering the bootstrap code onto the @@ -317,7 +316,7 @@ %package reg Summary: VDSM registration package -BuildArch: noarch + Requires: %{name} = %{version}-%{release} Requires: m2crypto @@ -334,7 +333,7 @@ Requires: python-cpopen >= 1.2.3-5 Requires: m2crypto Requires: python-ioprocess >= 0.5-1 -BuildArch: noarch + %description python Shared libraries between the various VDSM packages. @@ -343,7 +342,7 @@ Summary: VDSM Debug Plugin Requires: %{name} Requires: %{name}-xmlrpc = %{version}-%{release} -BuildArch: noarch + %description debug-plugin Used by the trained monkeys at Red Hat to insert chaos and mayhem in to VDSM. @@ -353,14 +352,14 @@ Requires: %{name} = %{version}-%{release} Requires: dracut Requires: python-nose -BuildArch: noarch + %description tests A test suite for verifying the functionality of a running vdsm instance %package hook-checkimages Summary: Qcow2 disk image format check hook for VDSM -BuildArch: noarch + Requires: %{name} %description hook-checkimages @@ -369,7 +368,7 @@ %package hook-ethtool-options Summary: Allow setting custom ethtool options for vdsm controlled nics -BuildArch: noarch + Requires: %{name} = %{version}-%{release} %description hook-ethtool-options @@ -380,7 +379,7 @@ %package hook-vhostmd Summary: VDSM hook set for interaction with vhostmd Requires: vhostmd -BuildArch: noarch + %description hook-vhostmd VDSM hook to use vhostmd per VM according to Virtualization Manager requests. @@ -388,7 +387,7 @@ %package hook-faqemu Summary: Fake qemu process for VDSM quality assurance -BuildArch: noarch + Requires: %{name} %description hook-faqemu @@ -399,7 +398,7 @@ %package hook-directlun Summary: Direct LUN support for VDSM -BuildArch: noarch + %description hook-directlun VDSM hook enable user to add storage LUN for VDSM @@ -407,7 +406,7 @@ %package hook-macbind Summary: Bind a vNIC to a Bridge -BuildArch: noarch + Requires: %{name} >= 4.14 %description hook-macbind @@ -415,7 +414,7 @@ %package hook-macspoof Summary: Disables MAC spoofing filtering -BuildArch: noarch + %description hook-macspoof VDSM hooks which allow to disable mac spoof filtering @@ -424,7 +423,7 @@ %package hook-extnet Summary: Force a vNIC to connect to a specific libvirt network -BuildArch: noarch + Requires: %{name} = %{version}-%{release} %description hook-extnet @@ -433,7 +432,7 @@ %package hook-fakevmstats Summary: Generate random VM statistics -BuildArch: noarch + Requires: %{name} %description hook-fakevmstats @@ -441,7 +440,7 @@ %package hook-fileinject Summary: Allow uploading file to VMs disk -BuildArch: noarch + Requires: python-libguestfs %description hook-fileinject @@ -450,14 +449,14 @@ %package hook-floppy Summary: Allow adding floppy to VM -BuildArch: noarch + %description hook-floppy Allow adding floppy to VM %package hook-hostusb Summary: Allow attaching USB device from host -BuildArch: noarch + Requires: usbutils %description hook-hostusb @@ -466,7 +465,7 @@ %package hook-hugepages Summary: Huge pages enable user to handle VM with 2048KB page files. -BuildArch: noarch + %description hook-hugepages Hook is getting number of huge pages reserve them for the VM, @@ -474,7 +473,7 @@ %package hook-isolatedprivatevlan Summary: Isolated network environment for VMs -BuildArch: noarch + %description hook-isolatedprivatevlan limit VM traffic to a specific gateway by its mac address, @@ -484,7 +483,7 @@ %package hook-nestedvt Summary: Nested Virtualization support for VDSM -BuildArch: noarch + %description hook-nestedvt If the nested virtualization is enabled in your kvm module @@ -492,7 +491,7 @@ %package hook-numa Summary: NUMA support for VDSM -BuildArch: noarch + %description hook-numa Hooks is getting number/rage of NUMA nodes and NUMA mode, @@ -500,14 +499,14 @@ %package hook-openstacknet Summary: OpenStack Network vNICs support for VDSM -BuildArch: noarch + %description hook-openstacknet Hook for OpenStack Network vNICs. %package hook-pincpu Summary: Hook pin VM so specific CPUs -BuildArch: noarch + %description hook-pincpu pincpu is hook for VDSM. @@ -515,7 +514,7 @@ %package hook-promisc Summary: Network interface promiscuous mode support for VDSM -BuildArch: noarch + %description hook-promisc VDSM promiscuous mode let user define a VM interface that will capture @@ -523,7 +522,7 @@ %package hook-qemucmdline Summary: QEMU cmdline hook for VDSM -BuildArch: noarch + Requires: %{name} %description hook-qemucmdline @@ -533,14 +532,14 @@ %package hook-qos Summary: QoS network in/out traffic support for VDSM -BuildArch: noarch + %description hook-qos Hook adds QoS in/out traffic to VMs interfaces %package hook-scratchpad Summary: One time disk creation for VDSM -BuildArch: noarch + %description hook-scratchpad scratchpad hook for VDSM @@ -550,7 +549,7 @@ %package hook-smbios Summary: Adding custom smbios entries to libvirt domain via VDSM -BuildArch: noarch + %description hook-smbios Adding custom smbios entries to libvirt domain via VDSM @@ -561,7 +560,7 @@ %if 0%{?rhel} >= 7 || 0%{?fedora} >= 18 Requires: libvirt-daemon-driver-nodedev %endif -BuildArch: noarch + %description hook-sriov sr-iov hook enable to add virtual functions exposed by the device @@ -570,7 +569,7 @@ %package hook-spiceoptions Summary: To configure spice options for vm -BuildArch: noarch + %description hook-spiceoptions This vdsm hook can be used to configure some of @@ -578,7 +577,7 @@ %package hook-vmfex Summary: vmfex support for VDSM -BuildArch: noarch + Conflicts: hook-vmfex-dev %description hook-vmfex @@ -586,7 +585,7 @@ %package hook-vmfex-dev Summary: VM-FEX vNIC support for VDSM -BuildArch: noarch + Requires: %{name} = %{version}-%{release} Conflicts: hook-vmfex @@ -596,7 +595,7 @@ %package hook-vmdisk Summary: External disk support for VDSM -BuildArch: noarch + %description hook-vmdisk Hook adds additional disk image for a VM (raw or qcow2) @@ -604,7 +603,7 @@ %if 0%{?with_gluster} %package gluster Summary: Gluster Plugin for VDSM -BuildArch: noarch + Requires: %{name} = %{version}-%{release} Requires: glusterfs-server @@ -1051,7 +1050,6 @@ %{_datadir}/%{vdsm_name}/storage/volume.py* %{_datadir}/%{vdsm_name}/storage/imageRepository/__init__.py* %{_datadir}/%{vdsm_name}/storage/imageRepository/formatConverter.py* -%{_libexecdir}/%{vdsm_name}/safelease %{_libexecdir}/%{vdsm_name}/spmprotect.sh %{_libexecdir}/%{vdsm_name}/spmstop.sh %dir %{_libexecdir}/%{vdsm_name}/hooks diff --git a/vdsm/storage/protect/Makefile.am b/vdsm/storage/protect/Makefile.am index feb40a9..1eb4661 100644 --- a/vdsm/storage/protect/Makefile.am +++ b/vdsm/storage/protect/Makefile.am @@ -6,11 +6,6 @@ # LICENSE_GPL_v2 which accompany this distribution. # -vdsmexec_PROGRAMS = safelease - dist_vdsmexec_SCRIPTS = \ spmprotect.sh \ spmstop.sh - -safelease_SOURCES = \ - safelease.c diff --git a/vdsm/storage/protect/safelease.c b/vdsm/storage/protect/safelease.c deleted file mode 100644 index 3814231..0000000 --- a/vdsm/storage/protect/safelease.c +++ /dev/null @@ -1,716 +0,0 @@ -/* Locker */ -#define _GNU_SOURCE 1 -#include <stdarg.h> -#include <stdio.h> -#include <stdlib.h> -#include <unistd.h> -#include <string.h> -#include <sys/types.h> -#include <sys/time.h> -#include <sys/stat.h> -#include <fcntl.h> -#include <errno.h> -#include <time.h> -#include <signal.h> - -#define WARN(fmt, args...) warn(__FUNCTION__, fmt, ## args) -#define PANIC(fmt, args...) panic(__FUNCTION__, fmt, ## args) -#define DEBUG(fmt, args...) do { if (debug) warn(__FUNCTION__, fmt, ## args); } while (0) - -char *freetag = "------FREE------0000000000000000"; -enum { - idlen = 16, - stamplen = 16, - taglen = idlen + stamplen, -}; - -char *progname; -int debug; - -char *id; -char *path; -char *request; - -long lease_ms; -long op_max_ms; -char *iobuf; - -inline unsigned long long tv2msec(struct timeval *tv); -int renew(int fd, off_t offset, char *id, long long *ts); - -void -panic(const char const *fn, char *msg, ...) -{ - char buf[512]; - va_list va; - int n; - - va_start(va, msg); - n = vsprintf(buf, msg, va); - va_end(va); - buf[n] = 0; - - fprintf(stderr, "panic: [%d] %s: %s: (%m)\n", getpid(), fn, buf); - - exit(-1); -} - -void -warn(const char const *fn, char *msg, ...) -{ - struct timeval tv; - long long unsigned tscurr; - char buf[512]; - va_list va; - int n; - - va_start(va, msg); - n = vsprintf(buf, msg, va); - va_end(va); - buf[n] = 0; - - gettimeofday(&tv, 0); - tscurr = tv2msec(&tv); - - fprintf(stderr, "[%s:%d:%llu]: %s: %s\n", progname, getpid(), tscurr, fn, buf); -} - -void -usage(void) -{ - fprintf(stderr, "Usage: %s [ -h ] <op> [...]\n", progname); - fprintf(stderr, "Ops:\n" - "acquire [ -r <path> ] [ -b ] [ -o offset ] <path> <id> <lease_ms> <op_max_ms>\n" - "renew [ -r <path> ] [ -o offset ] [ -t laststamp ] <path> <id> <lease_ms> <op_max_ms>\n" - "release [ -f ] [ -o offset ] <path> <id>\n" - "query [ -o offset ] <path>\n" - "protect [ -r <path> -i <id>] [ -o offset ] <path> <lease_ms> <op_max_ms> <progname> [<param1> ...]\n" - ); - fprintf(stderr, "\nNotes:\n" - "-b - busy loop on lease until lease acquired\n" - "-f - force release even if lease id is not equal to id\n" - "-o - offset to lease in path (default is 0)\n" - "-t - timestamp of last successful renewal\n" - "Path is a path to a device or a file to use as a sync object.\n" - "Id is an arbitrary unique string\n" - "lease_ms is the maximum time in msec that the owner of the lease\n" - " may hold it without renewing it\n" - "op_max_ms is the maximum time in msec that a single IO operation may take (must be <= lease_ms).\n" - "if -r option is used, the path is a readable file/device.\n" - " The program then validates that its 'id' is written at the given offset.\n" - " If this is not the case, acquire and renew will fail immediately.\n" - ); - exit(1); -} - -inline unsigned long long -tv2msec(struct timeval *tv) -{ - return tv->tv_sec * 1000ull + tv->tv_usec/1000; -} - -int -withintimelimits(struct timeval *start, struct timeval *stop) -{ - unsigned long long delta; - if (op_max_ms <= 0) - return 1; - delta = tv2msec(stop) - tv2msec(start); - if (delta > op_max_ms) { - DEBUG("Error - time limit breached: op_max_ms - %ld, time passed - %lld", op_max_ms, delta); - errno = -ETIMEDOUT; - return 0; - } - return 1; -} - -int -sametag(const char *tag1, const char *tag2) -{ - return !memcmp(tag1, tag2, taglen); -} - -int -isfree(const char *tag) -{ - return sametag(tag, freetag); -} - -void -settag(char *tag, const char *src) -{ - memcpy(tag, src, taglen); -} - -void -buildtag(char *tag, const char *id, long long ts) -{ - snprintf(tag, taglen+1, "%-*s%0*llx", idlen, id, stamplen, ts); - DEBUG("'%s' ts %lld", tag, ts); -} - -int -sameid(const char *tag, const char *id) -{ - char _id[idlen+1]; - - snprintf(_id, idlen+1, "%-*s", idlen, id); - return !memcmp(tag, _id, idlen); -} - -void -querytag(const char *tag, char *id, long long *ts) -{ - char _stamp[stamplen+1] = ""; - - memcpy(id, tag, idlen); - id[idlen] = 0; - memcpy(_stamp, tag+idlen, stamplen); - *ts = strtoull(_stamp, 0, 16); -} - -int -readtag(int fd, off_t offset, char *tag, int limit) -{ - struct timeval start, stop; - int r; - - DEBUG("fd %d offset %ld", fd, offset); - gettimeofday(&start, 0); - r = pread(fd, iobuf, 512, offset); - gettimeofday(&stop, 0); - DEBUG("r %d %m", r); - if (r <= 0 || (limit && !withintimelimits(&start, &stop))) - return -1; - memcpy(tag, iobuf, taglen); - return r; -} - -int -writetag(int fd, off_t offset, const char *tag, int limit) -{ - struct timeval start, stop; - int r; - - DEBUG("Enter"); - memcpy(iobuf, tag, taglen); - gettimeofday(&start, 0); - r = pwrite(fd, iobuf, 512, offset) < taglen ? -1 : 0; - gettimeofday(&stop, 0); - DEBUG("Exit r=%ld", r); - if (r < 0 || (limit && !withintimelimits(&start, &stop))) - return -1; - return r; -} - -int -writetimestamp(int fd, off_t offset, const char *id, char *tag, long long *ts) -{ - struct timeval tv; - long long t; - int r; - - gettimeofday(&tv, 0); - t = tv.tv_sec * 1000000ll + tv.tv_usec; - buildtag(tag, id, t); - r = writetag(fd, offset, tag, 1); - if (r > 0) - *ts = t; - return r; -} - -/* - * Attempt to acquire the lease. - * Return 1 if succedded, 0 if not , and < 0 on errors. - */ -int -acquire(int fd, off_t offset, char *id, int busyloop, long long *ts) -{ - char curr[taglen+1] = "", last[taglen+1] = "", tag[taglen+1] = ""; - long backoff_usec = (lease_ms + 6 * op_max_ms) * 1000; - long contend_usec = (2 * op_max_ms) * 1000; - char dummyid[idlen+1]; - - if (readtag(fd, offset, curr, 1) < 0) - return -errno; - - settag(last, freetag); - - do { - DEBUG("restart: curr tag is '%s'", curr); - if (!sametag(curr, last) && !isfree(curr)) do { - DEBUG("backoff: curr tag is '%s'", curr); - settag(last, curr); - usleep(backoff_usec); - if (readtag(fd, offset, curr, 1) < 0) - return -errno; - } while (busyloop && !sametag(curr, last) && !isfree(curr)); - if (!sametag(curr, last) && !isfree(curr)) { - DEBUG("fail: curr tag is '%s'", curr); - return 0; - } - DEBUG("contend: curr tag is '%s'", curr); - if (writetimestamp(fd, offset, id, tag, ts) < 0) { - DEBUG("lost (writetimestamp failed) : curr tag is %s", curr); - return -errno; - } - usleep(contend_usec); - if (readtag(fd, offset, curr, 1) < 0) { - DEBUG("lost (readtag failed) : curr tag is %s", curr); - return -errno; - } - } while (busyloop && !sametag(curr, tag)); - - if (busyloop || sametag(curr, tag)) { - DEBUG("won : curr tag is %s", curr); - querytag(curr, dummyid, ts); - return renew(fd, offset, id, ts); - } - DEBUG("lost : curr tag is %s\n our tag is %s", curr, tag); - return 0; -} - -static void -handler(int sig) -{ - PANIC("IO op too long"); -} - -long long -timeleft_ms(long long tsprev) -{ - struct timeval tv; - long long tscurr; - - tsprev /= 1000; - gettimeofday(&tv, 0); - tscurr = tv2msec(&tv); - DEBUG("time elapsed: %lld/%lld", tscurr - tsprev, lease_ms); - return lease_ms - (tscurr - tsprev); -} - -/* - * Attempt to renew the lease. - * Return 1 if succeded, 0 if not , and < 0 on errors. - */ -int -renew(int fd, off_t offset, char *id, long long *ts) -{ - char curr[taglen+1] = "", tag[taglen+1] = ""; - char dummyid[idlen+1]; - struct sigaction sa; - long long msleft; - int rc = 0; - - sa.sa_flags = !SA_RESTART; - sigemptyset(&sa.sa_mask); - sa.sa_handler = handler; - if (sigaction(SIGALRM, &sa, NULL) == -1) - PANIC("sigaction: can't set alarm"); - - if (readtag(fd, offset, curr, 0) < 0) { - rc = -errno; - goto out; - } - - DEBUG("curr tag is '%s'", curr); - if (!sameid(curr, id)) { - *ts = 0; - goto out; - } - - querytag(curr, dummyid, ts); - msleft = timeleft_ms(*ts); - if (msleft <= 0) { - rc = -ETIMEDOUT; - goto out; - } - - alarm(msleft / 1000); - DEBUG("updating tag: msleft %lld", msleft); - if (writetimestamp(fd, offset, id, tag, ts) < 0) { - rc = -errno; - goto out; - } - - DEBUG("All good"); - /* disable the alarm because usleep might use the same signal */ - alarm(0); - return 1; - -out: - alarm(0); - return rc; -} - -/* - * Attempt to release the lease. - * Return 1 if succedded, 0 if not , and < 0 on errors. - */ -int -release(int fd, off_t offset, char *id, int force) -{ - char curr[taglen+1] = ""; - - if (!force) { - if (readtag(fd, offset, curr, 0) < 0) - return -errno; - - if (!sameid(curr, id)) - return 0; - } - - return writetag(fd, offset, freetag, 0) < 0 ? -1 : 1; -} - -/* - * Qeury the lease. - * Return 1 if succedded, 0 if not , and < 0 on errors. - */ -int -query(int fd, off_t offset) -{ - char curr[taglen+1] = ""; - char id[idlen+1] = ""; - long long ts; - time_t tsec; - int tusec; - char *t; - - if (readtag(fd, offset, curr, 0) < 0) - return -errno; - - querytag(curr, id, &ts); - tsec = ts / 1000000; - tusec = ts % 1000000; - - t = ctime(&tsec); - t[strlen(t)-1] = 0; - - printf("%s: ID %-*s TS %0*llx (%s, %d usec)\n", - sameid(curr, freetag) ? "FREE" : "LOCKED", - idlen, id, stamplen, ts, t, tusec); - - return 1; -} - -void -validate_path(const char *path) -{ - if (access(path, R_OK | W_OK) < 0) - PANIC("can't access '%s'", path); -} - -void -validate_id(const char *id) -{ - if (strlen(id) > idlen) - PANIC("id must be <= 8 characters"); - if (!strncmp(id, freetag, idlen)) - PANIC("can't lease free stamp"); -} - -void -validate_lease_params(int lease_ms, int op_max_ms) -{ - if (lease_ms <= 0 || op_max_ms <= 0 || lease_ms < op_max_ms || - op_max_ms < 1000 || op_max_ms % 1000 != 0) - PANIC("bad lease/op max timeouts"); -} - -/* - * Initialize the timeout to one op_max_ms. - */ -long long -renew_timeout(void) -{ - struct timeval tv; - - gettimeofday(&tv, 0); - return tv.tv_sec * 1000000ull + tv.tv_usec - (lease_ms - op_max_ms) * 1000; -} - -int -cmd_acquire(int argc, char **argv) -{ - int opt, fd, r, b = 0; - off_t offset = 0; - long long ts; - - optind = 0; - while ((opt = getopt(argc, argv, "+hdr:bo:")) != -1) { - switch (opt) { - case 'h': - usage(); - break; - case 'd': - debug++; - break; - case 'r': - request = optarg; - break; - case 'b': - b = 1; - break; - case 'o': - offset = strtoul(optarg, 0, 0); - break; - } - } - if (argc - optind < 4) - usage(); - - path = argv[optind++]; - validate_path(path); - id = argv[optind++]; - validate_id(id); - lease_ms = strtoul(argv[optind++], 0, 0); - op_max_ms = strtoul(argv[optind++], 0, 0); - validate_lease_params(lease_ms, op_max_ms); - - DEBUG("path '%s' offset %ld id '%s' lease_ms %ld op_max_ms %ld", - path, offset, id, lease_ms, op_max_ms); - - if ((fd = open(path, O_RDWR | O_DIRECT)) < 0) - panic("can't open '%s'", path); - - r = acquire(fd, offset, id, b, &ts); - - close(fd); - - if (r == 1) { - /* print last successful timestamp == aquire time */ - printf("%lld", ts); - DEBUG("Succeeded"); - return 0; - } else - DEBUG("%s (%s)", "Failed", strerror(r)); - - return 1; -} - -int -cmd_renew(int argc, char **argv) -{ - long long ts = renew_timeout(); - off_t offset = 0; - int opt, fd, r; - - optind = 0; - while ((opt = getopt(argc, argv, "+hdr:o:t:")) != -1) { - switch (opt) { - case 'h': - usage(); - break; - case 'd': - debug++; - break; - case 'r': - request = optarg; - break; - case 'o': - offset = strtoul(optarg, 0, 0); - break; - case 't': - ts = strtoll(optarg, 0, 0); - break; - } - } - if (argc - optind < 4) - usage(); - - path = argv[optind++]; - validate_path(path); - id = argv[optind++]; - validate_id(id); - lease_ms = strtoul(argv[optind++], 0, 0); - op_max_ms = strtoul(argv[optind++], 0, 0); - validate_lease_params(lease_ms, op_max_ms); - - DEBUG("path '%s' offset %ld id '%s' lease_ms %ld op_max_ms %ld", - path, offset, id, lease_ms, op_max_ms); - - if ((fd = open(path, O_RDWR | O_DIRECT)) < 0) - panic("can't open '%s'", path); - - r = renew(fd, offset, id, &ts); - - close(fd); - - /* print out the last successful renewal timestamp, or zero for don't renew */ - printf("%lld\n", ts); - - if (r == 1) { - DEBUG("Succeeded"); - return 0; - } - - DEBUG("%s (%s)", "Failed", strerror(r)); - return 1; -} - -int -cmd_release(int argc, char **argv) -{ - int opt, fd, r; - int force = 0; - off_t offset = 0; - - optind = 0; - while ((opt = getopt(argc, argv, "+hdfo:")) != -1) { - switch (opt) { - case 'h': - usage(); - break; - case 'd': - debug++; - break; - case 'f': - force++; - break; - case 'o': - offset = strtoul(optarg, 0, 0); - break; - } - } - if (argc - optind < 2) - usage(); - - path = argv[optind++]; - validate_path(path); - id = argv[optind++]; - validate_id(id); - - DEBUG("path '%s' offset %ld id '%s' force %d", path, offset, id, force); - - if ((fd = open(path, O_RDWR | O_DIRECT)) < 0) - panic("can't open '%s'", path); - - r = release(fd, offset, id, force); - - close(fd); - - if (r == 1) { - DEBUG("Succeeded"); - return 0; - } else - DEBUG("%s (%s)", "Failed", strerror(r)); - - return 1; -} - -int -cmd_query(int argc, char **argv) -{ - int opt, fd, r; - off_t offset = 0; - - optind = 0; - while ((opt = getopt(argc, argv, "+hdr:o:")) != -1) { - switch (opt) { - case 'h': - usage(); - break; - case 'd': - debug++; - break; - case 'r': - request = optarg; - break; - case 'o': - offset = strtoul(optarg, 0, 0); - break; - } - } - if (argc - optind < 4) - usage(); - - path = argv[optind++]; - validate_path(path); - - DEBUG("path '%s' offset %ld id '%s'", path, offset, id); - - if ((fd = open(path, O_RDWR | O_DIRECT)) < 0) - panic("can't open '%s'", path); - - r = query(fd, offset); - - close(fd); - - if (r == 1) { - DEBUG("Succeeded"); - return 0; - } else - DEBUG("%s (%s)", "Failed", strerror(r)); - - return 1; -} - -int -cmd_protect(int argc, char **argv) -{ - return 0; -} - -void -sig_handler(int sig) -{ - fprintf(stderr, "%s: Exiting due to signal %d\n", progname, sig); - exit(0); -} - -int -main(int argc, char **argv) -{ - int opt; - void *v = 0; - - signal(SIGTERM, sig_handler); - signal(SIGINT, sig_handler); - signal(SIGTRAP, sig_handler); - - if (posix_memalign(&v, 4096, 512) != 0) - { - fprintf(stderr, "fatal memory allocation error\n"); - return 1; - } - - iobuf = v; - memset(iobuf, 0, 512); - - progname = strrchr(argv[0], '/'); - if (!progname) - progname = argv[0]; - else - progname++; - - while ((opt = getopt(argc, argv, "+hd")) != -1) { - switch (opt) { - case 'h': - usage(); - break; - case 'd': - debug++; - break; - } - } - if (optind >= argc) - usage(); - - if (!strcmp(argv[optind], "acquire")) - return cmd_acquire(argc - optind, argv + optind); - if (!strcmp(argv[optind], "renew")) - return cmd_renew(argc - optind, argv + optind); - if (!strcmp(argv[optind], "release")) - return cmd_release(argc - optind, argv + optind); - if (!strcmp(argv[optind], "query")) - return cmd_query(argc - optind, argv + optind); - if (!strcmp(argv[optind], "protect")) - return cmd_protect(argc - optind, argv + optind); - - fprintf(stderr, "unknonwn op <%s>\n", argv[optind]); - usage(); - - return 1; -} - diff --git a/vdsm/storage/protect/spmprotect.sh b/vdsm/storage/protect/spmprotect.sh index f569a91..e864b34 100755 --- a/vdsm/storage/protect/spmprotect.sh +++ b/vdsm/storage/protect/spmprotect.sh @@ -24,7 +24,7 @@ SETSID="/usr/bin/setsid" LOGFILE="/var/log/vdsm/spm-lock.log" VDS_CLIENT="/usr/bin/vdsClient" -LEASE_UTIL="./safelease" +LEASE_UTIL="../safelease/safelease" KILL="/bin/kill" PKILL="/usr/bin/pkill" sdUUID=$2 -- To view, visit http://gerrit.ovirt.org/31297 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I94bfc8f9d93d8f7a73e69b9390329c342f2904e7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yoav Kleinberger <ykleinbe(a)redhat.com>

5 6

Change in vdsm[master]: vdsm: require kernel-2.6.32-465 on for NFSv4 fixes
by ykleinbe＠redhat.com 08 Oct '15

08 Oct '15

Yoav Kleinberger has uploaded a new change for review. Change subject: vdsm: require kernel-2.6.32-465 on for NFSv4 fixes ...................................................................... vdsm: require kernel-2.6.32-465 on for NFSv4 fixes This patch updates vdsm's dependencies to incorporate, on Fedora, fixes to NFSv4 introduces in https://bugzilla.redhat.com/show_bug.cgi?id=1075123. This patch fixes https://bugzilla.redhat.com/show_bug.cgi?id=1100285. Change-Id: Ib7fae41d22065061588ef72516070bf8b55bf526 Signed-off-by: Yoav Kleinberger <ykleinbe(a)redhat.com> --- M vdsm.spec.in 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/10/33810/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 0dcd82b..35cc892 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -200,7 +200,7 @@ Requires: device-mapper-multipath >= 0.4.9-52 Requires: e2fsprogs >= 1.41.12-11 Requires: fence-agents -Requires: kernel >= 2.6.32-279.9.1 +Requires: kernel >= 2.6.32-465.9.1 Requires: initscripts >= 9.03.31-2.el6_3.1 Requires: policycoreutils >= 2.0.83-19.30 Requires: policycoreutils-python >= 2.0.83-19.30 -- To view, visit http://gerrit.ovirt.org/33810 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib7fae41d22065061588ef72516070bf8b55bf526 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yoav Kleinberger <ykleinbe(a)redhat.com>

4 6

Change in vdsm[master]: tests: added NFS support to functional tests
by ykleinbe＠redhat.com 08 Oct '15

08 Oct '15

Yoav Kleinberger has uploaded a new change for review. Change subject: tests: added NFS support to functional tests ...................................................................... tests: added NFS support to functional tests In this patch, I introduce a new NFS storage context for the storage functional tests. Since there is much shared logic between NFS and LocalFS storage context, I introduce a common superclass. Change-Id: I1781fc400c0604855d3143dde22ccb29e6cc8013 Signed-off-by: Yoav Kleinberger <ykleinbe(a)redhat.com> --- M tests/functional/basicStorageTest.py M tests/functional/testlib/storagecontexts/base.py A tests/functional/testlib/storagecontexts/filebased.py M tests/functional/testlib/storagecontexts/iscsi.py M tests/functional/testlib/storagecontexts/localfs.py A tests/functional/testlib/storagecontexts/nfs.py 6 files changed, 249 insertions(+), 114 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/90/32990/1 diff --git a/tests/functional/basicStorageTest.py b/tests/functional/basicStorageTest.py index c01d14b..438f8c8 100644 --- a/tests/functional/basicStorageTest.py +++ b/tests/functional/basicStorageTest.py @@ -4,6 +4,7 @@ from functional.testlib import controlvdsm from functional.testlib.storagecontexts import localfs from functional.testlib.storagecontexts import iscsi +from functional.testlib.storagecontexts import nfs class TestBasicStorageDomain(object): @@ -23,6 +24,9 @@ def testCreateVolumeISCSI(self): self._testCreateVolume(iscsi.ISCSI) + def testCreateVolumeNFS(self): + self._testCreateVolume(nfs.NFS) + def _testCreateVolume(self, storageBackend): with storageBackend() as (vdsm, verify): storageServerID = vdsm.connectStorageServer() diff --git a/tests/functional/testlib/storagecontexts/base.py b/tests/functional/testlib/storagecontexts/base.py index 7179694..2e308ad 100644 --- a/tests/functional/testlib/storagecontexts/base.py +++ b/tests/functional/testlib/storagecontexts/base.py @@ -73,16 +73,47 @@ class StorageBackend(object): def __init__(self): self._vdsmCaller = vdsmcaller.VDSMCaller() + self._domainID = self._newUUID() + self._poolID = self._newUUID() + self._imageID = self._newUUID() + self._volumeID = self._newUUID() + self._connectionID = self._newUUID() + + def connectionID(self): + return self._connectionID + + def volumeID(self): + return self._volumeID + + def imageID(self): + return self._imageID + + def poolID(self): + return self._poolID + + def domainID(self): + return self._domainID def vdsm(self): return self._vdsmCaller - def newUUID(self): + def _newUUID(self): return str(uuid.uuid4()) def randomName(self, base): return "%s_%04d" % (base, random.randint(1, 10000)) + def createStoragePool(self): + POOL_TYPE_DEPRECATED = 0 + self.vdsm().createStoragePool( + POOL_TYPE_DEPRECATED, + self.poolID(), + self.randomName('pool'), + self.domainID(), + [self.domainID()], + 1) + return self.poolID() + def connectStoragePool(self, poolID, masterDomainID): SCSI_KEY_DEPRECATED = 0 self.vdsm().connectStoragePool( diff --git a/tests/functional/testlib/storagecontexts/filebased.py b/tests/functional/testlib/storagecontexts/filebased.py new file mode 100644 index 0000000..bdb15d7 --- /dev/null +++ b/tests/functional/testlib/storagecontexts/filebased.py @@ -0,0 +1,69 @@ +import os +import storage.volume +import storage.image +from . import base +import logging + + +class Verify(base.Verify): + + def rhevMountPoint(self): + raise Exception('you must override this function') + + def storageServerConnected(self): + self.sleepWhileVDSMCompletesTask(duration=2) + assert os.path.exists(self.rhevMountPoint()) + + def storageDomainCreated(self, domainID): + self.sleepWhileVDSMCompletesTask(duration=1) + self._directIOTestFileExists() + domainRoot = os.path.join(self.rhevMountPoint(), domainID) + expectedFiles = [ + 'images', + 'dom_md', + 'dom_md/leases', + 'dom_md/inbox', + 'dom_md/outbox', + 'dom_md/metadata', + 'dom_md/ids'] + expectedFullPaths =\ + [os.path.join(domainRoot, path) for path in expectedFiles] + for path in expectedFullPaths: + logging.info('verifying path: %s' % path) + assert os.path.exists(path) + + def _directIOTestFileExists(self): + directIOTestFile = os.path.join( + self.rhevMountPoint(), + '__DIRECT_IO_TEST__') + assert os.path.exists(directIOTestFile) + + def volumeCreated(self, volumeInfo): + taskID, domainID, imageID, volumeID = volumeInfo + self.waitUntilVDSMTaskFinished(taskID, 20) + domainRoot = os.path.join(self.rhevMountPoint(), domainID) + imageDirectory = os.path.join(domainRoot, 'images', imageID) + assert os.path.exists(imageDirectory) + volumeFile = os.path.join(imageDirectory, volumeID) + volumeLease = '%s.lease' % volumeFile + volumeMeta = '%s.meta' % volumeFile + for path in volumeFile, volumeLease, volumeMeta: + logging.info('verifying path: %s' % path) + assert os.path.exists(path) + + +class FileBased(base.StorageBackend): + def createVolume(self, size): + PREALLOCATE = 1 + result = self.vdsm().createVolume( + self.domainID(), + self.poolID(), + self.imageID(), + self.largeIntegerXMLRPCWorkaround(size), + storage.volume.RAW_FORMAT, + PREALLOCATE, + storage.image.DATA_DISK_TYPE, + self.volumeID(), + self.randomName('volume_description')) + taskID = result['uuid'] + return taskID, self.domainID(), self.imageID(), self.volumeID() diff --git a/tests/functional/testlib/storagecontexts/iscsi.py b/tests/functional/testlib/storagecontexts/iscsi.py index 0e891ee..eb19715 100644 --- a/tests/functional/testlib/storagecontexts/iscsi.py +++ b/tests/functional/testlib/storagecontexts/iscsi.py @@ -60,11 +60,7 @@ base.StorageBackend.__init__(self) self._iqn = 'iqn.1970-01.functional.test:%04d' %\ random.randint(1, 10000) - self._volumeGroup = {'uuid': self.newUUID(), 'vgs_uuid': None} - self._poolID = self.newUUID() - self._connectionID = self.newUUID() - self._imageID = self.newUUID() - self._volumeID = self.newUUID() + self._volumeGroup = {'uuid': self.domainID(), 'vgs_uuid': None} def _targetcli(self, command): commandAsList = command.split() @@ -77,6 +73,16 @@ self._fileioBackstore = self.randomName('backfile') logging.info('using %s, %s' % (self._fileioBackstore, self._storageFile)) + self._setupISCSITarget() + return (self, + Verify( + self._iqn, + self._volumeGroup, + self.vdsm, + self.volumeID()) + ) + + def _setupISCSITarget(self): self._targetcli('/backstores/fileio create %s %s 10G' % (self._fileioBackstore, self._storageFile)) self._targetcli('/iscsi create %s' % self._iqn) @@ -88,13 +94,6 @@ self._targetcli('/iscsi/%s/tpg1 set attribute ' 'generate_node_acls=1 ' 'cache_dynamic_acls=1' % self._iqn) - return (self, - Verify( - self._iqn, - self._volumeGroup, - self.vdsm, - self._volumeID) - ) def __exit__(self, *args): doubleDashed = self._volumeGroup['uuid'].replace('-', '--') @@ -145,27 +144,13 @@ 'user': '', 'tpgt': '1', 'password': '', - 'id': self._connectionID, + 'id': self.connectionID(), 'port': '3260' } self.vdsm().disconnectStorageServer( storage.sd.ISCSI_DOMAIN, - self._poolID, + self.poolID(), [connection]) - - def createStoragePool(self): - POOL_TYPE_DEPRECATED = 0 - self.vdsm().createStoragePool( - POOL_TYPE_DEPRECATED, - self._poolID, - self.randomName('pool'), - self._domainID(), - [self._domainID()], - 1) - return self._poolID - - def _domainID(self): - return self._volumeGroup['uuid'] def _createVG(self): lun = self._findLUN() @@ -194,14 +179,14 @@ def createVolume(self, size): PREALLOCATE = 1 result = self.vdsm().createVolume( - self._domainID(), - self._poolID, - self._imageID, + self.domainID(), + self.poolID(), + self.imageID(), self.largeIntegerXMLRPCWorkaround(size), storage.volume.RAW_FORMAT, PREALLOCATE, storage.image.DATA_DISK_TYPE, - self._volumeID, + self.volumeID(), self.randomName('iscsi_description')) logging.info('createVolume result: %s' % result) return result['uuid'] diff --git a/tests/functional/testlib/storagecontexts/localfs.py b/tests/functional/testlib/storagecontexts/localfs.py index ef341a0..6ba2914 100644 --- a/tests/functional/testlib/storagecontexts/localfs.py +++ b/tests/functional/testlib/storagecontexts/localfs.py @@ -5,67 +5,24 @@ import storage.sd import storage.volume import storage.image -from . import base +from . import filebased -class Verify(base.Verify): +class Verify(filebased.Verify): def __init__(self, directory, vdsm): - base.Verify.__init__(self, vdsm) + filebased.Verify.__init__(self, vdsm) self._directory = directory - def storageServerConnected(self): - self.sleepWhileVDSMCompletesTask(duration=2) + def rhevMountPoint(self): transformedDirectory = self._directory.replace('/', '_') - expectedSymlink = os.path.join( + result = os.path.join( '/rhev/data-center/mnt/', transformedDirectory) - assert os.path.lexists(expectedSymlink) - - def storageDomainCreated(self, domainID): - self.sleepWhileVDSMCompletesTask(duration=1) - self._directIOTestFileExists() - domainRoot = os.path.join(self._directory, domainID) - expectedFiles = [ - 'images', - 'dom_md', - 'dom_md/leases', - 'dom_md/inbox', - 'dom_md/outbox', - 'dom_md/metadata', - 'dom_md/ids'] - expectedFullPaths =\ - [os.path.join(domainRoot, path) for path in expectedFiles] - for path in expectedFullPaths: - assert os.path.exists(path) - - def _directIOTestFileExists(self): - self.sleepWhileVDSMCompletesTask(duration=1) - directIOTestFile = os.path.join(self._directory, '__DIRECT_IO_TEST__') - assert os.path.exists(directIOTestFile) - - def volumeCreated(self, volumeInfo): - taskID, domainID, imageID, volumeID = volumeInfo - self.waitUntilVDSMTaskFinished(taskID, 20) - domain_directory = os.path.join(self._directory, domainID) - image_directory = os.path.join(domain_directory, 'images', imageID) - assert os.path.exists(image_directory) - volume_file = os.path.join(image_directory, volumeID) - volume_lease = '%s.lease' % volume_file - volume_meta = '%s.meta' % volume_file - for path in volume_file, volume_lease, volume_meta: - assert os.path.exists(path) + return result -class LocalFS(base.StorageBackend): +class LocalFS(filebased.FileBased): _NON_EXISTANT_POOL = '00000000-0000-0000-0000-000000000000' - - def __init__(self): - base.StorageBackend.__init__(self) - self._domainID = self.newUUID() - self._poolID = self.newUUID() - self._imageID = self.newUUID() - self._volumeID = self.newUUID() - self._connectionID = self.newUUID() def __enter__(self): self._createDirectoryForLocalFSStorage() @@ -87,13 +44,13 @@ 'user': '', 'tpgt': '1', 'password': '******', - 'id': self._connectionID, + 'id': self.connectionID(), 'port': ''} result = self.vdsm().connectStorageServer( storage.sd.LOCALFS_DOMAIN, self._NON_EXISTANT_POOL, [localFilesystemConnection]) - assert result['statuslist'][0]['id'] == self._connectionID + assert result['statuslist'][0]['id'] == self.connectionID() def disconnectStorageServer(self): localFilesystemConnection = { @@ -102,46 +59,20 @@ 'user': '', 'tpgt': '1', 'password': '******', - 'id': self._connectionID, + 'id': self.connectionID(), 'port': ''} self.vdsm().disconnectStorageServer( storage.sd.LOCALFS_DOMAIN, self._NON_EXISTANT_POOL, [localFilesystemConnection]) - def createStoragePool(self): - POOL_TYPE_DEPRECATED = 0 - self.vdsm().createStoragePool( - POOL_TYPE_DEPRECATED, - self._poolID, - self.randomName('pool'), - self._domainID, - [self._domainID], - 1) - return self._poolID - def createStorageDomain(self): DOMAIN_VERSION = 3 self.vdsm().createStorageDomain( storage.sd.LOCALFS_DOMAIN, - self._domainID, + self.domainID(), 'some_name', self._directory, storage.sd.DATA_DOMAIN, DOMAIN_VERSION) - return self._domainID - - def createVolume(self, size): - PREALLOCATE = 1 - result = self.vdsm().createVolume( - self._domainID, - self._poolID, - self._imageID, - self.largeIntegerXMLRPCWorkaround(size), - storage.volume.RAW_FORMAT, - PREALLOCATE, - storage.image.DATA_DISK_TYPE, - self._volumeID, - self.randomName('localfs_description')) - taskID = result['uuid'] - return taskID, self._domainID, self._imageID, self._volumeID + return self.domainID() diff --git a/tests/functional/testlib/storagecontexts/nfs.py b/tests/functional/testlib/storagecontexts/nfs.py new file mode 100644 index 0000000..cdde3a1 --- /dev/null +++ b/tests/functional/testlib/storagecontexts/nfs.py @@ -0,0 +1,115 @@ +import subprocess +import tempfile +import pwd +import shutil +import os +import storage.sd +from . import filebased + + +class Verify(filebased.Verify): + def __init__(self, directory, vdsm): + filebased.Verify.__init__(self, vdsm) + self._directory = directory + + def rhevMountPoint(self): + transformedDirectory = self._directory.replace('/', '_') + result = os.path.join( + '/rhev/data-center/mnt/', + '127.0.0.1:%s' % transformedDirectory) + return result + + +class NFS(filebased.FileBased): + _NON_EXISTANT_POOL = '00000000-0000-0000-0000-000000000000' + + def __enter__(self): + self._setupNFSExport() + return self, Verify(self._directory, self.vdsm) + + def _setupNFSExport(self): + self._export() + self._restartNFSServer() + mountPoint = tempfile.mkdtemp() + COMMAND = 'sudo --non-interactive mount -t nfs localhost:%s %s' %\ + (self._directory, mountPoint) + subprocess.check_call(COMMAND, shell=True) + self._setPermissions(mountPoint) + subprocess.check_call( + 'sudo --non-interactive umount %s' % mountPoint, shell=True) + os.rmdir(mountPoint) + + def _export(self): + self._directory = tempfile.mkdtemp('', 'nfstest', '/var/tmp') + self._previousExports = open('/etc/exports').read() + with open('/etc/exports', 'a') as f: + line = '%s 127.0.0.1(rw,sync,no_root_squash,no_all_squash,fsid=0)\n'\ + % self._directory + f.write(line) + + def _unexport(self): + self._umount() + with open('/etc/exports', 'w') as f: + f.write(self._previousExports) + self._restartNFSServer() + + def _umount(self): + COMMAND = 'sudo --non-interactive umount 127.0.0.1:%s'\ + % self._directory + code = subprocess.call(COMMAND, shell=True) + + def _setPermissions(self, directory): + vdsmUser = pwd.getpwnam('vdsm') + os.chown(directory, vdsmUser.pw_uid, vdsmUser.pw_gid) + os.chmod(directory, 0755) + + def _restartNFSServer(self): + subprocess.check_call( + 'sudo --non-interactive service nfs-server restart', shell=True) + + def __exit__(self, *args): + self._unexport() + shutil.rmtree(self._directory) + + def connectStorageServer(self): + connection = { + 'connection': self._nfsPath(), + 'iqn': '', + 'user': '', + 'tpgt': '1', + 'password': '******', + 'id': self.connectionID(), + 'port': ''} + result = self.vdsm().connectStorageServer( + storage.sd.NFS_DOMAIN, + self._NON_EXISTANT_POOL, + [connection]) + assert result['statuslist'][0]['id'] == self.connectionID() + + def disconnectStorageServer(self): + connection = { + 'connection': self._nfsPath(), + 'iqn': '', + 'user': '', + 'tpgt': '1', + 'password': '******', + 'id': self.connectionID(), + 'port': ''} + self.vdsm().disconnectStorageServer( + storage.sd.LOCALFS_DOMAIN, + self._NON_EXISTANT_POOL, + [connection]) + + def createStorageDomain(self): + DOMAIN_VERSION = 3 + self.vdsm().createStorageDomain( + storage.sd.NFS_DOMAIN, + self.domainID(), + 'some_name', + self._nfsPath(), + storage.sd.DATA_DOMAIN, + DOMAIN_VERSION) + return self.domainID() + + def _nfsPath(self): + return '127.0.0.1:%s' % self._directory -- To view, visit http://gerrit.ovirt.org/32990 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1781fc400c0604855d3143dde22ccb29e6cc8013 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yoav Kleinberger <ykleinbe(a)redhat.com>

4 6

Change in vdsm[master]: tests: new functional tests for vdsm storage
by ykleinbe＠redhat.com 08 Oct '15

08 Oct '15

Yoav Kleinberger has uploaded a new change for review. Change subject: tests: new functional tests for vdsm storage ...................................................................... tests: new functional tests for vdsm storage Ultimately, the purpose of this patch is to replace the existing tests/functional/storageTests.py, henceforth "the old test". The old test does not, in fact, verify VDSM behaviour. It only checks for the return codes that VDSM returns to its caller. This patch introduces a framework of "test contexts" that is extensible to various storage backends. Each test context, be it iscsi, nfs or some other storage type, knows how to tell VDSM to create its particular type of storage domain, and also knows how to verify that observable actions (e.g. the creation of a logical volume in an LVM volume group) have actually been performed. Currently, only localfs and iscsi are supported. Other storage backends will be added later. Change-Id: I1703e7c1dc223ff707775865cd14c7dd62314caf Bug-Url: https://bugzilla.redhat.com/?????? Signed-off-by: Yoav Kleinberger <ykleinbe(a)redhat.com> --- A tests/functional/basicStorageTest.py A tests/functional/testlib/__init__.py A tests/functional/testlib/controlvdsm.py A tests/functional/testlib/testcontexts/__init__.py A tests/functional/testlib/testcontexts/base.py A tests/functional/testlib/testcontexts/iscsi.py A tests/functional/testlib/testcontexts/localfs.py A tests/run_functional_storage_tests.sh 8 files changed, 459 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/96/32496/1 diff --git a/tests/functional/basicStorageTest.py b/tests/functional/basicStorageTest.py new file mode 100644 index 0000000..c78e4b2 --- /dev/null +++ b/tests/functional/basicStorageTest.py @@ -0,0 +1,42 @@ +import storage.volume +import storage.image +import logging +from testlib import controlvdsm +from testlib.testcontexts import localfs +from testlib.testcontexts import iscsi + +class TestBasicStorageDomain: + @classmethod + def setup_class(cls): + logging.basicConfig(level=logging.DEBUG, format='%(asctime)s TEST %(levelname)s: %(message)s') + + def setup(self): + controlVDSM = controlvdsm.ControlVDSM() + controlVDSM.cleanup() + + def testCreateVolumeLocalFS(self): + self._testCreateVolume(localfs.LocalFS) + + def testCreateVolumeISCSI(self): + self._testCreateVolume(iscsi.ISCSI) + + def _testCreateVolume(self, storageContext): + with storageContext() as (vdsm, verify): + storageServerID = vdsm.connectStorageServer() + verify.storageServerConnected() + + domainID = vdsm.createStorageDomain() + verify.storageDomainCreated(domainID) + + poolID = vdsm.createStoragePool() + verify.storagePoolCreated(poolID, masterDomainID = domainID) + + vdsm.connectStoragePool(poolID, masterDomainID = domainID) + vdsm.spmStart(poolID) + verify.spmStarted(poolID) + + vdsm.activateStorageDomain(domainID, poolID) + GIGABYTE = 1024 ** 3 + volumeInfo = vdsm.createVolume(1 * GIGABYTE) + verify.volumeCreated(volumeInfo) + vdsm.disconnectStorageServer() diff --git a/tests/functional/testlib/__init__.py b/tests/functional/testlib/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/tests/functional/testlib/__init__.py diff --git a/tests/functional/testlib/controlvdsm.py b/tests/functional/testlib/controlvdsm.py new file mode 100644 index 0000000..a983bf3 --- /dev/null +++ b/tests/functional/testlib/controlvdsm.py @@ -0,0 +1,53 @@ +import subprocess +import logging +import vdsm.vdscli +import socket +import vdsm.config +import time + +class ControlVDSM: + def cleanup(self): + self._stopService() + assert not self._serviceRunning() + self._brutallyCleanFiles() + self._restartService() + return self._checkConnection() + + def _checkConnection(self): + useSSL = vdsm.config.config.getboolean('vars', 'ssl') + vdsmClient = vdsm.vdscli.connect(useSSL=useSSL) + RETRIES = 5 + for _ in range(RETRIES): + try: + vdsmClient.getStorageDomainsList() + logging.info('connection to VDSM succeeded') + return + except socket.error as e: + logging.warning('could not talk to VDSM: %s' % e) + time.sleep(1) + + raise Exception('could not connect to VDSM') + + def _stopService(self): + self._run("sudo service vdsmd stop") + + def _serviceRunning(self): + returnCode = subprocess.call('sudo service vdsmd status', shell=True, stdout=open('/dev/null','w'), stderr=open('/dev/null','w')) + logging.info('vdsm running: %s' % (returnCode == 0)) + return returnCode == 0 + + def _restartService(self): + self._run("sudo vdsm-tool configure --force") + self._run("sudo service vdsmd start") + + def _run(self, command): + logging.info('running: %s' % command) + returnCode = subprocess.call(command, shell=True, close_fds=True, stdout=open('/dev/null','w'), stderr=open('/dev/null','w')) + if returnCode != 0: + logging.warning('failure! command was: %s' % command) + else: + logging.info('finished.') + + def _brutallyCleanFiles(self): + logging.warning('removing /rhev/data-center without asking too many questions') + self._run('sudo rm -fr /rhev/data-center/*') diff --git a/tests/functional/testlib/testcontexts/__init__.py b/tests/functional/testlib/testcontexts/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/tests/functional/testlib/testcontexts/__init__.py diff --git a/tests/functional/testlib/testcontexts/base.py b/tests/functional/testlib/testcontexts/base.py new file mode 100644 index 0000000..bccebdf --- /dev/null +++ b/tests/functional/testlib/testcontexts/base.py @@ -0,0 +1,101 @@ +import os +import vdsm.vdscli +import vdsm.config +import random +import uuid +import logging +import time + +class Verify: + def __init__(self, vdsm): + self._vdsm = vdsm + + def assertPathExists(self, path, link = False): + logging.info('verifying path: %s' % path) + if link: + assert os.path.lexists(path) + else: + assert os.path.exists(path) + + def assertPathDoesNotExist(self, path, link = False): + if link: + assert not os.path.lexists(path) + else: + assert not os.path.exists(path) + + def waitForVDSMToFinishTask(self, duration = 1): + time.sleep(duration) + + def storagePoolCreated(self, poolID, masterDomainID): + self.waitForVDSMToFinishTask() + linkToDomain = os.path.join('/rhev/data-center', poolID, masterDomainID) + masterAliasLink = os.path.join('/rhev/data-center', poolID, 'mastersd') + logging.info('verifying domain %s in pool %s' % (masterDomainID, poolID)) + assert os.path.lexists(linkToDomain) + logging.info('verifying symlink to master domain') + assert os.path.lexists(masterAliasLink) + + def waitFor(self, timeout, description, predicate, *args, **kwargs): + logging.info('waiting for "%s"' % description) + start = time.time() + for _ in xrange(timeout): + if predicate(*args, **kwargs): + logging.info('it took %s seconds' % (time.time() - start)) + return + time.sleep(1) + + assert False, 'waited %s seconds for "%s" but it did not happen' % (timeout, description) + + def spmStarted(self, poolID): + self.waitForVDSMToFinishTask() + masterDomainDirectory = '/rhev/data-center/%s/mastersd' % poolID + master = os.path.join(masterDomainDirectory, 'master') + tasks = os.path.join(master, 'tasks') + vms = os.path.join(master, 'vms') + allExist = lambda: ( os.path.exists(master) and os.path.exists(tasks) and os.path.exists(vms) ) + self.waitFor(60, 'SPM related subdirectories exist', allExist) + + def waitOnVDSMTask(self, taskID, timeout): + taskFinished = lambda: (self._taskStatus(taskID)['taskState'] == 'finished') + self.waitFor(timeout, 'vdsm task to be finished', taskFinished) + taskStatus = self._taskStatus(taskID) + assert taskStatus['code'] == 0, taskStatus['message'] + + def _taskStatus(self, taskID): + result = self._vdsm.getTaskStatus(taskID) + assert result['status']['code'] == 0 + taskStatus = result['taskStatus'] + return taskStatus + +class StorageBackend: + def __init__(self): + useSSL = vdsm.config.config.getboolean('vars', 'ssl') + self._vdsm = vdsm.vdscli.connect(useSSL=useSSL) + + def newUUID(self): + return str(uuid.uuid4()) + + def randomName(self, base): + return "%s_%04d" % (base, random.randint(1,10000)) + + def connectStoragePool(self, poolID, masterDomainID): + SCSI_KEY_DEPRECATED = 0 + result = self._vdsm.connectStoragePool(poolID, 1, SCSI_KEY_DEPRECATED, masterDomainID, 1) + self.verifyVDSMSuccess(result) + + def spmStart(self, poolID): + RECOVERY_MODE_DEPRECATED = 0 + SCSI_FENCING_DEPRECATED = 0 + result = self._vdsm.spmStart(poolID, -1, '-1', SCSI_FENCING_DEPRECATED, RECOVERY_MODE_DEPRECATED) + self.verifyVDSMSuccess(result) + + def activateStorageDomain(self, domainID, poolID): + result = self._vdsm.activateStorageDomain(domainID,poolID) + self.verifyVDSMSuccess(result) + + def stringForXMLRPC(self, number): + return str(number) + + def verifyVDSMSuccess(self, result): + if result[ 'status' ][ 'code' ] != 0: + raise Exception('expected OK result from VDSM, got "%s" instead' % str(result)) diff --git a/tests/functional/testlib/testcontexts/iscsi.py b/tests/functional/testlib/testcontexts/iscsi.py new file mode 100644 index 0000000..96d6958 --- /dev/null +++ b/tests/functional/testlib/testcontexts/iscsi.py @@ -0,0 +1,150 @@ +import os +import random +import logging +import tempfile +import shutil +import subprocess +import glob +import storage.sd +import storage.volume +import storage.image +from . import base + +class Verify(base.Verify): + def __init__(self, iqn, volumeGroup, vdsm, volumeID): + base.Verify.__init__(self, vdsm) + self._iqn = iqn + self._volumeGroup = volumeGroup + self._volumeID = volumeID + + def storageServerConnected(self): + targetNameFiles = glob.glob('/sys/devices/platform/host*/session*/iscsi_session/*/targetname') + targetNames = [ open(file).read().strip() for file in targetNameFiles ] + assert self._iqn in targetNames + + def storageDomainCreated(self, domainID): + self.waitFor(10, 'storage domain exists', self._storageDomainVolumesExist, domainID) + + def _storageDomainVolumesExist(self, domainID): + for name in [ 'ids', 'inbox', 'leases', 'master', 'metadata', 'outbox' ]: + doubleDashed = domainID.replace('-', '--') + expectedPath = os.path.join('/dev/mapper', '%s-%s' % (doubleDashed, name)) + logging.info('verifying: %s' % expectedPath) + if not os.path.lexists(expectedPath): + return False + + return True + + def volumeCreated(self, taskID): + self.waitOnVDSMTask(taskID, 10) + result = subprocess.call('sudo lvs %s | grep %s' % (self._volumeGroup['uuid'], self._volumeID), shell = True) + assert result == 0, "did not find logical volume in volume group" + +class ISCSI(base.StorageBackend): + _NULL_UUID = '00000000-0000-0000-0000-000000000000' + def __init__(self): + base.StorageBackend.__init__(self) + self._iqn = 'iqn.1970-01.functional.test:%04d' % random.randint(1,10000) + self._volumeGroup = { 'uuid': self.newUUID(), 'vgs_uuid': None } + self._poolID = self.newUUID() + self._connectionID = self.newUUID() + self._imageID = self.newUUID() + self._volumeID = self.newUUID() + logging.info( 'using the following attributes:' ) + for name in [ '_volumeGroup', '_iqn', '_poolID', '_connectionID', '_imageID', '_volumeID' ]: + logging.info( '%s => %s' % ( name, getattr(self,name) ) ) + + def _targetcli(self, command): + commandAsList = command.split() + logging.info('running targetcli: targetcli %s' % commandAsList) + subprocess.check_call( [ 'targetcli' ] + commandAsList, close_fds = True ) + + def __enter__(self): + self._testDirectory = tempfile.mkdtemp() + self._storageFile = os.path.join(self._testDirectory, 'testfile') + self._fileioBackstore = self.randomName('backfile') + logging.info('using %s, %s' % (self._fileioBackstore, self._storageFile)) + self._targetcli('/backstores/fileio create %s %s 10G' % (self._fileioBackstore, self._storageFile)) + self._targetcli('/iscsi create %s' % self._iqn) + self._targetcli('/iscsi/%s/tpg1/luns create /backstores/fileio/%s' % (self._iqn, self._fileioBackstore)) + self._targetcli('/iscsi/%s/tpg1 set attribute authentication=0 demo_mode_write_protect=0' % self._iqn) + self._targetcli('/iscsi/%s/tpg1 set attribute generate_node_acls=1 cache_dynamic_acls=1' % self._iqn) + return self, Verify(self._iqn, self._volumeGroup, self._vdsm, self._volumeID) + + def __exit__(self, *args): + doubleDashed = self._volumeGroup['uuid'].replace('-', '--') + mapperDevices = glob.glob('/dev/mapper/%s*' % doubleDashed ) + [ '/dev/mapper/%s' % self._lunGUID ] + for device in mapperDevices: + logging.info('removing %s' % device) + result = subprocess.call('sudo dmsetup remove %s' % device, shell=True) + if result != 0: + logging.warning('could not remove %s' % device) + self._targetcli( '/iscsi delete %s' % self._iqn ) + self._targetcli( '/backstores/fileio delete %s' % self._fileioBackstore ) + shutil.rmtree(self._testDirectory) + + def connectStorageServer(self): + connection = {'connection': '127.0.0.1', 'iqn': self._iqn, 'user': '', 'tpgt': '1', 'password': '', 'id': self._NULL_UUID, 'port': '3260'} + result = self._vdsm.connectStorageServer( + storage.sd.ISCSI_DOMAIN, + self._NULL_UUID, + [ connection ]) + self.verifyVDSMSuccess(result) + connectionDetails = result['statuslist'][0] + assert connectionDetails['status'] == 0 + logging.info( 'got id %s', connectionDetails['id'] ) + + def _findLUN(self): + result = self._vdsm.getDeviceList(storage.sd.ISCSI_DOMAIN) + for device in result['devList']: + for path in device['pathlist']: + if path['iqn'] == self._iqn: + logging.info( 'vdsm reports our LUN %s' % device ) + return device + + raise Exception("LUN not found!") + + def disconnectStorageServer(self): + connection = {'connection': '127.0.0.1', 'iqn': self._iqn, 'user': '', 'tpgt': '1', 'password': '', 'id': self._connectionID, 'port': '3260'} + result = self._vdsm.disconnectStorageServer( + storage.sd.ISCSI_DOMAIN, + self._poolID, + [ connection ]) + self.verifyVDSMSuccess(result) + + def createStoragePool(self): + POOL_TYPE_DEPRECATED = 0 + result = self._vdsm.createStoragePool(POOL_TYPE_DEPRECATED, self._poolID, self.randomName('pool'), self._domainID(), [ self._domainID() ], 1) + self.verifyVDSMSuccess(result) + return self._poolID + + def _domainID(self): + return self._volumeGroup['uuid'] + + def _createVG(self): + lun = self._findLUN() + result = self._vdsm.createVG(self._volumeGroup['uuid'], [ lun['GUID'] ]) + logging.info('createVG returned %s' % result) + self.verifyVDSMSuccess(result) + self._volumeGroup[ 'vgs_uuid' ] = result[ 'uuid' ] + self._lunGUID = lun['GUID'] + + def createStorageDomain(self): + self._createVG() + self._createStorageDomain() + return self._volumeGroup['uuid'] + + def _createStorageDomain(self): + domainID = self._volumeGroup['uuid'] + DOMAIN_VERSION = 3 + domainName = self.randomName( 'some_name' ) + result = self._vdsm.createStorageDomain(storage.sd.ISCSI_DOMAIN, domainID, domainName, self._volumeGroup['vgs_uuid'], storage.sd.DATA_DOMAIN, DOMAIN_VERSION) + self.verifyVDSMSuccess(result) + + def createVolume(self, size): + PREALLOCATE = 1 + result = self._vdsm.createVolume(self._domainID(), self._poolID, self._imageID, + self.stringForXMLRPC(size), storage.volume.RAW_FORMAT, PREALLOCATE, storage.image.DATA_DISK_TYPE, self._volumeID, self.randomName('iscsi_description')) + logging.info( 'createVolume result: %s' % result) + self.verifyVDSMSuccess(result) + return result[ 'uuid' ] diff --git a/tests/functional/testlib/testcontexts/localfs.py b/tests/functional/testlib/testcontexts/localfs.py new file mode 100644 index 0000000..b498b6e --- /dev/null +++ b/tests/functional/testlib/testcontexts/localfs.py @@ -0,0 +1,104 @@ +import tempfile +import pwd +import shutil +import os +import storage.sd +import storage.volume +import storage.image +from . import base + +class Verify(base.Verify): + def __init__(self, directory, vdsm): + base.Verify.__init__(self, vdsm) + self._directory = directory + + def storageServerConnected(self): + self.waitForVDSMToFinishTask(2) + transformedDirectory = self._directory.replace( '/', '_' ) + expectedSymlink = os.path.join('/rhev/data-center/mnt/', transformedDirectory) + self.assertPathExists(expectedSymlink, link = True) + + def storageDomainCreated(self, domainID): + self.waitForVDSMToFinishTask() + self._directIOTestFileExists() + domainRoot = os.path.join(self._directory, domainID) + expectedFiles = [ 'images', 'dom_md', 'dom_md/leases', 'dom_md/inbox', 'dom_md/outbox', 'dom_md/metadata', 'dom_md/ids' ] + expectedFullPaths = [ os.path.join(domainRoot, path) for path in expectedFiles ] + for path in expectedFullPaths: + assert os.path.exists(path) + + def _directIOTestFileExists(self): + self.waitForVDSMToFinishTask() + directIOTestFile = os.path.join(self._directory, '__DIRECT_IO_TEST__') + self.assertPathExists(directIOTestFile) + + def volumeCreated(self, volumeInfo): + taskID, domainID, imageID, volumeID = volumeInfo + self.waitOnVDSMTask(taskID, 20) + domain_directory = os.path.join(self._directory, domainID) + image_directory = os.path.join(domain_directory, 'images', imageID) + self.assertPathExists(image_directory) + volume_file = os.path.join(image_directory, volumeID) + volume_lease = '%s.lease' % volume_file + volume_meta = '%s.meta' % volume_file + for path in volume_file, volume_lease, volume_meta: + self.assertPathExists(path) + +class LocalFS(base.StorageBackend): + _NON_EXISTANT_POOL = '00000000-0000-0000-0000-000000000000' + def __init__(self): + base.StorageBackend.__init__(self) + self._domainID = self.newUUID() + self._poolID = self.newUUID() + self._imageID = self.newUUID() + self._volumeID = self.newUUID() + self._connectionID = self.newUUID() + + def __enter__(self): + self._createDirectoryForLocalFSStorage() + return self, Verify(self._directory, self._vdsm) + + def _createDirectoryForLocalFSStorage(self): + self._directory = tempfile.mkdtemp('', 'localfstest', '/var/tmp') + vdsm_user = pwd.getpwnam('vdsm') + os.chown(self._directory, vdsm_user.pw_uid, vdsm_user.pw_gid) + os.chmod(self._directory, 0755) + + def __exit__(self, *args): + shutil.rmtree(self._directory) + + def connectStorageServer(self): + localFilesystemConnection = {'connection': self._directory, 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': self._connectionID, 'port': ''} + result = self._vdsm.connectStorageServer( + storage.sd.LOCALFS_DOMAIN, + self._NON_EXISTANT_POOL, + [ localFilesystemConnection ]) + self.verifyVDSMSuccess(result) + assert result[ 'statuslist' ][ 0 ][ 'id' ] == self._connectionID + + def disconnectStorageServer(self): + localFilesystemConnection = {'connection': self._directory, 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': self._connectionID, 'port': ''} + result = self._vdsm.disconnectStorageServer( + storage.sd.LOCALFS_DOMAIN, + self._NON_EXISTANT_POOL, + [ localFilesystemConnection ]) + self.verifyVDSMSuccess(result) + + def createStoragePool(self): + POOL_TYPE_DEPRECATED = 0 + result = self._vdsm.createStoragePool(POOL_TYPE_DEPRECATED, self._poolID, self.randomName('pool'), self._domainID, [ self._domainID ], 1) + self.verifyVDSMSuccess(result) + return self._poolID + + def createStorageDomain(self): + DOMAIN_VERSION = 3 + result = self._vdsm.createStorageDomain(storage.sd.LOCALFS_DOMAIN, self._domainID, 'some_name', self._directory, storage.sd.DATA_DOMAIN, DOMAIN_VERSION) + self.verifyVDSMSuccess(result) + return self._domainID + + def createVolume(self, size): + PREALLOCATE = 1 + result = self._vdsm.createVolume(self._domainID, self._poolID, self._imageID, self.stringForXMLRPC(size), storage.volume.RAW_FORMAT, PREALLOCATE, storage.image.DATA_DISK_TYPE, self._volumeID, self.randomName('localfs_description')) + self.verifyVDSMSuccess(result) + taskID = result[ 'uuid' ] + return taskID, self._domainID, self._imageID, self._volumeID diff --git a/tests/run_functional_storage_tests.sh b/tests/run_functional_storage_tests.sh new file mode 100755 index 0000000..cb544ee --- /dev/null +++ b/tests/run_functional_storage_tests.sh @@ -0,0 +1,9 @@ +#!/bin/bash +if [[ "$1" = "--help" ]]; then + echo "run_functional_storage_tests.sh [--verbose]" + exit -1 +fi +if [[ "$1" = "--verbose" ]]; then + verbose="--nologcapture" +fi +sudo PYTHONPATH=../lib:../vdsm:functional nosetests -s $verbose functional/basicStorageTest.py -- To view, visit http://gerrit.ovirt.org/32496 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1703e7c1dc223ff707775865cd14c7dd62314caf Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yoav Kleinberger <ykleinbe(a)redhat.com>

6 26

Change in vdsm[master]: tests: alignmentScanTests.py is a functional test
by ykleinbe＠redhat.com 08 Oct '15

08 Oct '15

Yoav Kleinberger has uploaded a new change for review. Change subject: tests: alignmentScanTests.py is a functional test ...................................................................... tests: alignmentScanTests.py is a functional test Since alignmentScanTests invokes actual storage code, and does not mock it out, it should be categorized as a functional test. Change-Id: I4e20d17c3ebee1203bb5a721ce44d5867570ce8e Signed-off-by: Yoav Kleinberger <ykleinbe(a)redhat.com> --- M tests/Makefile.am M tests/functional/Makefile.am R tests/functional/alignmentScanTests.py 3 files changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/45/29745/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 4ef8f7d..f7f4f97 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -23,7 +23,6 @@ SUBDIRS = functional test_modules = \ - alignmentScanTests.py \ blocksdTests.py \ bridgeTests.py \ cPopenTests.py \ diff --git a/tests/functional/Makefile.am b/tests/functional/Makefile.am index b87fdf0..4fc2cc8 100644 --- a/tests/functional/Makefile.am +++ b/tests/functional/Makefile.am @@ -33,6 +33,7 @@ vmRecoveryTests.py \ storageTests.py \ veth.py \ + alignmentScanTests.py \ $(NULL) dist_vdsmfunctests_DATA = \ diff --git a/tests/alignmentScanTests.py b/tests/functional/alignmentScanTests.py similarity index 100% rename from tests/alignmentScanTests.py rename to tests/functional/alignmentScanTests.py -- To view, visit http://gerrit.ovirt.org/29745 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4e20d17c3ebee1203bb5a721ce44d5867570ce8e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yoav Kleinberger <ykleinbe(a)redhat.com>

9 39

Change in vdsm[master]: tests: use 'localhost' explicitly in test
by ykleinbe＠redhat.com 08 Oct '15

08 Oct '15

Yoav Kleinberger has uploaded a new change for review. Change subject: tests: use 'localhost' explicitly in test ...................................................................... tests: use 'localhost' explicitly in test Previously tests could on some machines (in case the machine has a non-default hostname). Now, since we use 'localhost' explicitly, this will not happen. Change-Id: I89990cff46e64120262e250eee9238b49c4edee4 Signed-off-by: Yoav Kleinberger <ykleinbe(a)redhat.com> --- M tests/functional/storageTests.py 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/07/28107/1 diff --git a/tests/functional/storageTests.py b/tests/functional/storageTests.py index 80ba312..76ca91d 100644 --- a/tests/functional/storageTests.py +++ b/tests/functional/storageTests.py @@ -79,7 +79,8 @@ isSSL = config.getboolean('vars', 'ssl') if isSSL and os.geteuid() != 0: raise SkipTest("Must be root to use SSL connection to server") - self.s = vdscli.connect(useSSL=isSSL) + address = 'localhost:%s' % config.get('addresses', 'management_port') + self.s = vdscli.connect(hostPort=address, useSSL=isSSL) def assertVdsOK(self, vdsResult): # code == 0 means OK -- To view, visit http://gerrit.ovirt.org/28107 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I89990cff46e64120262e250eee9238b49c4edee4 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yoav Kleinberger <ykleinbe(a)redhat.com>

8 20

Change in vdsm[master]: network: Inexistence of a network in libvirt should not be f...
by osvoboda＠redhat.com 29 Sep '15

29 Sep '15

Ondřej Svoboda has uploaded a new change for review. Change subject: network: Inexistence of a network in libvirt should not be fatal to us ...................................................................... network: Inexistence of a network in libvirt should not be fatal to us vdsm-restore-net-config (with unified persistence) is called often during functional network tests. This is a rollback scenario and connectivity is at stake. A network missing in libvirt is a sign of a different failure (and we are after you, traffic control!) but it must not harm the restoration of networking (not only) after failed tests. Change-Id: Ia6ae4b359d543b6e3ee560a5cb021e31d037c035 Signed-off-by: Ondřej Svoboda <osvoboda(a)redhat.com> --- M vdsm/network/api.py 1 file changed, 3 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/88/34988/1 diff --git a/vdsm/network/api.py b/vdsm/network/api.py index 1b09c6c..4a2b0c5 100755 --- a/vdsm/network/api.py +++ b/vdsm/network/api.py @@ -691,9 +691,9 @@ del libvirt_nets[network] _netinfo.updateDevices() elif 'remove' in networkAttrs: - raise ConfigNetworkError(ne.ERR_BAD_BRIDGE, "Cannot delete " - "network %r: It doesn't exist in the " - "system" % network) + logger.warning("Not deleting network {0!r}: it is unknown to " + "libvirtd".format(network)) + del networks[network] else: networksAdded.add(network) -- To view, visit http://gerrit.ovirt.org/34988 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia6ae4b359d543b6e3ee560a5cb021e31d037c035 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ondřej Svoboda <osvoboda(a)redhat.com>

4 8

Change in vdsm[master]: netinfo: Replace getVlanDevice/ID in bootstrap with ipwrappe...
by osvoboda＠redhat.com 28 Sep '15

28 Sep '15

Ondřej Svoboda has uploaded a new change for review. Change subject: netinfo: Replace getVlanDevice/ID in bootstrap with ipwrapper.getLink ...................................................................... netinfo: Replace getVlanDevice/ID in bootstrap with ipwrapper.getLink Change-Id: I4bfb0fa3c341a517bf444832d0531b05393ae4a9 Signed-off-by: Ondřej Svoboda <osvoboda(a)redhat.com> --- M lib/vdsm/netinfo.py M vdsm_reg/deployUtil.py.in 2 files changed, 16 insertions(+), 16 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/60/35360/1 diff --git a/lib/vdsm/netinfo.py b/lib/vdsm/netinfo.py index 10bbecb..aaae19f 100644 --- a/lib/vdsm/netinfo.py +++ b/lib/vdsm/netinfo.py @@ -223,7 +223,7 @@ def vlanSpeed(vlanName): """Returns the vlan's underlying device speed.""" - vlanDevName = getVlanDevice(vlanName) + vlanDevName = _getVlanDevice(vlanName) vlanDev = getLink(vlanDevName) if vlanDev.isNIC(): speed = nicSpeed(vlanDevName) @@ -328,8 +328,8 @@ nics = [] for iface in ports(bridge): if iface in vlans(): - iface = getVlanDevice(iface) - vlan = getVlanID(iface) + iface = _getVlanDevice(iface) + vlan = _getVlanID(iface) if iface in bondings(): bonding = iface nics = slaves(iface) @@ -690,13 +690,13 @@ return any(vlan.startswith(dev + '.') for vlan in vlans()) -def getVlanDevice(vlan): +def _getVlanDevice(vlan): """ Return the device of the given VLAN. """ vlanLink = getLink(vlan) return vlanLink.device -def getVlanID(vlan): +def _getVlanID(vlan): """ Return the ID of the given VLAN. """ vlanLink = getLink(vlan) return int(vlanLink.vlanid) @@ -785,7 +785,7 @@ if iface == interface: yield (network, None) elif interface.startswith(iface + '.'): - yield (network, getVlanID(interface)) + yield (network, _getVlanID(interface)) def getBridgelessNetworksAndVlansForIface(self, iface): """ Returns tuples of (network, vlan) connected to nic/bond """ @@ -794,11 +794,11 @@ if iface == netdict['iface']: yield (network, None) elif netdict['iface'].startswith(iface + '.'): - yield (network, getVlanID(netdict['iface'])) + yield (network, _getVlanID(netdict['iface'])) def getVlansForIface(self, iface): for vlanDevName in self.getVlanDevsForIface(iface): - yield getVlanID(vlanDevName) + yield _getVlanID(vlanDevName) def getVlanDevsForIface(self, iface): for v, vdict in self.vlans.iteritems(): @@ -852,8 +852,8 @@ for port in ports: if port in self.vlans: assert vlan is None - nic = getVlanDevice(port) - vlan = getVlanID(port) + nic = _getVlanDevice(port) + vlan = _getVlanID(port) assert self.vlans[port]['iface'] == nic port = nic if port in self.bondings: diff --git a/vdsm_reg/deployUtil.py.in b/vdsm_reg/deployUtil.py.in index 2899411..5910bd7 100644 --- a/vdsm_reg/deployUtil.py.in +++ b/vdsm_reg/deployUtil.py.in @@ -972,21 +972,21 @@ This helper function extract the relevant parameters of the existing RHEL interface, in order to create a managment bridge. """ - netinfo = vdsmImport("netinfo") + ipwrapper = vdsmImport("ipwrapper") vlan = '' bonding = '' nic = None try: - vlans = netinfo.vlans() - if mgtIface in vlans: - nic = netinfo.getVlanDevice(mgtIface) - vlan = netinfo.getVlanID(mgtIface) + link = ipwrapper.getLink(mgtIface) + if link.isVLAN(): + nic = link.device + vlan = link.vlanid else: nic = mgtIface - if nic in netinfo.bondings(): + if ipwrapper.getLink(nic).isBOND(): logging.error( ( "_getRHELBridgeParams Found bonding: %s." -- To view, visit http://gerrit.ovirt.org/35360 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4bfb0fa3c341a517bf444832d0531b05393ae4a9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ondřej Svoboda <osvoboda(a)redhat.com>

5 8

Change in vdsm[master]: zombiereaper: Add ability to wait on process
by smizrahi＠redhat.com 19 Sep '15

19 Sep '15

Saggi Mizrahi has uploaded a new change for review. Change subject: zombiereaper: Add ability to wait on process ...................................................................... zombiereaper: Add ability to wait on process This adds the ability to wait on a process with timeout in an efficient manner. When the process receives SIGCHLD in addition to clearing the tracked pids it will now call poll() on requested C\Popen objects. This will allow the called to synchronously wait for a process to exit with timeout. Change-Id: Idcd69b6187bc1c412c11f6cba9af431557ea7077 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/vdsm/infra/zombiereaper/__init__.py M lib/vdsm/infra/zombiereaper/tests.py 2 files changed, 79 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/35055/1 diff --git a/lib/vdsm/infra/zombiereaper/__init__.py b/lib/vdsm/infra/zombiereaper/__init__.py index f85ec30..2169ffa 100644 --- a/lib/vdsm/infra/zombiereaper/__init__.py +++ b/lib/vdsm/infra/zombiereaper/__init__.py @@ -28,8 +28,10 @@ import os import signal +from threading import Condition _trackedPids = set() +_waited_on_procs = {} def autoReapPID(pid): @@ -46,6 +48,7 @@ pid, rv = os.waitpid(pid, os.WNOHANG) if pid != 0: _trackedPids.discard(pid) + except OSError: _trackedPids.discard(pid) @@ -54,6 +57,52 @@ for pid in _trackedPids.copy(): _tryReap(pid) + for proc in _waited_on_procs.keys(): + _tryWait(proc) + + +def _tryWait(proc): + res = None + err = None + try: + res = proc.poll() + except Exception as e: + err = e + finally: + try: + c = _waited_on_procs[proc] + except KeyError: + pass + else: + with c: + if c == _waited_on_procs[proc]: + _waited_on_procs[proc] = (res, err) + c.notify_all() + + +def waitproc(proc, timeout=None): + """Wait until a process exits + + Will block until the process exists or timeout (in seconds) has passed. + Passing None as timeout (the default) will block forever but in that case + it's recommended to just use the built in wait() functionality. + """ + c = Condition() + with c: + if c != _waited_on_procs.setdefault(proc, c): + raise ValueError("proc already being tracked") + + c.wait(timeout) + res = _waited_on_procs.pop(proc) + if res == c: + return None + else: + rv, err = res + if err is not None: + raise err + + return rv + def registerSignalHandler(): """ diff --git a/lib/vdsm/infra/zombiereaper/tests.py b/lib/vdsm/infra/zombiereaper/tests.py index ee4fb02..817aa13 100644 --- a/lib/vdsm/infra/zombiereaper/tests.py +++ b/lib/vdsm/infra/zombiereaper/tests.py @@ -19,16 +19,17 @@ # from time import sleep import os +from threading import Thread + +import nose.tools as nt from .. import zombiereaper from cpopen import CPopen -from unittest import TestCase - zombiereaper.registerSignalHandler() -class zombieReaperTests(TestCase): +class TestZombieReaper(object): def testProcessDiesAfterBeingTracked(self): p = CPopen(["sleep", "1"]) zombiereaper.autoReapPID(p.pid) @@ -36,7 +37,7 @@ sleep(4) # Throws error because pid is not found or is not child - self.assertRaises(OSError, os.waitpid, p.pid, os.WNOHANG) + nt.assert_raises(OSError, os.waitpid, p.pid, os.WNOHANG) def testProcessDiedBeforeBeingTracked(self): p = CPopen(["sleep", "0"]) @@ -46,4 +47,28 @@ zombiereaper.autoReapPID(p.pid) # Throws error because pid is not found or is not child - self.assertRaises(OSError, os.waitpid, p.pid, os.WNOHANG) + nt.assert_raises(OSError, os.waitpid, p.pid, os.WNOHANG) + + def testWaitProc(self): + p = CPopen(["sleep", "1"]) + nt.assert_is_not_none(zombiereaper.waitproc(p, 2)) + nt.assert_is_not_none(p.returncode) + + def testWaitProcCallTwice(self): + p = CPopen(["sleep", "3"]) + nt.assert_is_none(zombiereaper.waitproc(p, 1)) + nt.assert_is_not_none(zombiereaper.waitproc(p, 3)) + nt.assert_is_not_none(p.returncode) + + def testWaitProcRegisterTwice(self): + p = CPopen(["sleep", "2"]) + t = Thread( + name="waiter", + target=zombiereaper.waitproc, + args=(p, 4), + ) + t.setDaemon(True) + t.start() + sleep(1) + nt.assert_raises(ValueError, zombiereaper.waitproc, p, 1) + t.join() -- To view, visit http://gerrit.ovirt.org/35055 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idcd69b6187bc1c412c11f6cba9af431557ea7077 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

7 11

Change in vdsm[master]: vmxml.py, vm.py: Remove QEMU passthrough hack in ppc64
by vdelima＠redhat.com 16 Sep '15

16 Sep '15

Vitor de Lima has uploaded a new change for review. Change subject: vmxml.py, vm.py: Remove QEMU passthrough hack in ppc64 ...................................................................... vmxml.py, vm.py: Remove QEMU passthrough hack in ppc64 Recent libvirt versions already include a virtual USB keyboard and mouse in guests with graphical consoles. This patch does not include any information about input devices in the domain XML in order to libvirt handle the creation of these devices automatically. Change-Id: I64d660bf7534203d5d5cdbc318ffd1429a16f954 Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1122627 Signed-off-by: Vitor de Lima <vdelima(a)redhat.com> --- M vdsm/virt/vm.py M vdsm/virt/vmxml.py 2 files changed, 3 insertions(+), 27 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/33871/1 diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index 1eebb69..5b39752 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -2679,7 +2679,9 @@ _VMCHANNEL_DEVICE_NAME) domxml._appendAgentDevice(self._qemuguestSocketFile.decode('utf-8'), _QEMU_GA_DEVICE_NAME) - domxml.appendInput() + + if self.arch == caps.Architecture.X86_64: + domxml.appendInput() if self.arch == caps.Architecture.PPC64: domxml.appendEmulator() @@ -2694,9 +2696,6 @@ for drive in self._devices[DISK_DEVICES][:]: for leaseElement in drive.getLeasesXML(): domxml._devices.appendChild(leaseElement) - - if self.arch == caps.Architecture.PPC64: - domxml.appendKeyboardDevice() return domxml.toxml() diff --git a/vdsm/virt/vmxml.py b/vdsm/virt/vmxml.py index eb07ba9..a83adb1 100644 --- a/vdsm/virt/vmxml.py +++ b/vdsm/virt/vmxml.py @@ -165,14 +165,6 @@ domainAttrs = {'type': domainType} - # Hack around libvirt issue BZ#988070, this is going to be removed as - # soon as the domain XML format supports the specification of USB - # keyboards - - if self.arch == caps.Architecture.PPC64: - domainAttrs['xmlns:qemu'] = \ - 'http://libvirt.org/schemas/domain/qemu/1.0' - self.dom = Element('domain', **domainAttrs) self.doc.appendChild(self.dom) @@ -444,21 +436,6 @@ inputAttrs = {'type': 'mouse', 'bus': mouseBus} self._devices.appendChildWithArgs('input', **inputAttrs) - - def appendKeyboardDevice(self): - """ - Add keyboard device for ppc64 using a QEMU argument directly. - This is a workaround to the issue BZ#988070 in libvirt - - <qemu:commandline> - <qemu:arg value='-usbdevice'/> - <qemu:arg value='keyboard'/> - </qemu:commandline> - """ - commandLine = Element('qemu:commandline') - commandLine.appendChildWithArgs('qemu:arg', value='-usbdevice') - commandLine.appendChildWithArgs('qemu:arg', value='keyboard') - self.dom.appendChild(commandLine) def appendEmulator(self): emulatorPath = '/usr/bin/qemu-system-' + self.arch -- To view, visit http://gerrit.ovirt.org/33871 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I64d660bf7534203d5d5cdbc318ffd1429a16f954 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Vitor de Lima <vdelima(a)redhat.com>

8 59

← Newer
1
2
3
4
5
6
7
8
9
...
64
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches November 2014