Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=c09aa3a299201a0c…
Commit: c09aa3a299201a0cf34485fb8b1738fb81215bfa
Parent: 0000000000000000000000000000000000000000
Author: Alasdair G Kergon <agk(a)redhat.com>
AuthorDate: 2016-11-30 23:15 +0000
Committer: Alasdair G Kergon <agk(a)redhat.com>
CommitterDate: 2016-11-30 23:15 +0000
annotated tag: v2_02_168 has been created
at c09aa3a299201a0cf34485fb8b1738fb81215bfa (tag)
tagging 57e24817b70cf577eea5e57e30a6c3d21ac550fc (commit)
replaces v2_02_167
Release 2.02.168
59 files changed, 1663 insertions(+), 1143 deletions(-)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
iEYEABECAAYFAlg/XeoACgkQIoGRwVZ+LBeBxACfeV5nIv3nXba0q1+KEPrfzHku
6WMAoPzr6td3KBlnWf0esLqTZB5ydToz
=rtKi
-----END PGP SIGNATURE-----
Alasdair G Kergon (4):
post-release
lvconvert: Introduce enum to separate cases.
raid: Remove fixed FIXME
pre-release
Bryn M. Reeves (1):
libdm: separate dm_stats_populate() error cases
Heinz Mauelshagen (5):
dmsetup: return 0 for [--]{version,help}
dmsetup: [v2] return 0 for [--]{version,help}
lvchange: allow a transiently failed RaidLV to be refreshed
WHATS_NEW: Allow a transiently failed RaidLV to be refreshed
raid: fix sync percent on large RaidLVs
Peter Rajnoha (4):
dbus: only log msg as debug if lvm2-lvmdbusd unit missing for D-Bus notification
dbus: also recognize error state for missing service that comes from original D-Bus
man: remove duplicate paragraph about selection criteria from lvm(8) man page
scripts: fix comment about blk-availability init script
Tony Asleson (35):
lvmdbustest.py: Remove raid4 use
lvmdbus: Make bus name configurable
lvmdbusd: Use one thread to fetch state updates
lvmdbusd: Long running operations use own thread
lvmdbustest.py: Support concurrent test runs
lvmdbusd: _lv_create, simplify return path
lvmdbusd: Add cfg.load to pv remove
lvmdbusd: WS corrections
lvmdbusd: Remove un-used import
lvmdbustest.py: Reduce test client introspection calls
lvmdbusd: Bubble up invalid JSON
lvmdbusd: Extra report FD read on no data
lvmdbusd: Place Manager.UseLvmShell request on queue
lvmdbustest: Move std_err_print to testlib
lvmdbustest: Reduce dbus object churn
lvmdbusd: Remove debug JSON file
lvmdbusd: cmdhandler.py, fix imports
lvmdbusd: Fix source documentation
lvmdbusd: Remove extraneous finally
lvmdbusd: Only read whats buffered
lvmdbusd: Make lvm shell read more robust
lvmdbusd: Simplfy reading streams
lvmdbusd: WS fix
lvmdbusd: Remove debug log_error
lvmdbusd: Remove unused variable
lvmdbusd: Add --blackboxsize command line argument
lvmdbusd: Supress protected member access warning
lvmdbusd: Remove TODO on concurrent access to properties
lvmdbusd: Emit signal on Job completion
lvmdbustest.py: Re-enable test_vg_uuid_gen
lvmdbustest.py: Remove outdated TODOs
lvmdbustest.py: Remove redundant import
lvmdbustest.py: Rename env test variable
WHATS_NEW: New argument --blackboxsize
lvmdbusd: Only allow 0..N for --blackboxsize argument
Zdenek Kabelac (34):
raid: faster rmeta clearing
tests: test raid0_meta type
tests: for repair we need neewer version
lvconvert: repair accepts interval and background
cleanup: skip checking for just assigned string
conf: support zero for missing_stripe_filler
tests: update make targets
tests: avoid using polling
tests: do not set zero interval in aux lvm.conf
cleanup: add dots to debug messages
cleanup: avoid using double __ in extracted image name
cleanup: debug trace and indent change
tests: fix checking for pvmove LV
raid: implement transient status check
raid: lvconvert uses transient check for raid
mirror: preserve MIRRORED status for temporara image
cleanup: use display_lvname and msg cleanup
cleanup: make this error an internal error
cleanup: use lv_update_and_reload
cleanup: hide gcc warning
cleanup: single LVM_SYSTEM_DIR string
cleanup: add doc for raid status states
cleanup: deactivate has builtin check for active lv
cleanup: use predefined macro for tmp image name
cleanup: defines
cleanup: indent changes
debug: stack tracing corrections
debug: more stacktrace corrections
cleanup: add some dots and use display_lvname
cleanup: add missing WARNING into log_warn
cleanup: indent
gcc: quiet warning about unused function
makefiles: drop file source and indent
tests: ensure there is dbus config file
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=ad7fd775f2b7d2d8…
Commit: ad7fd775f2b7d2d8fa9fe9bbec5101d9ecda9a92
Parent: ea2eb2df97252a5dc0f7fd27018171162d1e07ce
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Wed Nov 30 14:58:29 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Wed Nov 30 15:59:06 2016 -0600
lvmdbustest.py: Rename env test variable
Use LVM_DBUSD_TEST_MODE env variable to customize what we test.
Default is the same where we try to test all combinations of all
modes. Renamed to make it consistent with the other env variables
that are used in the unit test.
---
test/dbus/lvmdbustest.py | 10 ++++++----
1 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/test/dbus/lvmdbustest.py b/test/dbus/lvmdbustest.py
index 2e3ec22..af4fef1 100755
--- a/test/dbus/lvmdbustest.py
+++ b/test/dbus/lvmdbustest.py
@@ -30,6 +30,12 @@ use_session = os.getenv('LVM_DBUSD_USE_SESSION', False)
# Only use the devices listed in the ENV variable
pv_device_list = os.getenv('LVM_DBUSD_PV_DEVICE_LIST', None)
+# Default is to test all modes
+# 0 == Only test fork & exec mode
+# 1 == Test both fork & exec & lvm shell mode (default)
+# Other == Test just lvm shell mode
+test_shell = os.getenv('LVM_DBUSD_TEST_MODE', 1)
+
# Empty options dictionary (EOD)
EOD = dbus.Dictionary({}, signature=dbus.Signature('sv'))
# Base interfaces on LV objects
@@ -1661,10 +1667,6 @@ class AggregateResults(object):
if __name__ == '__main__':
r = AggregateResults()
-
- # Default is to test all modes
- test_shell = os.getenv('LVM_DBUS_TEST_SHELL', 1)
-
mode = int(test_shell)
if mode == 0:
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=b75d1a591c2ad3c8…
Commit: b75d1a591c2ad3c85b9bc25139d2044f439e8cc7
Parent: bb845cab76b6e8003673f0f4c472fff24a6ef7f4
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Wed Nov 30 14:51:20 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Wed Nov 30 15:59:06 2016 -0600
lvmdbustest.py: Remove outdated TODOs
- We check that all properties match the introspection data. We
don't verify values for every property as only lvm knows what they
should be.
- We are testing vg.Move
---
test/dbus/lvmdbustest.py | 3 ---
1 files changed, 0 insertions(+), 3 deletions(-)
diff --git a/test/dbus/lvmdbustest.py b/test/dbus/lvmdbustest.py
index 217a641..11fa19f 100755
--- a/test/dbus/lvmdbustest.py
+++ b/test/dbus/lvmdbustest.py
@@ -528,7 +528,6 @@ class TestDbusService(unittest.TestCase):
if path:
lv = ClientProxy(self.bus, path, interfaces=proxy_interfaces)
- # TODO verify object properties
# We are quick enough now that we can get VolumeType changes from
# 'I' to 'i' between the time it takes to create a RAID and it returns
@@ -885,8 +884,6 @@ class TestDbusService(unittest.TestCase):
self.assertEqual(rc, '/')
self._check_consistency()
- # Test Vg.Move
- # TODO Test this more!
vg.update()
lv.update()
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=53ef4967192ce8bf…
Commit: 53ef4967192ce8bfe5eaebf8104a1f7cbf8ab020
Parent: 1d520909534e1a4e1adf2513a5061adfe631d6af
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Wed Nov 30 13:39:48 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Wed Nov 30 15:59:06 2016 -0600
lvmdbusd: Emit signal on Job completion
Added a properties changed signal on the job dbus object so that client
can wait for a signal that the job is complete instead of polling or
blocking on the wait method.
---
daemons/lvmdbusd/job.py | 8 +++++++-
daemons/lvmdbusd/request.py | 1 -
daemons/lvmdbusd/utils.py | 5 +++++
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/daemons/lvmdbusd/job.py b/daemons/lvmdbusd/job.py
index ec6d6b6..609b747 100644
--- a/daemons/lvmdbusd/job.py
+++ b/daemons/lvmdbusd/job.py
@@ -8,7 +8,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from .automatedproperties import AutomatedProperties
-from .utils import job_obj_path_generate, mt_async_result
+from .utils import job_obj_path_generate, mt_async_result, mt_run_no_wait
from . import cfg
from .cfg import JOB_INTERFACE
import dbus
@@ -180,9 +180,15 @@ class Job(AutomatedProperties):
def Complete(self):
return dbus.Boolean(self.state.Complete)
+ @staticmethod
+ def _signal_complete(obj):
+ obj.PropertiesChanged(
+ JOB_INTERFACE, dict(Complete=dbus.Boolean(obj.state.Complete)), [])
+
@Complete.setter
def Complete(self, value):
self.state.Complete = value
+ mt_run_no_wait(Job._signal_complete, self)
@property
def GetError(self):
diff --git a/daemons/lvmdbusd/request.py b/daemons/lvmdbusd/request.py
index ca45e8c..a2c2ac9 100644
--- a/daemons/lvmdbusd/request.py
+++ b/daemons/lvmdbusd/request.py
@@ -126,7 +126,6 @@ class RequestEntry(object):
mt_async_result(self.cb_error, error_exception)
else:
# We have a job and it's complete, indicate that it's done.
- # TODO: We need to signal the job is done too.
self._job.Complete = True
self._job = None
diff --git a/daemons/lvmdbusd/utils.py b/daemons/lvmdbusd/utils.py
index abee659..b95f80a 100644
--- a/daemons/lvmdbusd/utils.py
+++ b/daemons/lvmdbusd/utils.py
@@ -515,6 +515,11 @@ def mt_async_result(call_back, results):
GLib.idle_add(_async_result, call_back, results)
+# Take the supplied function and run it on the main thread and not wait for
+# a result!
+def mt_run_no_wait(function, param):
+ GLib.idle_add(function, param)
+
# Run the supplied function and arguments on the main thread and wait for them
# to complete while allowing the ability to get the return value too.
#
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=1d520909534e1a4e…
Commit: 1d520909534e1a4e1adf2513a5061adfe631d6af
Parent: 37f05ccab1a576bdd71be1f4f80ab6f621042d71
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Wed Nov 30 13:03:25 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Wed Nov 30 15:59:06 2016 -0600
lvmdbusd: Remove TODO on concurrent access to properties
As the code now uses a single thread to handle all changes to the
dbus model we no longer need to handle this potential race
condition.
---
daemons/lvmdbusd/automatedproperties.py | 5 +----
1 files changed, 1 insertions(+), 4 deletions(-)
diff --git a/daemons/lvmdbusd/automatedproperties.py b/daemons/lvmdbusd/automatedproperties.py
index 5e14d61..7b22d19 100644
--- a/daemons/lvmdbusd/automatedproperties.py
+++ b/daemons/lvmdbusd/automatedproperties.py
@@ -159,10 +159,7 @@ class AutomatedProperties(dbus.service.Object):
cfg.om.lookup_update(self, new_id[0], new_id[1])
# Grab the properties values, then replace the state of the object
- # and retrieve the new values
- # TODO: We need to add locking to prevent concurrent access to the
- # properties so that a client is not accessing while we are
- # replacing.
+ # and retrieve the new values.
o_prop = get_properties(self)
self.state = new_state
n_prop = get_properties(self)
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=37f05ccab1a576bd…
Commit: 37f05ccab1a576bdd71be1f4f80ab6f621042d71
Parent: 3bc69cb23c29e8e016282508514ce191c756b89c
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Wed Nov 30 12:59:46 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Wed Nov 30 15:59:06 2016 -0600
lvmdbusd: Supress protected member access warning
We want _run to be protected so that users outside of the class
don't mistakenly use it. It's for internal use only.
---
daemons/lvmdbusd/utils.py | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/daemons/lvmdbusd/utils.py b/daemons/lvmdbusd/utils.py
index 561b158..abee659 100644
--- a/daemons/lvmdbusd/utils.py
+++ b/daemons/lvmdbusd/utils.py
@@ -525,6 +525,7 @@ class MThreadRunner(object):
@staticmethod
def runner(obj):
+ # noinspection PyProtectedMember
obj._run()
with obj.cond:
obj.function_complete = True
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=3bc69cb23c29e8e0…
Commit: 3bc69cb23c29e8e016282508514ce191c756b89c
Parent: b0757ac96e5c0f50e8d088200eb1ad62e535fcb6
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Tue Nov 29 18:01:56 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Wed Nov 30 15:59:06 2016 -0600
lvmdbusd: Add --blackboxsize command line argument
Allows the user to override the number of commands that get dumped
to the log when we encounter a lvm error. Also useful during
development when you don't want to see the blackbox output.
---
daemons/lvmdbusd/cmdhandler.py | 13 +++++++------
daemons/lvmdbusd/main.py | 16 ++++++++++++++--
2 files changed, 21 insertions(+), 8 deletions(-)
diff --git a/daemons/lvmdbusd/cmdhandler.py b/daemons/lvmdbusd/cmdhandler.py
index f221f93..f1591aa 100644
--- a/daemons/lvmdbusd/cmdhandler.py
+++ b/daemons/lvmdbusd/cmdhandler.py
@@ -55,18 +55,19 @@ class LvmExecutionMeta(object):
class LvmFlightRecorder(object):
- def __init__(self):
- self.queue = collections.deque(maxlen=16)
+ def __init__(self, size=16):
+ self.queue = collections.deque(maxlen=size)
def add(self, lvm_exec_meta):
self.queue.append(lvm_exec_meta)
def dump(self):
with cmd_lock:
- log_error("LVM dbus flight recorder START")
- for c in self.queue:
- log_error(str(c))
- log_error("LVM dbus flight recorder END")
+ if len(self.queue):
+ log_error("LVM dbus flight recorder START")
+ for c in self.queue:
+ log_error(str(c))
+ log_error("LVM dbus flight recorder END")
cfg.blackbox = LvmFlightRecorder()
diff --git a/daemons/lvmdbusd/main.py b/daemons/lvmdbusd/main.py
index 4b9f94e..9bf0871 100644
--- a/daemons/lvmdbusd/main.py
+++ b/daemons/lvmdbusd/main.py
@@ -29,6 +29,7 @@ from .utils import log_debug, log_error
import argparse
import os
import sys
+from .cmdhandler import LvmFlightRecorder
class Lvm(objectmanager.ObjectManager):
@@ -75,6 +76,12 @@ def main():
help="Use the lvm shell, not fork & exec lvm",
default=False,
dest='use_lvm_shell')
+ parser.add_argument(
+ "--blackboxsize",
+ help="Size of the black box flight recorder, 0 to disable",
+ default=10,
+ type=int,
+ dest='bb_size')
use_session = os.getenv('LVMDBUSD_USE_SESSION', False)
@@ -83,6 +90,11 @@ def main():
cfg.args = parser.parse_args()
+ # We create a flight recorder in cmdhandler too, but we replace it here
+ # as the user may be specifying a different size. The default one in
+ # cmdhandler is for when we are running other code with a different main.
+ cfg.blackbox = LvmFlightRecorder(cfg.args.bb_size)
+
if cfg.args.use_lvm_shell and not cfg.args.use_json:
log_error("You cannot specify --lvmshell and --nojson")
sys.exit(1)
@@ -118,8 +130,8 @@ def main():
# thread that is handling the dbus interface
thread_list.append(threading.Thread(target=process_request))
- # Have a single thread handling updating lvm and the dbus model so we don't
- # have multiple threads doing this as the same time
+ # Have a single thread handling updating lvm and the dbus model so we
+ # don't have multiple threads doing this as the same time
updater = StateUpdate()
thread_list.append(updater.thread)
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=0b8bf73a63d8dbd9…
Commit: 0b8bf73a63d8dbd9fa32a32c7d47a277d4fb8eb1
Parent: 58f4d98af11eaf21df00aaaf987ca6a92039db7f
Author: Heinz Mauelshagen <heinzm(a)redhat.com>
AuthorDate: Wed Nov 30 22:56:37 2016 +0100
Committer: Heinz Mauelshagen <heinzm(a)redhat.com>
CommitterDate: Wed Nov 30 22:57:54 2016 +0100
lvchange: allow a transiently failed RaidLV to be refreshed
In case any SubLV of a RaidLV transiently fails, it needs
two "lvchange --refresh RaidLV" runs to get it to fully
operational mode again. Reason being, that lvm reloads all
targets for the RaidLV tree but doesn't resume the SubLVs
until after the whole tree has been reloaded in the first
refresh run. Thus the live mapping table of the SubLVs
still point to an "error" mapping and the dm-raid target
can't retrieve any superblock from the MetaLV(s) in processing
the constructor during this preload thus not discovering the
again accessible SubLVs. In the second run, the SubLV targets
map proper (meta)data, hence the constructor discovers those
fine now.
Solve by resuming the SubLVs of the RaidLV before
preloading the respective top-level RaidLV target.
Resolves: rhbz1399844
---
lib/metadata/lv_manip.c | 29 ++++++++++++++++++++++++++++-
1 files changed, 28 insertions(+), 1 deletions(-)
diff --git a/lib/metadata/lv_manip.c b/lib/metadata/lv_manip.c
index 10c0446..120217f 100644
--- a/lib/metadata/lv_manip.c
+++ b/lib/metadata/lv_manip.c
@@ -1382,7 +1382,7 @@ int replace_lv_with_error_segment(struct logical_volume *lv)
return 1;
}
-int lv_refresh_suspend_resume(const struct logical_volume *lv)
+static int _lv_refresh_suspend_resume(const struct logical_volume *lv)
{
struct cmd_context *cmd = lv->vg->cmd;
int r = 1;
@@ -1407,6 +1407,33 @@ int lv_refresh_suspend_resume(const struct logical_volume *lv)
return r;
}
+int lv_refresh_suspend_resume(const struct logical_volume *lv)
+{
+ /*
+ * FIXME:
+ *
+ * in case of RAID, refresh the SubLVs before
+ * refreshing the top-level one in order to cope
+ * with transient failures of SubLVs.
+ */
+ if (lv_is_raid(lv)) {
+ uint32_t s;
+ struct lv_segment *seg = first_seg(lv);
+
+ for (s = 0; s < seg->area_count; s++) {
+ if (seg_type(seg, s) == AREA_LV &&
+ !_lv_refresh_suspend_resume(seg_lv(seg, s)))
+ return 0;
+ if (seg->meta_areas &&
+ seg_metatype(seg, s) == AREA_LV &&
+ !_lv_refresh_suspend_resume(seg_metalv(seg, s)))
+ return 0;
+ }
+ }
+
+ return _lv_refresh_suspend_resume(lv);
+}
+
/*
* Remove given number of extents from LV.
*/
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=d882edb32d8695ff…
Commit: d882edb32d8695ffb1498759649af93f407f39db
Parent: 25b5413f895bc61357ef4afab255e3125335d4ec
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Tue Nov 29 12:37:59 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Tue Nov 29 16:50:30 2016 -0600
lvmdbusd: Simplfy reading streams
Remove redundant code and make code paths the same for all streams.
---
daemons/lvmdbusd/lvm_shell_proxy.py | 52 +++++++++++++++++------------------
1 files changed, 25 insertions(+), 27 deletions(-)
diff --git a/daemons/lvmdbusd/lvm_shell_proxy.py b/daemons/lvmdbusd/lvm_shell_proxy.py
index 464da79..ae8049f 100755
--- a/daemons/lvmdbusd/lvm_shell_proxy.py
+++ b/daemons/lvmdbusd/lvm_shell_proxy.py
@@ -43,6 +43,13 @@ def _quote_arg(arg):
class LVMShellProxy(object):
+ @staticmethod
+ def _read(stream):
+ tmp = stream.read()
+ if tmp:
+ return tmp.decode("utf-8")
+ return ''
+
# Read until we get prompt back and a result
# @param: no_output Caller expects no output to report FD
# Returns stdout, report, stderr (report is JSON!)
@@ -62,29 +69,17 @@ class LVMShellProxy(object):
try:
rd_fd = [
self.lvm_shell.stdout.fileno(),
- self.report_r,
+ self.report_stream.fileno(),
self.lvm_shell.stderr.fileno()]
ready = select.select(rd_fd, [], [], 2)
for r in ready[0]:
if r == self.lvm_shell.stdout.fileno():
- tmp = self.lvm_shell.stdout.read()
- if tmp:
- stdout += tmp.decode("utf-8")
- elif r == self.report_r:
- while True:
- tmp = os.read(self.report_r, 16384)
- if tmp:
- report += tmp.decode("utf-8")
- if len(tmp) != 16384:
- break
- else:
- break
-
+ stdout += LVMShellProxy._read(self.lvm_shell.stdout)
+ elif r == self.report_stream.fileno():
+ report += LVMShellProxy._read(self.report_stream)
elif r == self.lvm_shell.stderr.fileno():
- tmp = self.lvm_shell.stderr.read()
- if tmp:
- stderr += tmp.decode("utf-8")
+ stderr += LVMShellProxy._read(self.lvm_shell.stderr)
# Check to see if the lvm process died on us
if self.lvm_shell.poll():
@@ -131,6 +126,11 @@ class LVMShellProxy(object):
assert (num_written == len(cmd_bytes))
self.lvm_shell.stdin.flush()
+ @staticmethod
+ def _make_non_block(stream):
+ flags = fcntl(stream, F_GETFL)
+ fcntl(stream, F_SETFL, flags | os.O_NONBLOCK)
+
def __init__(self):
# Create a temp directory
@@ -143,7 +143,10 @@ class LVMShellProxy(object):
except FileExistsError:
pass
- self.report_r = os.open(tmp_file, os.O_NONBLOCK)
+ # We have to open non-blocking as the other side isn't open until
+ # we actually fork the process.
+ self.report_fd = os.open(tmp_file, os.O_NONBLOCK)
+ self.report_stream = os.fdopen(self.report_fd, 'rb', 0)
# Setup the environment for using our own socket for reporting
local_env = copy.deepcopy(os.environ)
@@ -154,9 +157,6 @@ class LVMShellProxy(object):
# when utilizing the lvm shell.
local_env["LVM_LOG_FILE_MAX_LINES"] = "0"
- flags = fcntl(self.report_r, F_GETFL)
- fcntl(self.report_r, F_SETFL, flags | os.O_NONBLOCK)
-
# run the lvm shell
self.lvm_shell = subprocess.Popen(
[LVM_CMD + " 32>%s" % tmp_file],
@@ -164,10 +164,8 @@ class LVMShellProxy(object):
stderr=subprocess.PIPE, close_fds=True, shell=True)
try:
- flags = fcntl(self.lvm_shell.stdout, F_GETFL)
- fcntl(self.lvm_shell.stdout, F_SETFL, flags | os.O_NONBLOCK)
- flags = fcntl(self.lvm_shell.stderr, F_GETFL)
- fcntl(self.lvm_shell.stderr, F_SETFL, flags | os.O_NONBLOCK)
+ LVMShellProxy._make_non_block(self.lvm_shell.stdout)
+ LVMShellProxy._make_non_block(self.lvm_shell.stderr)
# wait for the first prompt
errors = self._read_until_prompt(no_output=True)[2]
@@ -176,8 +174,8 @@ class LVMShellProxy(object):
except:
raise
finally:
- # These will get deleted when the FD count goes to zero so we can be
- # sure to clean up correctly no matter how we finish
+ # These will get deleted when the FD count goes to zero so we
+ # can be sure to clean up correctly no matter how we finish
os.unlink(tmp_file)
os.rmdir(tmp_dir)
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=25b5413f895bc613…
Commit: 25b5413f895bc61357ef4afab255e3125335d4ec
Parent: b0bda090054db75995ea350f55b59ae8444690da
Author: Tony Asleson <tasleson(a)redhat.com>
AuthorDate: Tue Nov 29 11:07:21 2016 -0600
Committer: Tony Asleson <tasleson(a)redhat.com>
CommitterDate: Tue Nov 29 16:50:30 2016 -0600
lvmdbusd: Make lvm shell read more robust
Make sure JSON is correct before we stop trying to read.
---
daemons/lvmdbusd/lvm_shell_proxy.py | 98 ++++++++++++++++++-----------------
1 files changed, 50 insertions(+), 48 deletions(-)
diff --git a/daemons/lvmdbusd/lvm_shell_proxy.py b/daemons/lvmdbusd/lvm_shell_proxy.py
index 50f2201..464da79 100755
--- a/daemons/lvmdbusd/lvm_shell_proxy.py
+++ b/daemons/lvmdbusd/lvm_shell_proxy.py
@@ -42,18 +42,22 @@ def _quote_arg(arg):
class LVMShellProxy(object):
+
+ # Read until we get prompt back and a result
+ # @param: no_output Caller expects no output to report FD
+ # Returns stdout, report, stderr (report is JSON!)
def _read_until_prompt(self, no_output=False):
stdout = ""
report = ""
stderr = ""
keep_reading = True
- extra_passes = 2
+ extra_passes = 3
+ report_json = {}
+ prev_report_len = 0
# Try reading from all FDs to prevent one from filling up and causing
- # a hang. We were assuming that we won't get the lvm prompt back
- # until we have already received all the output from stderr and the
- # report descriptor too, this is an incorrect assumption. Lvm will
- # return the prompt before we get the report!
+ # a hang. Keep reading until we get the prompt back and the report
+ # FD does not contain valid JSON
while keep_reading:
try:
rd_fd = [
@@ -87,26 +91,39 @@ class LVMShellProxy(object):
raise Exception(self.lvm_shell.returncode, "%s" % stderr)
if stdout.endswith(SHELL_PROMPT):
- # It appears that lvm doesn't write the report and flush
- # that before it writes the shell prompt as occasionally
- # we get the prompt with no report.
if no_output:
keep_reading = False
else:
- # Most of the time we have data, if we have none lets
- # take another spin and hope we get it.
- if len(report) != 0:
- keep_reading = False
+ cur_report_len = len(report)
+ if cur_report_len != 0:
+ # Only bother to parse if we have more data
+ if prev_report_len != cur_report_len:
+ prev_report_len = cur_report_len
+ # Parse the JSON if it's good we are done,
+ # if not we will try to read some more.
+ try:
+ report_json = json.loads(report)
+ keep_reading = False
+ except ValueError:
+ pass
else:
+ log_error("RACE!", 'bg_black', 'fg_light_red')
+
+ if keep_reading:
extra_passes -= 1
if extra_passes <= 0:
- keep_reading = False
+ if len(report):
+ raise ValueError("Invalid json: %s" %
+ report)
+ else:
+ raise ValueError(
+ "lvm returned no JSON output!")
except IOError as ioe:
log_debug(str(ioe))
pass
- return stdout, report, stderr
+ return stdout, report_json, stderr
def _write_cmd(self, cmd):
cmd_bytes = bytes(cmd, "utf-8")
@@ -169,33 +186,24 @@ class LVMShellProxy(object):
self._write_cmd('lastlog\n')
# read everything from the STDOUT to the next prompt
- stdout, report, stderr = self._read_until_prompt()
-
- try:
- log = json.loads(report)
-
- if 'log' in log:
- error_msg = ""
- # Walk the entire log array and build an error string
- for log_entry in log['log']:
- if log_entry['log_type'] == "error":
- if error_msg:
- error_msg += ', ' + log_entry['log_message']
- else:
- error_msg = log_entry['log_message']
+ stdout, report_json, stderr = self._read_until_prompt()
+ if 'log' in report_json:
+ error_msg = ""
+ # Walk the entire log array and build an error string
+ for log_entry in report_json['log']:
+ if log_entry['log_type'] == "error":
+ if error_msg:
+ error_msg += ', ' + log_entry['log_message']
+ else:
+ error_msg = log_entry['log_message']
- return error_msg
+ return error_msg
- return 'No error reason provided! (missing "log" section)'
- except ValueError:
- log_error("Invalid JSON returned from LVM")
- log_error("BEGIN>>\n%s\n<<END" % report)
- return "Invalid JSON returned from LVM when retrieving exit code"
+ return 'No error reason provided! (missing "log" section)'
def call_lvm(self, argv, debug=False):
rc = 1
error_msg = ""
- json_result = ""
if self.lvm_shell.poll():
raise Exception(
@@ -210,27 +218,21 @@ class LVMShellProxy(object):
self._write_cmd(cmd)
# read everything from the STDOUT to the next prompt
- stdout, report, stderr = self._read_until_prompt()
+ stdout, report_json, stderr = self._read_until_prompt()
# Parse the report to see what happened
- if report and len(report):
- try:
- json_result = json.loads(report)
- if 'log' in json_result:
- if json_result['log'][-1:][0]['log_ret_code'] == '1':
- rc = 0
- else:
- error_msg = self.get_error_msg()
- except ValueError:
- # Bubble up the invalid json.
- error_msg = "Invalid json %s" % report
+ if 'log' in report_json:
+ if report_json['log'][-1:][0]['log_ret_code'] == '1':
+ rc = 0
+ else:
+ error_msg = self.get_error_msg()
if debug or rc != 0:
log_error(('CMD: %s' % cmd))
log_error(("EC = %d" % rc))
log_error(("ERROR_MSG=\n %s\n" % error_msg))
- return rc, json_result, error_msg
+ return rc, report_json, error_msg
def exit_shell(self):
try:
Gitweb: http://git.fedorahosted.org/git/?p=lvm2.git;a=commitdiff;h=b7cf7b1d3a817498…
Commit: b7cf7b1d3a81749811bc33e1c0840e8f608af991
Parent: 108d9a63fde1608aff815359d58da641fc9a18af
Author: Zdenek Kabelac <zkabelac(a)redhat.com>
AuthorDate: Fri Nov 25 15:02:36 2016 +0100
Committer: Zdenek Kabelac <zkabelac(a)redhat.com>
CommitterDate: Fri Nov 25 15:02:36 2016 +0100
gcc: quiet warning about unused function
Once this function will need to be used, git revert.
---
lib/metadata/raid_manip.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/lib/metadata/raid_manip.c b/lib/metadata/raid_manip.c
index 9c8ba82..7e591e9 100644
--- a/lib/metadata/raid_manip.c
+++ b/lib/metadata/raid_manip.c
@@ -34,6 +34,7 @@ static int _check_restriping(uint32_t new_stripes, struct logical_volume *lv)
return 1;
}
+__attribute__ ((__unused__))
/* Check that all lv has segments have exactly the required number of areas */
static int _check_num_areas_in_lv_segments(struct logical_volume *lv, unsigned num_areas)
{