main - cov: drop unneeded constness
by Zdenek Kabelac
Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=4d648380484ecf58c33...
Commit: 4d648380484ecf58c33f92a84950bb0bf39424ef
Parent: 17ee5df8576acf628d88fd6840ce55aa1388d6d1
Author: Zdenek Kabelac <zkabelac(a)redhat.com>
AuthorDate: Thu May 11 19:17:18 2023 +0200
Committer: Zdenek Kabelac <zkabelac(a)redhat.com>
CommitterDate: Fri Jun 9 18:01:16 2023 +0200
cov: drop unneeded constness
If we want to have const structure - use a const pointer to avoid,
however making individual members const make it only assingle
in construct time and we already violate this logic since
we memcpy into the structure - so drop these unnecessary consts...
---
daemons/lvmpolld/lvmpolld-data-utils.h | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/daemons/lvmpolld/lvmpolld-data-utils.h b/daemons/lvmpolld/lvmpolld-data-utils.h
index 96a27f1c8..eeae4137c 100644
--- a/daemons/lvmpolld/lvmpolld-data-utils.h
+++ b/daemons/lvmpolld/lvmpolld-data-utils.h
@@ -45,18 +45,18 @@ struct lvmpolld_lv {
* accessing following vars doesn't
* require struct lvmpolld_lv lock
*/
- struct lvmpolld_state *const ls;
+ struct lvmpolld_state *ls;
const enum poll_type type;
- const char *const lvid;
- const char *const lvmpolld_id;
- const char *const devicesfile;
- const char *const lvname; /* full vg/lv name */
+ const char *lvid;
+ const char *lvmpolld_id;
+ const char *devicesfile;
+ const char *lvname; /* full vg/lv name */
const unsigned pdtimeout; /* in seconds */
- const char *const sinterval;
- const char *const lvm_system_dir_env;
- struct lvmpolld_store *const pdst;
- const char *const *cmdargv;
- const char *const *cmdenvp;
+ const char *sinterval;
+ const char *lvm_system_dir_env;
+ struct lvmpolld_store *pdst;
+ const char **cmdargv;
+ const char **cmdenvp;
/* only used by write */
pid_t cmd_pid;
11 months, 2 weeks
main - vdo: man updates for lvm_import_vdo
by Zdenek Kabelac
Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=17ee5df8576acf628d8...
Commit: 17ee5df8576acf628d88fd6840ce55aa1388d6d1
Parent: be6c34212b05d8c59c0ba68e26c56cddf61c9f04
Author: Zdenek Kabelac <zkabelac(a)redhat.com>
AuthorDate: Fri Jun 9 13:24:19 2023 +0200
Committer: Zdenek Kabelac <zkabelac(a)redhat.com>
CommitterDate: Fri Jun 9 18:01:16 2023 +0200
vdo: man updates for lvm_import_vdo
---
man/lvm_import_vdo.8_main | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/man/lvm_import_vdo.8_main b/man/lvm_import_vdo.8_main
index 23755a3aa..20be6847d 100644
--- a/man/lvm_import_vdo.8_main
+++ b/man/lvm_import_vdo.8_main
@@ -61,6 +61,20 @@ Answer "yes" at any prompts.
.BR --dry-run
Print verbosely commands without running them.
.
+.TP
+.BR --no-snapshot
+With this option conversion tool will not use snapshot (COW storage) for conversion
+in your $TMPDIR filesystem and it will directly manipulate with VDO device in-place.
+Warning: the snapshot usage makes the whole conversion transactional and
+the snapshot can be just simply merged once the whole conversion
+has successfully proceeded. In the case of error the snapshot is just removed.
+Without the use of snapshot there is higher risk of problems with recoverability in
+case some unexpected error occurs.
+.
+.TP
+.BR --vdo-config
+Configuration file for VDO manager. Can be used to specify configuration for vdo manager.
+.
.
.SH DIAGNOSTICS
.
@@ -83,6 +97,15 @@ The temporary directory name for mount points. Defaults to "\fI/tmp\fP".
.B DM_DEV_DIR
The device directory name.
Defaults to "\fI/dev\fP" and must be an absolute path.
+.TP
+.B DM_UUID_PREFIX
+Specify uuid prefix for snapshot volume used during vdo conversion.
+.TP
+.B LVM_BINARY
+Allow to overide command called from lvm. Defaults to "\fIlvm\fP".
+.TP
+.B VDO_BINARY
+Allow to overide command called from vdo. Defaults to "\fIvdo\fP".
.
.SH SEE ALSO
.
11 months, 2 weeks
main - vdo: enhance lvm_vdo_import
by Zdenek Kabelac
Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=be6c34212b05d8c59c0...
Commit: be6c34212b05d8c59c0ba68e26c56cddf61c9f04
Parent: 00633f8b668d94a58756d96f5927fd690b8f41d3
Author: Zdenek Kabelac <zkabelac(a)redhat.com>
AuthorDate: Thu May 11 20:07:37 2023 +0200
Committer: Zdenek Kabelac <zkabelac(a)redhat.com>
CommitterDate: Fri Jun 9 18:01:16 2023 +0200
vdo: enhance lvm_vdo_import
Add support for usage of 'dm-snapshot' for doing whole device conversion
in a snapshot which could be merged once the whole conversion has been
made.
This helps with cases where there would be any unexpected failure in the
middle of conversion process and user can continue using original
device until problem in conversion is fixed.
Import tool now uses 'truncate' tool to create a small backend file (20M) for loop device
to hold COW exception store.
Option --vdo-config has been added to allow specifing location of vdo
configuration file.
Option --no-snapshot allows to use import tool without creation of
snapshot, however recovery after finished VDO conversion and unfinished
lvm2 conversion is then very difficult.
Option --uuid-prefix allow to specify DM UUID prefix for snapshot
device.
Use read with -r.
---
scripts/lvm_import_vdo.sh | 375 ++++++++++++++++++++++++++++++++++------------
1 file changed, 282 insertions(+), 93 deletions(-)
diff --git a/scripts/lvm_import_vdo.sh b/scripts/lvm_import_vdo.sh
index 134c96513..bae4b52d1 100755
--- a/scripts/lvm_import_vdo.sh
+++ b/scripts/lvm_import_vdo.sh
@@ -1,6 +1,6 @@
#!/bin/bash
#
-# Copyright (C) 2021 Red Hat, Inc. All rights reserved.
+# Copyright (C) 2021-2023 Red Hat, Inc. All rights reserved.
#
# This file is part of LVM2.
#
@@ -19,7 +19,7 @@
# Needed utilities:
# lvm, dmsetup,
# vdo,
-# grep, awk, sed, blockdev, readlink, stat, mkdir
+# grep, awk, sed, blockdev, readlink, stat, mkdir, truncate
#
# Conversion is using 'vdo convert' support from VDO manager to move
# existing VDO header by 2M which makes space to place in PV header
@@ -29,6 +29,9 @@
set -euE -o pipefail
TOOL=lvm_import_vdo
+IMPORT_NAME="VDO_${TOOL}_${RANDOM}$$"
+test ${#IMPORT_NAME} -lt 100 || error "Random name \"$IMPORT_NAME\" is too long!"
+TEMPDIR="${TMPDIR:-/tmp}/$IMPORT_NAME"
_SAVEPATH=$PATH
PATH="/sbin:/usr/sbin:/bin:/usr/sbin:$PATH"
@@ -36,26 +39,40 @@ PATH="/sbin:/usr/sbin:/bin:/usr/sbin:$PATH"
# user may override lvm location by setting LVM_BINARY
LVM=${LVM_BINARY:-lvm}
VDO=${VDO_BINARY:-vdo}
-VDOCONF=${VDOCONF:-}
BLOCKDEV="blockdev"
+LOSETUP="losetup"
READLINK="readlink"
READLINK_E="-e"
STAT="stat"
MKDIR="mkdir"
+TRUNCATE="truncate"
DMSETUP="dmsetup"
-TEMPDIR="${TMPDIR:-/tmp}/${TOOL}_${RANDOM}$$"
DM_DEV_DIR="${DM_DEV_DIR:-/dev}"
-
-DEVICENAME=""
+DM_UUID_PREFIX="${DM_UUID_PREFIX:-}"
+DM_VG_NAME=
+DM_LV_NAME=
+VDO_CONFIG=${VDO_CONFIG:-} # can be overriden with --vdo-config
+VDOCONF=
+test -n "$VDO_CONFIG" && VDOCONF="-f $VDO_CONFIG"
+
+DEVICE=
+VGNAME=
+LVNAME=
DEVMAJOR=0
DEVMINOR=0
PROMPTING=""
+USE_VDO_DM_SNAPSHOT=1
+VDO_DM_SNAPSHOT_NAME=
+VDO_DM_SNAPSHOT_DEVICE=
+VDO_SNAPSHOT_LOOP=
DRY=0
VERB=""
FORCE=""
YES=""
+ABORT_AFTER_VDO_CONVERT=0
+VDO_ALLOCATION_PARAMS=
# default name for converted VG and its VDO LV
DEFAULT_NAME="vdovg/vdolvol"
@@ -74,6 +91,9 @@ tool_usage() {
echo " -v | --verbose Be verbose"
echo " -y | --yes Answer \"yes\" at any prompts"
echo " --dry-run Print verbosely commands without running them"
+ echo " --no-snapshot Do not use snapshot for converted VDO device"
+ echo " --uuid-prefix Prefix for DM snapshot uuid"
+ echo " --vdo-config Configuration file for VDO manager"
exit
}
@@ -102,12 +122,79 @@ dry() {
cleanup() {
trap '' 2
+ test -n "$VDO_DM_SNAPSHOT_NAME" && { "$DMSETUP" remove "$VDO_DM_SNAPSHOT_NAME" || true ; }
+ test -n "$VDO_SNAPSHOT_LOOP" && { "$LOSETUP" -d "$VDO_SNAPSHOT_LOOP" || true ; }
+
test -z "$PROMPTING" || echo "No"
- rm -rf "$TEMPDIR"
+ rm -rf "$TEMPDIR" || true
# error exit status for break
exit "${1:-1}"
}
+# Create snapshot target like for persistent snapshot with 16KiB chunksize
+snapshot_target_line_() {
+ echo "0 $("$BLOCKDEV" --getsize "$1") snapshot${3:-} $1 $2 P 32"
+}
+
+snapshot_create_() {
+ VDO_DM_SNAPSHOT_NAME="${IMPORT_NAME}_snap"
+ local file="$TEMPDIR/$VDO_DM_SNAPSHOT_NAME"
+
+ # TODO: maybe use ramdisk via 'brd' device ?)
+ "$TRUNCATE" -s 20M "$file"
+ VDO_SNAPSHOT_LOOP=$("$LOSETUP" -f --show "$file")
+ "$DMSETUP" create "$VDO_DM_SNAPSHOT_NAME" -u "${DM_UUID_PREFIX}-${VDO_DM_SNAPSHOT_NAME}-priv" --table "$(snapshot_target_line_ "$1" "$VDO_SNAPSHOT_LOOP")"
+ VDO_DM_SNAPSHOT_DEVICE="$DM_DEV_DIR/mapper/$VDO_DM_SNAPSHOT_NAME"
+ verbose "Snapshot of VDO device $1 created: $VDO_DM_SNAPSHOT_DEVICE."
+}
+
+snapshot_merge_() {
+ local status
+ local inital_status
+
+ initial_status=( $("$DMSETUP" status "$VDO_DM_SNAPSHOT_NAME") )
+ "$DMSETUP" reload "$VDO_DM_SNAPSHOT_NAME" --table "$(snapshot_target_line_ "$1" "$VDO_SNAPSHOT_LOOP" -merge)"
+ "$DMSETUP" suspend "$VDO_DM_SNAPSHOT_NAME" || {
+ error "ABORTING: Failed to initialize snapshot merge! Origin volume is unchanged."
+ }
+
+ verbose "Merging converted VDO volume..."
+ # Running merging
+ "$DMSETUP" resume "$VDO_DM_SNAPSHOT_NAME"
+
+ #du -h "$TEMPDIR/$VDO_DM_SNAPSHOT_NAME"
+
+ # Loop for a while, till the snapshot is merged.
+ # Should be nearly instantenious.
+ # FIXME: Recovery when something prevents merging is hard
+ for i in $(seq 1 20) ; do
+ status=( $("$DMSETUP" status "$VDO_DM_SNAPSHOT_NAME") )
+ # Check if merging is finished
+ test "${status[3]%/*}" = "${status[4]}" && break
+ # Wait a bit and retry
+ sleep .2
+ done
+ test "${status[3]%/*}" = "${status[4]}" || {
+ # FIXME: Now what shall we do ??? Help....
+ # Keep snapshot in table for possible analysis...
+ VDO_DM_SNAPSHOT_NAME=
+ VDO_SNAPSHOT_LOOP=
+ echo "Initial snapshot status ${initial_status[*]}"
+ echo "Failing merge snapshot status ${status[*]}"
+ error "ABORTING: Snapshot failed to merge! (Administrator required...)"
+ }
+ sync
+ "$DMSETUP" remove "$VDO_DM_SNAPSHOT_NAME" || {
+ sleep 1 # sleep and retry once more
+ "$DMSETUP" remove "$VDO_DM_SNAPSHOT_NAME" || {
+ error "ABORTING: Cannot remove snapshot $VDO_DM_SNAPSHOT_NAME! (check volume autoactivation...)"
+ }
+ }
+ VDO_DM_SNAPSHOT_NAME=
+ "$LOSETUP" -d "$VDO_SNAPSHOT_LOOP"
+ VDO_SNAPSHOT_LOOP=
+}
+
get_enabled_value_() {
case "$1" in
enabled) echo "1" ;;
@@ -221,46 +308,191 @@ parse_yaml_() {
) < "$yaml_file"
}
-# convert existing VDO volume into lvm2 volume
+#
+# Convert VDO volume on LV to VDOPool within this VG
+#
+# This conversion requires the size of VDO virtual volume has to be expressed in the VG's extent size.
+# Currently this enforces a user to reduce the VG extent size to the smaller size (up to 4KiB).
+#
+# TODO: We may eventually relax this condition just like we are doing rounding for convert_non_lv_()
+# Let's if there would be any singly user requiring this feauture.
+# It may allow to better use larger VDO volume size (in TiB ranges).
+#
+convert_lv_() {
+ local vdo_logicalSize=$1
+ local extent_size
+ local pvfree
+
+ pvfree=$("$LVM" lvs -o size --units b --nosuffix --noheadings "$DM_VG_NAME/$DM_LV_NAME")
+ pvfree=$(( pvfree / 1024 )) # to KiB
+ # select largest possible extent size that can exactly express both sizes
+ extent_size=$(get_largest_extent_size_ "$pvfree" "$vdo_logicalSize")
+
+ # validate existing VG extent_size can express virtual VDO size
+ vg_extent_size=$("$LVM" vgs -o vg_extent_size --units b --nosuffix --noheadings "$VGNAME")
+ vg_extent_size=$(( vg_extent_size / 1024 ))
+
+ test "$vg_extent_size" -le "$extent_size" || {
+ error "Please vgchange extent_size to at most $extent_size KiB or extend and align virtual size of VDO device on $vg_extent_size KiB before retrying conversion."
+ }
+ verbose "Renaming existing LV to be used as _vdata volume for VDO pool LV."
+ dry "$LVM" lvrename $YES $VERB "$VGNAME/$DM_LV_NAME" "$VGNAME/${LVNAME}_vpool" || {
+ error "Rename of LV \"$VGNAME/$DM_LV_NAME\" failed, while VDO header has been already moved!"
+ }
+
+ verbose "Converting to VDO pool."
+ dry "$LVM" lvconvert $YES $VERB $FORCE --config "$VDO_ALLOCATION_PARAMS" -Zn -V "${vdo_logicalSize}k" -n "$LVNAME" --type vdo-pool "$VGNAME/${LVNAME}_vpool"
+
+ verbose "Removing now unused VDO entry from VDO configuration."
+ dry "$VDO" remove $VDOCONF $VERB --force --name "$VDONAME"
+}
+
+#
+# Convert VDO volume on a device to VG with VDOPool LV
+#
+# Convert device with the use of snapshot on top of original VDO volume (can be optionally disabled)
+# Once the whole conversion is finished, snapshot is merged (During the short periof time of merging
+# user must ensure there will be no power-off!)
+#
+# For best use the latest version of vdoprepareforlvm tool is required.
+convert_non_lv_() {
+ local vdo_logicalSize=$1
+ local extent_size
+ local output
+ local pvfree
+
+ if [ "$USE_VDO_DM_SNAPSHOT" = "1" ]; then
+ dry snapshot_create_ "$DEVICE"
+ sed "s:$DEVICE:$VDO_DM_SNAPSHOT_DEVICE:" "$TEMPDIR/vdoconf.yml" > "$TEMPDIR/vdo_snap.yml"
+ # Let VDO manager operate on snapshot volume
+ VDOCONF="-f $TEMPDIR/vdo_snap.yml"
+ fi
+
+ verbose "Moving VDO header."
+ output=$(dry "$VDO" convert $VDOCONF --force --name "$VDONAME")
+
+ if [ "$ABORT_AFTER_VDO_CONVERT" != "0" ] ; then
+ verbose "Aborting VDO coversion after moving VDO, volume is useless!"
+ cleanup 0
+ fi
+
+ # Parse result from VDO preparation/conversion tool
+ # New version of the tool provides output with alingment and offset
+ local vdo_length=0
+ local vdo_aligned=0
+ local vdo_offset=0
+ local vdo_non_converted=0
+ while IFS= read -r line ; do
+ case "$line" in
+ "Non converted"*) vdo_non_converted=1 ;;
+ "Length"*) vdo_length=${line##* = } ;;
+ "Conversion completed"*)
+ vdo_aligned=${line##*aligned on }
+ vdo_aligned=${vdo_aligned%%[!0-9]*}
+ vdo_offset=${line##*offset }
+ # backward compatibility with report from older version
+ vdo_offset=${vdo_offset##*by }
+ vdo_offset=${vdo_offset%%[!0-9]*}
+ ;;
+ esac
+ done <<< "$output"
+
+ # In case we operation with snapshot, all lvm2 operation will also run on top of snapshot
+ local devices=${VDO_DM_SNAPSHOT_DEVICE:-$DEVICE}
+
+ dry "$LVM" pvcreate $YES --devices "$devices" --dataalignment "$vdo_offset"b "$devices" || {
+ error "Creation of PV on \"$DEVICE\" failed, while VDO header has been already moved!"
+ }
+
+ # Obtain free space in this new PV
+ # after 'vdo convert' call there is ~(1-2)M free space at the front of the device
+ pvfree=$("$BLOCKDEV" --getsize64 "$DEVICE")
+ pvfree=$(( ( pvfree - vdo_offset ) / 1024 )) # to KiB
+ if [ -n "$vdo_aligned" ] && [ "$vdo_aligned" != "0" ]; then
+ extent_size=$(( vdo_aligned / 1024 ))
+ else
+ extent_size=$(get_largest_extent_size_ "$pvfree" "$vdo_logicalSize")
+ fi
+
+ # Round virtual size to the LOWER size expressed in extent units.
+ # lvm is parsing VDO metadata and can read real full size and use it instead of this smaller value.
+ # To precisely byte-synchronize the size of VDO LV, user can lvresize such VDO LV later.
+ vdo_logicalSize=$(( ( vdo_logicalSize / extent_size ) * extent_size ))
+
+ verbose "Creating VG \"${NAME%/*}\" with extent size $extent_size KiB."
+ dry "$LVM" vgcreate $YES $VERB --devices "$devices" -s "${extent_size}k" "$VGNAME" "$devices" || {
+ error "Creation of VG \"$VGNAME\" failed, while VDO header has been already moved!"
+ }
+
+ verbose "Creating VDO pool data LV from all extents in volume group $VGNAME."
+ dry "$LVM" lvcreate -Zn -Wn -an $YES $VERB --devices "$devices" -l100%VG -n "${LVNAME}_vpool" "$VGNAME" "$devices"
+
+ verbose "Converting to VDO pool."
+ dry "$LVM" lvconvert $YES $VERB $FORCE --devices "$devices" --config "$VDO_ALLOCATION_PARAMS" -Zn -V "${vdo_logicalSize}k" -n "$LVNAME" --type vdo-pool "$VGNAME/${LVNAME}_vpool"
+
+ dry "$LVM" vgchange -an $VERB $FORCE --devices "$devices" "$VGNAME"
+
+ if [ "$USE_VDO_DM_SNAPSHOT" = "1" ]; then
+ if [ -z "$YES" ]; then
+ PROMPTING=yes
+ echo "Warning: Do not interrupt merging process once it starts (VDO data may become irrecoverable)!"
+ echo -n "Do you want to merge converted VDO device \"$DEVICE\" to VDO LV \"$VGNAME/$LVNAME\"? [y|N]: "
+ read -r -n 1 -s ANSWER
+ case "${ANSWER:0:1}" in
+ y|Y ) echo "Yes" ;;
+ * ) echo "No" ; PROMPTING=""; cleanup 1 ;;
+ esac
+ PROMPTING=""
+ YES="-y" # From now, now prompting
+ fi
+
+ dry snapshot_merge_ "$DEVICE"
+ if [ -e "$TEMPDIR/vdo_snap.yml" ]; then
+ dry cp "$TEMPDIR/vdo_snap.yml" "$VDO_CONFIG"
+ else
+ dry rm -f "$VDO_CONFIG"
+ fi
+ verbose "Merging of VDO device finished."
+ fi
+
+ dry "$LVM" lvchange -ay $VERB $FORCE "$VGNAME/$LVNAME"
+}
+
+# Convert existing VDO volume into lvm2 volume
convert2lvm_() {
- local DEVICE=$1
- local VGNAME=${NAME%/*}
- local LVNAME=${NAME#*/}
local VDONAME
local TRVDONAME
- local EXTENTSZ
- local IS_LV=1
local FOUND=""
local MAJOR=0
local MINOR=0
- local DM_VG_NAME
- local DM_LV_NAME
+ VGNAME=${NAME%/*}
+ LVNAME=${NAME#*/}
DM_UUID=""
detect_lv_ "$DEVICE"
case "$DM_UUID" in
LVM-*) eval "$("$DMSETUP" splitname --nameprefixes --noheadings --separator ' ' "$DM_NAME")"
if [ -z "$VGNAME" ] || [ "$VGNAME" = "$LVNAME" ] ; then
VGNAME=$DM_VG_NAME
- verbose "Using existing volume group name $VGNAME."
+ verbose "Using existing volume group name \"$VGNAME\"."
test -n "$LVNAME" || LVNAME=$DM_LV_NAME
- elif test "$VGNAME" != "$DM_VG_NAME" ; then
+ elif [ "$VGNAME" != "$DM_VG_NAME" ]; then
error "Volume group name \"$VGNAME\" does not match name \"$DM_VG_NAME\" for VDO device \"$DEVICE\"."
fi
;;
- *) IS_LV=0
+ *)
# Check if we need to generate unused $VGNANE
if [ -z "$VGNAME" ] || [ "$VGNAME" = "$LVNAME" ] ; then
VGNAME=${DEFAULT_NAME%/*}
# Find largest matching VG name to our 'default' vgname
- LASTVGNAME=$(LC_ALL=C "$LVM" vgs -oname -O-name --noheadings -S name=~${VGNAME} | grep -E "${VGNAME}[0-9]? ?" | head -1 || true)
- if test -n "$LASTVGNAME" ; then
+ LASTVGNAME=$(LC_ALL=C "$LVM" vgs -oname -O-name --noheadings -S name=~"${VGNAME}" | grep -E "${VGNAME}[0-9]? ?" | head -1 || true)
+ if [ -n "$LASTVGNAME" ]; then
LASTVGNAME=${LASTVGNAME#*"${VGNAME}"}
# If the number is becoming too high, try some random number
test "$LASTVGNAME" -gt 99999999 2>/dev/null && LASTVGNAME=$RANDOM
# Generate new unused VG name
VGNAME="${VGNAME}$(( LASTVGNAME + 1 ))"
- verbose "Selected unused volume group name $VGNAME."
+ verbose "Selected unused volume group name \"$VGNAME\"."
fi
fi
# New VG is created, LV name should be always unused.
@@ -269,13 +501,14 @@ convert2lvm_() {
;;
esac
- verbose "Checked whether device $1 is already LV ($IS_LV)."
+ verbose "Checked whether device \"$DEVICE\" is already logical volume."
"$MKDIR" -p -m 0000 "$TEMPDIR" || error "Failed to create $TEMPDIR."
# TODO: might use directly /etc/vdoconf.yml (avoding need of 'vdo' manager)
verbose "Getting YAML VDO configuration."
"$VDO" printConfigFile $VDOCONF >"$TEMPDIR/vdoconf.yml"
+ test -s "$TEMPDIR/vdoconf.yml" || error "Cannot work without VDO configuration"
# Check list of devices in VDO configure file for their major:minor
# and match with given $DEVICE devmajor:devminor
@@ -290,8 +523,8 @@ convert2lvm_() {
}
done
- test -n "$FOUND" || error "Can't find matching device in vdo configuration file."
- verbose "Found matching device $FOUND $MAJOR:$MINOR"
+ test -n "$FOUND" || error "Can't find matching device in VDO configuration file."
+ verbose "Found matching device $FOUND $MAJOR:$MINOR."
VDONAME=$(awk -v DNAME="$FOUND" '/.*VDOService$/ {VNAME=substr($1, 0, length($1) - 1)} /[[:space:]]*device:/ { if ($2 ~ DNAME) {print VNAME}}' "$TEMPDIR/vdoconf.yml")
TRVDONAME=$(echo "$VDONAME" | tr '-' '_')
@@ -313,7 +546,7 @@ convert2lvm_() {
verbose "Converted VDO device has logical/physical size $vdo_logicalSize/$vdo_physicalSize KiB."
- PARAMS=$(cat <<EOF
+ VDO_ALLOCATION_PARAMS=$(cat <<EOF
allocation {
vdo_use_compression = $(get_enabled_value_ "$vdo_compression")
vdo_use_deduplication = $(get_enabled_value_ "$vdo_deduplication")
@@ -338,80 +571,31 @@ allocation {
}
EOF
)
- verbose "VDO conversion paramaters: $PARAMS"
+ verbose "VDO conversion parameters: $VDO_ALLOCATION_PARAMS"
+
+ verbose "Stopping VDO volume."
+ dry "$VDO" stop $VDOCONF --name "$VDONAME"
# If user has not provided '--yes', prompt before conversion
- if test -z "$YES" ; then
+ if [ -z "$YES" ] && [ "$USE_VDO_DM_SNAPSHOT" != "1" ]; then
PROMPTING=yes
echo -n "Convert VDO device \"$DEVICE\" to VDO LV \"$VGNAME/$LVNAME\"? [y|N]: "
- read -n 1 -s ANSWER
+ read -r -n 1 -s ANSWER
case "${ANSWER:0:1}" in
- y|Y ) echo "Yes" ;;
- * ) echo "No" ; PROMPTING=""; exit 1 ;;
+ y|Y ) echo "Yes" ;;
+ * ) echo "No" ; PROMPTING=""; cleanup 1 ;;
esac
PROMPTING=""
- YES="-y" # From now, now prompting
+ YES="-y" # From now, no prompting
fi
- verbose "Stopping VDO volume."
- dry "$VDO" stop $VDOCONF --name "$VDONAME"
-
- if [ "$IS_LV" = "0" ]; then
- verbose "Moving VDO header by 2MiB."
- dry "$VDO" convert $VDOCONF --force --name "$VDONAME"
+ # Make a backup of the existing VDO yaml configuration file
+ test -e "$VDO_CONFIG" && dry cp -a "$VDO_CONFIG" "${VDO_CONFIG}.backup"
- dry "$LVM" pvcreate $YES --dataalignment 2M "$DEVICE" || {
- error "Creation of PV on \"$DEVICE\" failed, while VDO header has been already moved!"
- }
-
- # Obtain free space in this new PV
- # after 'vdo convert' call there is +2M free space at the front of the device
- case "$DRY" in
- 0) pvfree=$("$LVM" pvs -o devsize --units b --nosuffix --noheadings "$DEVICE") ;;
- *) pvfree=$("$BLOCKDEV" --getsize64 "$DEVICE") ;;
- esac
-
- pvfree=$(( pvfree / 1024 - 2048 )) # to KiB
- else
- pvfree=$("$LVM" lvs -o size --units b --nosuffix --noheadings "$DM_VG_NAME/$DM_LV_NAME")
- pvfree=$(( pvfree / 1024 )) # to KiB
- fi
-
- # select largest possible extent size that can exactly express both sizes
- EXTENTSZ=$(get_largest_extent_size_ "$pvfree" "$vdo_logicalSize")
-
- if [ "$IS_LV" = "0" ]; then
- verbose "Creating VG \"${NAME%/*}\" with extent size $EXTENTSZ KiB."
- dry "$LVM" vgcreate $YES $VERB -s "${EXTENTSZ}k" "$VGNAME" "$DEVICE" || {
- error "Creation of VG \"$VGNAME\" failed, while VDO header has been already moved!"
- }
-
- verbose "Creating VDO pool data LV from all extents in volume group $VGNAME."
- dry "$LVM" lvcreate -Zn -Wn $YES $VERB -l100%VG -n "${LVNAME}_vpool" "$VGNAME"
- else
- # validate existing VG extent_size can express virtual VDO size
- vg_extent_size=$("$LVM" vgs -o vg_extent_size --units b --nosuffix --noheadings "$VGNAME" || true)
- vg_extent_size=$(( vg_extent_size / 1024 ))
-
- test "$vg_extent_size" -le "$EXTENTSZ" || {
- error "Please vgchange extent_size to at most $EXTENTSZ KiB or extend and align virtual size of VDO device on $vg_extent_size KiB."
- }
- verbose "Renaming existing LV to be used as _vdata volume for VDO pool LV."
- dry "$LVM" lvrename $YES $VERB "$VGNAME/$DM_LV_NAME" "$VGNAME/${LVNAME}_vpool" || {
- error "Rename of LV \"$VGNAME/$DM_LV_NAME\" failed, while VDO header has been already moved!"
- }
- fi
-
- verbose "Converting to VDO pool."
- dry "$LVM" lvconvert $YES $VERB $FORCE --config "$PARAMS" -Zn -V "${vdo_logicalSize}k" -n "$LVNAME" --type vdo-pool "$VGNAME/${LVNAME}_vpool"
-
- # Note: that this is spelled OPPOSITE the other $IS_LV checks.
- if [ "$IS_LV" = "1" ]; then
- verbose "Removing now-unused VDO entry from VDO config."
- dry "$VDO" remove $VDOCONF $VERB --force --name "$VDONAME"
- fi
-
- rm -fr "$TEMPDIR"
+ case "$DM_UUID" in
+ LVM-*) convert_lv_ "$vdo_logicalSize" ;;
+ *) convert_non_lv_ "$vdo_logicalSize" ;;
+ esac
}
#############################
@@ -431,14 +615,19 @@ do
"-n"|"--name" ) shift; NAME=$1 ;;
"-v"|"--verbose") VERB="-v" ;;
"-y"|"--yes" ) YES="-y" ;;
+ "--abort-after-vdo-convert" ) ABORT_AFTER_VDO_CONVERT=1; USE_VDO_DM_SNAPSHOT=0 ;; # For testing only
"--dry-run" ) DRY="1" ; VERB="-v" ;;
+ "--no-snapshot" ) USE_VDO_DM_SNAPSHOT=0 ;;
+ "--uuid-prefix" ) shift; DM_UUID_PREFIX=$1 ;; # For testing only
+ "--vdo-config" ) shift; VDO_CONFIG=$1 ; VDOCONF="-f $VDO_CONFIG" ;;
"-*") error "Wrong argument \"$1\". (see: $TOOL --help)" ;;
- *) DEVICENAME=$1 ;; # device name does not start with '-'
+ *) DEVICE=$1 ;; # device name does not start with '-'
esac
shift
done
-test -n "$DEVICENAME" || error "Device name is not specified. (see: $TOOL --help)"
+test -n "$DEVICE" || error "Device name is not specified. (see: $TOOL --help)"
+
+convert2lvm_
-# do conversion
-convert2lvm_ "$DEVICENAME"
+cleanup 0
11 months, 2 weeks
main - vdo: fix and enhance vdo metadata reader
by Zdenek Kabelac
Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=00633f8b668d94a5875...
Commit: 00633f8b668d94a58756d96f5927fd690b8f41d3
Parent: fcf2057b3824ab461ad43821101d09f2b06dcac8
Author: Zdenek Kabelac <zkabelac(a)redhat.com>
AuthorDate: Fri Jun 9 13:21:35 2023 +0200
Committer: Zdenek Kabelac <zkabelac(a)redhat.com>
CommitterDate: Fri Jun 9 18:01:16 2023 +0200
vdo: fix and enhance vdo metadata reader
Improve metadata parser to handle volume_geometry bio_offset,
which needs to be substracted from 'region' start_block when present.
This bio_offset block is non-zero i.e. with converted VDO volumes.
Also fix some converted structure value (but they are not in use).
---
device_mapper/vdo/vdo_reader.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/device_mapper/vdo/vdo_reader.c b/device_mapper/vdo/vdo_reader.c
index 9f1344f4b..de8677c79 100644
--- a/device_mapper/vdo/vdo_reader.c
+++ b/device_mapper/vdo/vdo_reader.c
@@ -146,13 +146,14 @@ static void _vdo_decode_header(struct vdo_header *h)
static void _vdo_decode_geometry_region(struct vdo_volume_region *vr)
{
vr->id = le32_to_cpu(vr->id);
- vr->start_block = le32_to_cpu(vr->start_block);
+ vr->start_block = le64_to_cpu(vr->start_block);
}
static void _vdo_decode_volume_geometry(struct vdo_volume_geometry *vg)
{
- vg->release_version = le64_to_cpu(vg->release_version);
+ vg->release_version = le32_to_cpu(vg->release_version);
vg->nonce = le64_to_cpu(vg->nonce);
+ vg->bio_offset = le64_to_cpu(vg->bio_offset);
_vdo_decode_geometry_region(&vg->regions[VDO_DATA_REGION]);
}
@@ -224,10 +225,14 @@ bool dm_vdo_parse_logical_size(const char *vdo_path, uint64_t *logical_blocks)
goto err;
}
+ if (h.id != 5) {
+ log_debug_activation("Expected geometry VDO block instead of block %u.", h.id);
+ goto err;
+ }
memcpy(&vg, buffer + MAGIC_NUMBER_SIZE + sizeof(h), sizeof(vg));
_vdo_decode_volume_geometry(&vg);
- regpos = vg.regions[VDO_DATA_REGION].start_block * 4096;
+ regpos = (vg.regions[VDO_DATA_REGION].start_block - vg.bio_offset) * 4096;
if ((regpos + sizeof(buffer)) > size) {
log_debug_activation("File/Device is shorter and can't provide requested VDO volume region at " FMTu64 " > " FMTu64 ".", regpos, size);
@@ -244,7 +249,6 @@ bool dm_vdo_parse_logical_size(const char *vdo_path, uint64_t *logical_blocks)
goto err;
}
-
memcpy(&vn, buffer + sizeof(struct vdo_geometry_block), sizeof(vn));
_vdo_decode_version(&vn);
11 months, 2 weeks
main - cmd_enum: correct inclusion
by Zdenek Kabelac
Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=fcf2057b3824ab461ad...
Commit: fcf2057b3824ab461ad43821101d09f2b06dcac8
Parent: 74feebdab723c1ea46d4316f8a581750c1d8cda3
Author: Zdenek Kabelac <zkabelac(a)redhat.com>
AuthorDate: Thu May 18 21:53:33 2023 +0200
Committer: Zdenek Kabelac <zkabelac(a)redhat.com>
CommitterDate: Fri Jun 9 18:01:16 2023 +0200
cmd_enum: correct inclusion
Ensure we use generated header file for this build from -Iinclude.
Reported-by: Anton Lundin <glance(a)ac2.se>
---
lib/commands/cmd_enum.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/commands/cmd_enum.h b/lib/commands/cmd_enum.h
index 9e9902224..9d0937a50 100644
--- a/lib/commands/cmd_enum.h
+++ b/lib/commands/cmd_enum.h
@@ -12,7 +12,7 @@
enum {
#define cmd(a, b) a ,
-#include "../../include/cmds.h"
+#include "cmds.h"
#undef cmd
};
11 months, 2 weeks
main - device_id: fix handling of non-PV with duplicate serial number
by David Teigland
Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=74feebdab723c1ea46d...
Commit: 74feebdab723c1ea46d4316f8a581750c1d8cda3
Parent: 8e53b3bc14a03c243d10fc6bd9a7a8dc43d0a4d4
Author: David Teigland <teigland(a)redhat.com>
AuthorDate: Thu Jun 8 12:24:05 2023 -0500
Committer: David Teigland <teigland(a)redhat.com>
CommitterDate: Thu Jun 8 12:24:05 2023 -0500
device_id: fix handling of non-PV with duplicate serial number
Fix in the code that matches devices to system.devices entries when
the devices have the same serial number. A non-PV device in
system.devices has no pvid value, and the code was segfaulting
when checking the null pvid value.
---
lib/device/device_id.c | 2 ++
test/shell/devicesfile-serial.sh | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 40 insertions(+)
diff --git a/lib/device/device_id.c b/lib/device/device_id.c
index 3e05a40a8..9a8716500 100644
--- a/lib/device/device_id.c
+++ b/lib/device/device_id.c
@@ -2625,6 +2625,8 @@ void device_ids_check_serial(struct cmd_context *cmd, struct dm_list *scan_devs,
* Match du to a dev based on PVID.
*/
dm_list_iterate_items(dul, &dus_check) {
+ if (!dul->du->pvid)
+ continue;
log_debug("Matching suspect serial device id %s PVID %s prev %s",
dul->du->idname, dul->du->pvid, dul->du->devname);
found = 0;
diff --git a/test/shell/devicesfile-serial.sh b/test/shell/devicesfile-serial.sh
index 94ce3d74a..014f97e8e 100644
--- a/test/shell/devicesfile-serial.sh
+++ b/test/shell/devicesfile-serial.sh
@@ -853,6 +853,44 @@ grep $PVID4 out4
vgcreate $vg2 $dev2 $dev3
vgs | grep $vg2
+# 3 devs with duplicate serial, 2 pvs with stale devnames, 1 non-pv device
+
+aux wipefs_a $dev1
+aux wipefs_a $dev2
+aux wipefs_a $dev3
+
+echo $SERIAL1 > $SYS_DIR/dev/block/$MAJOR1:$MINOR1/device/serial
+echo $SERIAL1 > $SYS_DIR/dev/block/$MAJOR2:$MINOR2/device/serial
+echo $SERIAL1 > $SYS_DIR/dev/block/$MAJOR3:$MINOR3/device/serial
+
+rm $DF
+touch $DF
+vgcreate $vg1 $dev1 $dev2
+lvmdevices --adddev $dev3
+cat $DF
+cp $DF $ORIG
+
+PVID1=`pvs "$dev1" --noheading -o uuid | tr -d - | awk '{print $1}'`
+PVID2=`pvs "$dev2" --noheading -o uuid | tr -d - | awk '{print $1}'`
+OPVID1=`pvs "$dev1" --noheading -o uuid | awk '{print $1}'`
+OPVID2=`pvs "$dev2" --noheading -o uuid | awk '{print $1}'`
+
+pvs -o+uuid,deviceid
+
+sed -e "s|DEVNAME=$dev1|DEVNAME=tmp|" $ORIG > tmp1
+sed -e "s|DEVNAME=$dev2|DEVNAME=$dev1|" tmp1 > tmp2
+sed -e "s|DEVNAME=tmp|DEVNAME=$dev2|" tmp2 > $DF
+cat $DF
+
+# pvs should report the correct info and fix the DF
+pvs -o+uuid,deviceid |tee out
+grep $dev1 out |tee out1
+grep $dev2 out |tee out2
+grep $OPVID1 out1
+grep $OPVID2 out2
+grep $SERIAL1 out1
+grep $SERIAL1 out2
+
remove_base
rmmod brd
11 months, 2 weeks
main - man: lvmraid update limitations
by David Teigland
Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=8e53b3bc14a03c243d1...
Commit: 8e53b3bc14a03c243d10fc6bd9a7a8dc43d0a4d4
Parent: e2ecc6c763557462a4bc3d88bbc883917eb0f010
Author: David Teigland <teigland(a)redhat.com>
AuthorDate: Mon Jun 5 10:43:51 2023 -0500
Committer: David Teigland <teigland(a)redhat.com>
CommitterDate: Mon Jun 5 10:43:51 2023 -0500
man: lvmraid update limitations
---
man/lvmraid.7_main | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/man/lvmraid.7_main b/man/lvmraid.7_main
index 26938da83..086f53491 100644
--- a/man/lvmraid.7_main
+++ b/man/lvmraid.7_main
@@ -839,7 +839,7 @@ to do this.)
RAID LVs with integrity cannot yet be used as sub LVs with other LV types.
.P
The following are not yet permitted on RAID LVs with integrity: lvreduce,
-pvmove, snapshots, splitmirror, raid syncaction commands, raid rebuild.
+pvmove, lvconvert --splitmirrors, lvchange --syncaction, lvchange --rebuild.
.
.SH RAID1 TUNING
.
11 months, 2 weeks