9:18 a.m.
----- Original Message -----
From: "Nir Soffer" <nsoffer(a)redhat.com>
To: sanlock-devel(a)lists.fedorahosted.org
Cc: "Allon Mureinik" <amureini(a)redhat.com>
Sent: Sunday, August 10, 2014 10:16:03 PM
Subject: sanlk-resetd lockspace registeration managment
Hi David,
When registering locksapce (reg xxxxx), sanlk-resetd does not check if
lockspace with same name is already registered, and will register
another listener for the same lockspace. Since listener are not closed
when lockspace is invalidated
(https://lists.fedorahosted.org/pipermail/sanlock-devel/2014-August/000480...)
This will create a listener leak.
Since having more then one listener on the same lockspace does not make
sense,
I think we should first end the lockspace, and then register new one.
When unregistering lockspace (end xxxxx), the code assume that there is only
one lockspace with that name, possibly leaving duplicates unwanted listeners.
This assumption is ok, if the registration code did enforce this though.
Here is an example:
Register 2 lockspaces...
$ sanlk-reset reg 67ebc639-f790-4b1b-b5d2-526b0f569ac4
add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Updated reg 67ebc639-f790-4b1b-b5d2-526b0f569ac4
Updated reg add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Register again...
$ sanlk-reset reg 67ebc639-f790-4b1b-b5d2-526b0f569ac4
add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Updated reg 67ebc639-f790-4b1b-b5d2-526b0f569ac4
Updated reg add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Server side:
# sanlk-resetd -f -D
1407765864 sanlk-resetd 3.1.0 started
1407765864 setup_wdmd 4
1407765882 process_update reg 67ebc639-f790-4b1b-b5d2-526b0f569ac4
1407765882 reg_event fd 7 ls 67ebc639-f790-4b1b-b5d2-526b0f569ac4
1407765882 process_update reg add39817-ed89-4ef7-88cd-f6e4c5a11d1d
1407765882 reg_event fd 8 ls add39817-ed89-4ef7-88cd-f6e4c5a11d1d
1407765902 process_update reg 67ebc639-f790-4b1b-b5d2-526b0f569ac4
1407765902 reg_event fd 9 ls 67ebc639-f790-4b1b-b5d2-526b0f569ac4
1407765902 process_update reg add39817-ed89-4ef7-88cd-f6e4c5a11d1d
1407765902 reg_event fd 10 ls add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Now unregister:
$ sanlk-reset end 67ebc639-f790-4b1b-b5d2-526b0f569ac4
add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Updated end 67ebc639-f790-4b1b-b5d2-526b0f569ac4
Updated end add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Server side:
1407766150 process_update end 67ebc639-f790-4b1b-b5d2-526b0f569ac4
1407766150 process_update end add39817-ed89-4ef7-88cd-f6e4c5a11d1d
Now lets send a message the host:
# sanlk-reset reset 67ebc639-f790-4b1b-b5d2-526b0f569ac4:3
add39817-ed89-4ef7-88cd-f6e4c5a11d1d:3
Server side:
1407766339 request to reset (1 0) from host 1 24 ls 67ebc639-f790-4b1b-b5d2-526b0f569ac4
Basically it worked, but I asked to stop listening to these lockspaces :-)
Here is the relevant code from reset/sanlk-resetd.c:
if (!strcmp(cmd, "reg")) {
log_debug("process_update reg %s", name);
for (i = 0; i < MAX_LS; i++) {
if (ls_names[i])
continue;
ls_names[i] = strdup(name); // There may be already lockspace with this
name
register_ls(i);
return;
}
} else if (!strcmp(cmd, "end")) {
log_debug("process_update end %s", name);
for (i = 0; i < MAX_LS; i++) {
if (!ls_names[i] || strcmp(ls_names[i], name))
continue;
unregister_ls(i);
return; // There may be another lockspace with this name
}
Last, there is an issue with the api. vdsm keeps the current lockspaces in
memory (actually is running a monitor thread for each one). So when vdsm
is asked to change the list of active lockspaces by engine, it can easily
end lockspaces which are not needed any more, and register new lockspaces.
However, if vdsm crashes and restart, it does know which lockspaces were
used before, so it cannot end these lockspaces. What will happen is actually
asking to register the same locksapces again, which given the current
sanlk-resetd behavior, will create a leak for all lockspaces on each vdsm
crash.
vdsm can try to end all lockspaces before registering them, but the new
set of lockspaces may be different from the old one, as the list is managed
by the engine, and vdsm keep no state locally.
Replacing listeners with same lockspace name will solve the duplicate issue,
however it will not solve the trouble with unused lockspaces.
I think that a new command, "clear", unregistering all lockspaces will
solve this issue. vdsm will clear the listeners each time it connects to
storage and maybe when going into maintenance mode.
Another option is a way to query the current locksapces that sanlk-resetd
is listening on. Given this list, vdsm can send the correct reg and end
commands easily. But this will require more work on both sanlk-resetd
and vdsm.
Nir
_______________________________________________
sanlock-devel mailing list
sanlock-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/sanlock-devel