Changes to 'review2'
by David Teigland
New branch 'review2' available with the following commits:
commit e0ee4ea4e7279af31efc4737065fbcce043a32e0
Author: David Teigland <teigland(a)redhat.com>
Date: Tue May 27 15:16:00 2014 -0500
sanlock: lockspace host events
Hosts can set/get events between each other using a common lockspace.
. set_event takes two opaque 64 bit values ("event" and "data"), and
a target host_id with optional host generation.
. reg_event creates an fd that a program can use in poll(2), and
. get_event on this fd returns the event values that have been set
along with the source host_id/generation.
. end_event releases the registered fd.
Each set_event call replaces generation/event/data values from the
previous set_event call, but host_id's from each are maintained.
set_event can be used to send the same event/data values to multiple
hosts at once using multiple set_event calls (generation is not
applicable in this case.)
set_event with different values within the same window of time results
in -EBUSY (unless the REPLACE flag is used). Otherwise, the second
set_event may result in the earlier target host(s) seeing later values
not intended for them, and missing the earlier values from the first
set_event.
Event values are passed through fields in the lockspace delta leases.
Event notification is done via the host_id bitmaps in lockspace delta
leases. Setting an event (or making a resource request) for a target host
results in the target host's bit being set in the sending host's bitmap.
The bit remains set for set_bitmap_seconds (60 sec with 10 sec io
timeout.) After this, the bit is cleared and the target host will no
longer be notified. The target host will typically generate three events
for each set_event, once from each delta lease renewal (every 20 seconds)
over the course of the set_bitmap_seconds.
Signed-off-by: David Teigland <teigland(a)redhat.com>
commit 2e5150be0ad662f218a5442bd1c40f12c825022d
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Jun 19 10:51:54 2014 -0500
sanlock: fix get_hosts off by one
When a host_id is specified, the information for
host_id+1 is returned.
Signed-off-by: David Teigland <teigland(a)redhat.com>
commit c895fb472b0bd1faf088e0ff11b41f0a6864168b
Author: David Teigland <teigland(a)redhat.com>
Date: Fri Apr 11 14:10:25 2014 -0500
sanlock: shutdown reply
A wait option (-w 1) can be added to sanlock shutdown.
The shutdown command will wait for a result from the
daemon indicating success (exiting), or failure (not
exiting because lockspaces exist).
Signed-off-by: David Teigland <teigland(a)redhat.com>
9 years, 5 months
- 1
- 0
Re: sanlock: shutdown reply
by Nir Soffer
> commit 93885cbfc6271ecb5007601369ad29cee3df6b99
> Author: David Teigland <teigland(a)redhat.com>
> Date: Fri Apr 11 14:10:25 2014 -0500
>
> sanlock: shutdown reply
>
> A wait option (-w 1) can be added to sanlock shutdown.
> The shutdown command will wait for a result from the
> daemon indicating success (exiting), or failure (not
> exiting because lockspaces exist).
>
This option seems useful for vdsm.
> Signed-off-by: David Teigland <teigland(a)redhat.com>
>
> diff --git a/src/client_cmd.c b/src/client_cmd.c
> index 7723f88..15c26a1 100644
> --- a/src/client_cmd.c
> +++ b/src/client_cmd.c
> @@ -636,15 +636,40 @@ int sanlock_log_dump(int max_size)
> return rv;
> }
>
> -int sanlock_shutdown(uint32_t force)
> +int sanlock_shutdown(uint32_t force, int wait_result)
> {
> + struct sm_header h;
> + int cmd;
> int fd;
> + int rv = 0;
> +
> + if (wait_result)
> + cmd = SM_CMD_SHUTDOWN_WAIT;
> + else
> + cmd = SM_CMD_SHUTDOWN;
>
> - fd = send_command(SM_CMD_SHUTDOWN, force);
> + fd = send_command(cmd, force);
> if (fd < 0)
> return fd;
What you mean is return -1 - right?
>
> + if (cmd != SM_CMD_SHUTDOWN_WAIT)
> + goto out;
> +
> + memset(&h, 0, sizeof(h));
> +
> + rv = recv(fd, &h, sizeof(h), MSG_WAITALL);
> + if (rv < 0) {
> + rv = -errno;
> + goto out;
> + }
> + if (rv != sizeof(h)) {
> + rv = -1;
> + goto out;
> + }
> +
> + rv = h.data;
> + out:
> close(fd);
> - return 0;
> + return rv;
> }
>
> diff --git a/src/client_cmd.h b/src/client_cmd.h
> index 35f82d9..f7c69c1 100644
> --- a/src/client_cmd.h
> +++ b/src/client_cmd.h
> @@ -12,6 +12,6 @@
> int sanlock_status(int debug, char sort_arg);
> int sanlock_host_status(int debug, char *lockspace_name);
> int sanlock_log_dump(int max_size);
> -int sanlock_shutdown(uint32_t force);
> +int sanlock_shutdown(uint32_t force, int wait_result);
>
> #endif
> diff --git a/src/cmd.c b/src/cmd.c
> index f19e8fb..a9a1eef 100644
> --- a/src/cmd.c
> +++ b/src/cmd.c
> @@ -1159,6 +1159,74 @@ static void cmd_get_lvb(struct task *task GNUC_UNUSED,
> struct cmd_args *ca)
> client_resume(ca->ci_in);
> }
>
> +static int shutdown_reply_ci = -1;
> +static int shutdown_reply_fd = -1;
> +
> +static int daemon_shutdown_start(int ci, int fd, int force)
> +{
> + int rv;
> +
> + if (force) {
> + shutdown_reply_ci = ci;
> + shutdown_reply_fd = fd;
> + external_shutdown = 2;
> + return 0;
> + }
> +
> + pthread_mutex_lock(&spaces_mutex);
> + if (list_empty(&spaces) &&
> + list_empty(&spaces_rem) &&
> + list_empty(&spaces_add)) {
> + shutdown_reply_ci = ci;
> + shutdown_reply_fd = fd;
> + external_shutdown = 1;
> + rv = 0;
> + } else {
> + rv = -EBUSY;
> + }
> + pthread_mutex_unlock(&spaces_mutex);
> +
> + return rv;
> +}
> +
> +static void cmd_shutdown_wait(struct task *task GNUC_UNUSED, struct cmd_args
> *ca)
> +{
> + int fd, result;
> +
> + fd = client[ca->ci_in].fd;
> +
> + result = daemon_shutdown_start(ca->ci_in, fd, ca->header.data);
> +
> + /*
> + * daemon_shutdown_reply will send the result at the
> + * end of main_loop.
> + */
> + if (!result)
> + return;
It will be more clear to do:
if (result == -EBUSY)
return;
> +
> + send_result(fd, &ca->header, result);
> + client_resume(ca->ci_in);
> +}
> +
> +void daemon_shutdown_reply(void)
> +{
> + struct sm_header h;
> +
> + /* shutdown wait was not used */
> + if (shutdown_reply_fd == -1)
> + return;
> +
> + memset(&h, 0, sizeof(h));
> + h.magic = SM_MAGIC;
> + h.version = SM_PROTO;
> + h.length = sizeof(h);
> +
> + send(shutdown_reply_fd, &h, sizeof(h), MSG_NOSIGNAL);
> + close(shutdown_reply_fd);
> +
> + client_resume(shutdown_reply_ci);
> +}
> +
> static void cmd_add_lockspace(struct cmd_args *ca)
> {
> struct sanlk_lockspace lockspace;
> @@ -1908,6 +1976,9 @@ void call_cmd_thread(struct task *task, struct cmd_args
> *ca)
> case SM_CMD_GET_LVB:
> cmd_get_lvb(task, ca);
> break;
> + case SM_CMD_SHUTDOWN_WAIT:
> + cmd_shutdown_wait(task, ca);
> + break;
> };
> }
>
> diff --git a/src/cmd.h b/src/cmd.h
> index 3f31283..30c19e6 100644
> --- a/src/cmd.h
> +++ b/src/cmd.h
> @@ -24,4 +24,6 @@ void call_cmd_thread(struct task *task, struct cmd_args
> *ca);
> /* cmds processed by main loop */
> void call_cmd_daemon(int ci, struct sm_header *h_recv, int client_maxi);
>
> +void daemon_shutdown_reply(void);
> +
> #endif
> diff --git a/src/main.c b/src/main.c
> index 5f3c6ed..dbb6da8 100644
> --- a/src/main.c
> +++ b/src/main.c
> @@ -840,6 +840,8 @@ static int main_loop(void)
>
> free_lockspaces(1);
>
> + daemon_shutdown_reply();
> +
> return 0;
> }
>
> @@ -1194,6 +1196,7 @@ static void process_connection(int ci)
> case SM_CMD_READ_RESOURCE_OWNERS:
> case SM_CMD_SET_LVB:
> case SM_CMD_GET_LVB:
> + case SM_CMD_SHUTDOWN_WAIT:
> rv = client_suspend(ci);
> if (rv < 0)
> return;
> @@ -1814,7 +1817,7 @@ static void print_usage(void)
> printf("sanlock client gets [-h 0|1]\n");
> printf("sanlock client host_status -s LOCKSPACE [-D]\n");
> printf("sanlock client log_dump\n");
> - printf("sanlock client shutdown [-f 0|1]\n");
> + printf("sanlock client shutdown [-f 0|1] [-w 0|1]\n");
For boolean option, it is more user friendly to use -w without a value:
sanlock client shutdown -w
But it seems that this is already use like this in other commands :-(
> printf("sanlock client init -s LOCKSPACE | -r RESOURCE\n");
> printf("sanlock client read -s LOCKSPACE | -r RESOURCE\n");
> printf("sanlock client align -s LOCKSPACE\n");
> @@ -2047,6 +2050,7 @@ static int read_command_line(int argc, char *argv[])
> break;
> case 'w':
> com.use_watchdog = atoi(optionarg);
> + com.wait = atoi(optionarg);
> break;
> case 'h':
> if (com.action == ACT_GETS || com.action == ACT_CLIENT_READ)
> @@ -2399,8 +2403,8 @@ static int do_client(void)
> break;
>
> case ACT_SHUTDOWN:
> - log_tool("shutdown");
> - rv = sanlock_shutdown(com.force_mode);
> + log_tool("shutdown force %d wait %d", com.force_mode, com.wait);
> + rv = sanlock_shutdown(com.force_mode, com.wait);
> log_tool("shutdown done %d", rv);
> break;
>
> diff --git a/src/sanlock.8 b/src/sanlock.8
> index 0d2b63c..7b92329 100644
> --- a/src/sanlock.8
> +++ b/src/sanlock.8
> @@ -273,7 +273,10 @@ Print the sanlock daemon internal debug log.
> Ask the sanlock daemon to exit. Without the force option (-f 0), the
> command will be ignored if any lockspaces exist. With the force option
> (-f 1), any registered processes will be killed, their resource leases
> -released, and lockspaces removed.
> +released, and lockspaces removed. With the wait option (-w 1), the
> +command will wait for a result from the daemon indicating that it has
> +shut down and is exiting, or cannot shut down because lockspaces
> +exist (command fails).
>
> .BR "sanlock client init -s" " LOCKSPACE"
>
> diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
> index c3fa574..a62135c 100644
> --- a/src/sanlock_internal.h
> +++ b/src/sanlock_internal.h
> @@ -265,6 +265,7 @@ struct command_line {
> int debug;
> int debug_renew;
> int quiet_fail;
> + int wait;
> int use_watchdog;
> int high_priority; /* -h */
> int get_hosts; /* -h */
> diff --git a/src/sanlock_sock.h b/src/sanlock_sock.h
> index c5c1ec9..8e8f5e2 100644
> --- a/src/sanlock_sock.h
> +++ b/src/sanlock_sock.h
> @@ -47,6 +47,7 @@ enum {
> SM_CMD_GET_LVB = 26,
> SM_CMD_CONVERT = 27,
> SM_CMD_VERSION = 28,
> + SM_CMD_SHUTDOWN_WAIT = 29,
> };
>
> struct sm_header {
How much time it takes to get a reply from the daemon?
We certainly like this feature in next release, but it is not a must.
Thanks,
Nir
9 years, 5 months
- 1
- 0
Changes to 'review'
by David Teigland
New branch 'review' available with the following commits:
commit 1b772c431669971119f8fb7eb5e906fe8b0ece48
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Jun 19 10:51:54 2014 -0500
sanlock: fix get_hosts off by one
When a host_id is specified, the information for
host_id+1 is returned.
Signed-off-by: David Teigland <teigland(a)redhat.com>
commit 153a0e2a0ba377f471455aad803f648556f8f06e
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Jun 5 15:38:30 2014 -0500
sanlock: set an event for all hosts in the bitmap
Signed-off-by: David Teigland <teigland(a)redhat.com>
commit 32f22cf728955bf4101752b515f712615a79e267
Author: David Teigland <teigland(a)redhat.com>
Date: Fri May 30 15:06:00 2014 -0500
reset: daemon and program for resetting hosts
The sanlk_resetd daemon gets events from specified lockspaces.
Events are defined to cause the sanlk_resetd to:
- use wdmd/watchdog to reset the host (event 1)
- use /proc/sysrq-trigger to reboot the host (event 4)
The sanlk_resetd daemon can be used alone, and events passed
to it from another host by:
sanlock client set_event -s lockspace_name -i host_id -e 1
The sanlk_reset program can also be used to communicate with
sanlk_resetd on another host. It will set the event(s), then
monitor the host status until the target host is dead.
Signed-off-by: David Teigland <teigland(a)redhat.com>
commit 77c93f9ee653be0d6b9d2097ca34a4ecd2993911
Author: David Teigland <teigland(a)redhat.com>
Date: Tue May 27 15:16:00 2014 -0500
sanlock: lockspace host events
Hosts can set/get events between each other using a common lockspace.
. set_event takes two opaque 64 bit values ("event" and "data"), and
a target host_id with optional host generation.
. reg_event creates an fd that a program can use in poll(2), and
. get_event on this fd returns the event values that have been set
along with the source host_id/generation.
. end_event releases the registered fd.
Each set_event call replaces generation/event/data values from the
previous set_event call, but host_id's from each are maintained.
set_event can be used to send the same event/data values to multiple
hosts at once using multiple set_event calls (generation is not
applicable in this case.)
set_event with different values within the same window of time can
result in the earlier target host(s) seeing later values not intended
for them, and missing the earlier values.
Event values are passed through fields in the lockspace delta leases.
Event notification is done via the host_id bitmaps in lockspace delta
leases. Setting an event (or making a resource request) for a target host
results in the target host's bit being set in the sending host's bitmap.
The bit remains set for set_bitmap_seconds (60 sec with 10 sec io
timeout.) After this, the bit is cleared and the target host will no
longer be notified. The target host will typically generate three events
for each set_event, once from each delta lease renewal (every 20 seconds)
over the course of the set_bitmap_seconds.
Signed-off-by: David Teigland <teigland(a)redhat.com>
commit 93885cbfc6271ecb5007601369ad29cee3df6b99
Author: David Teigland <teigland(a)redhat.com>
Date: Fri Apr 11 14:10:25 2014 -0500
sanlock: shutdown reply
A wait option (-w 1) can be added to sanlock shutdown.
The shutdown command will wait for a result from the
daemon indicating success (exiting), or failure (not
exiting because lockspaces exist).
Signed-off-by: David Teigland <teigland(a)redhat.com>
9 years, 5 months
- 1
- 0
- ← Newer
- 1
- Older →