Changes to 'refs/tags/sanlock-1.8'
by David Teigland
Changes since the dawn of time:
Daniel P. Berrange (15):
Fix const-ness of many APIs.
Fix warnings in watchdog module
Fix function prototypes for no-arg methods
Remove use of 'index' as a variable name
Make many functions static
Fix missing include in logging file
Annotate many unused parameters to avoid warnings
Remove redundant redeclaration of 'to' variable
Fix args to execv()
Remove redundant arg to 'run_command'
Rename optarg to optionarg to avoid clashing with getopt.h
Disable the read_request method since it is unused
Add many more compiler warning flags & safety checks
Hard code a sector size of 512 if the lease volume is a regular file
Ensure libsanlock.so is built with debug/warning flags
David Teigland (292):
sync_manager: initial commit
sync_manager: misc updates
sync_manager: misc updates
sync_manager: misc updates
sync_manager: misc updates
sync_manager: add more logging
sync_manager: misc updates
sync_manager: misc updates
sync_manager: num_hosts/MAX_HOSTS stuff
daemon: reworking notions of resource_id/token_name
sync_manager: resource lockfiles
sync_manager: lease arg processing
sync_manager: Began multiple lease support
sync_manager: use first command line arg as action
sync_manager: leader record changes and verify
sync_manager: clean up released leases
sync_manager: move functions around
sync_manager: add more tool actions
sync_manager: naming changes
sync_manager: separate token index and id
sync_manager: fix index usage and other misc
sync_manager: use pthread cond in acquire
sync_manager: write all log entries
sync_manager: simplify polling
sync_manager: fix waitpid use
sync_manager: acquire can fail early
sync_manager: write log entries at exit
sync_manager: add test program
sync_manager: move secondary pid check
sync_manager: fix disk paxos contention
devcount: fix verify checks
sync_manager: add GPL license file
sync_manager: fix leader block offsets
sync_manager: increase COMMAND_MAX
sync_manager: renewal should verify
sync_manager: use sector size from libblkid
sync_manager: use a real checksum function
sync_manager: add libblkid to spec file
sync_manager: print status info
sync_manager: one watchdog file per lease
sync_manager: lease_threads handle watchdog files
sync_manager: fix/add some text/comments
sync_manager: refactor read/write
sync_manager: move disk io functions
sync_manager: remove token arg
sync_manager: rename paxos_disk sync_disk
sync_manager: add aio read/write
sync_manager: make io_timeout_seconds a diskio arg
sync_manager: forgot to add new files
sync_manager: use log thread
sync_manager: client connections
sync_manager: connection processing
sync_manager: send/recv pid
sync_manager: add write_sectors
sync_manager: restructuring
sync_manager: write_sectors code factoring
sync_manager: daemonize
sync_manager: releasing leases
sync_manager: async releasing
sync_manager: release fixes
sync_manager: add direct and indirect acquire/release
sync_manager: reacquire resources
sync_manager: move code
sync_manager: same pid may reacquire resource
sync_manager: lease migration
sync_manager: handle client errors
sync_manager: improve error handling
sync_manager: host_id leases
sync_manager: remove empty files
sync_manager: print initialization info
sync_manager: rename files
sync_manager: clean up header org
sync_manager: delta_lease implementation
sync_manager: accept offset units
sync_manager: fix up init output
sync_manager: put back watchdog calls
sync_manager: fix start_host_id error paths
sync_manager: add log_error's for watchdog file errors
sync_manager: actual timeouts
sync_manager: change timeouts on cmd line
sanlock: create new external api
sanlock: build libsanlock
sanlock: use MAX_LEASES everywhere
sanlock: add libvirt plugin
sanlock plugin: couple minor fixes
sanlock: clean up /var file names
sanlock plugin: fix symbol needed by libvirt
sanlock: add some debug output
sanlock plugin: fix uuid copy
sanlock plugin: fix names
sanlock: add "owner_name"
sanlock: fix renewal checks
sanlock: clean up host_id types
sanlock: set_host_id command
sanlock: fix killing pids
sanlock: add status command
sanlock: set version to 1.0
sanlock: delta_lease cleanup
sanlock: changing num_hosts
sanlock: add dump command
sanlock: renewal timings
sanlock: add direct option
sanlock: check for watchdog file
sanlock: recovery fixes
lock_driver_sanlock: fix compile problems
sanlock: improve command options
sanlock: tidying help text
sanlock: move binary to /usr/sbin
sanlock: add init script
sanlock: fix sigterm shutdown
sanlock: init stop
sanlock: add wdtest command
sanlock.spec: new url
lock_driver_sanlock: remove close
sanlock: introduce lockspaces
lock_driver_sanlock: remove files
sanlock: better logging functions
sanlock: misc log message
sanlock.spec: sbin not libexec
sanlock init: remove watchdog reference
wdmd: watchdog multiplexing daemon
sanlock: add code to use wdmd
sanlock/wdmd: use wdmd in sanlock
sanlock/wdmd: add copyright header to source files
sanlock: rename sanlock source dir
sanlock: move tests dir
move COPYING file
wdmd: use signalfd for signal handling
Fix Makefile comments
wdmd: fix daemon debug option
wdmd: add init script
sanlock.spec: updates
sanlock.spec: src dir
sanlock: build with uninstalled libwdmd
sanlock: version 1.1
sanlock: high priority options
wdmd: high priority options
sanlock: return migration state
sanlock: migration.txt describes libvirt/sanlock steps
libsanlock: include admin functions
sanlock: fix host_id expiration check
sanlock: migration working
devcount: migrate test
sanlock: setowner improvements
sanlock: migrate to target fix
sanlock: fix wdmd stop order
sanlock: various fixes
sanlock: remove wdtest
sanlock: remove migration
sanlock: clean up command return data
sanlock: add resource string conversion functions
sanlock: rework internal structs
devcount: add relock test
sanlock: fix release and inquire
sanlock: add_lockspace EEXIST
sanlock: rework client handling
sanlock: clean up warnings
sanlock: debug message changes
sanlock: add lockspace checks
wdmd: enable test scripts
sanlock: add str_to_lockspace to lib
WIP devcount migrate
devcount: new migrate test
sanlock: read_id and live_id commands
sanlock: check lockspace name and host_id
sanlock: remove remaining cluster_mode
sanlock: add libsanlock_direct
devcountn: start multiple devcount tests
devcount: small changes
sanlock: new return values
sanlock: misc changes and fix
sanlock: log error of full bad block
sanlock: interval between renewal checks
sanlock: renewal changes
sanlock: fix log_dump
sanlock: fix find_client_pid
sanlock: fix host_id reads from paxos_acquire
sanlock: init with one write
devcount: improve output
devcount: new pause/resume
devcount: add expire test
sanlock: correct paxos usage
sanlock: direct read_leader
sanlock: paxos delays
sanlock: use thread pool
sanlock: client status output format changes
sanlock: fix inquire of dead pid
sanlock: use native linux aio
sanlock: i/o changes
sanlock: aio changes
sanlock: reduce paxos acquire read ops
sanlock: quiet error case
sanlock: don't free aio buf until event completes
sanlock: io timeout related changes
sanlock: read dblocks in single aligned io
sanlock: add sanlock_restrict api
sanlock: add sanlock_direct_sector_size api
sanlock: add checksum to dblocks
sanlock: fix init restart
sanlock: don't release tokens in dead lockspace
sanlock: fix adding lockspace
sanlock: official 1MB/8MB alignment
devcount: use aio in init
libsanlock: link with LDFLAGS
sanlock: increase version to 1.3
sanlock/wdmd: shut up warnings
sanlock: fix libwdmd linking
remove spec file
sanlock: use a completed read after renewal timeout
sanlock: use unique host name in delta leases
sanlock: remove sector_size api
sanlock: abort delta wait on shutdown
sanlock: fix add_lockspace failure
sanlk_load: add new test
sanlock: fix recv and inquire
sanlock: initial pid_dead check in acquire
sanlock: release 1.4
sanlock: generate a uuid for host id
sanlock: return -EINPROGRESS from add_lockspace
sanlk_load: periodically kill and replace a pid
sanlock: zero num_hosts uses DEFAULT_MAX_HOSTS
tests: misc changes
sanlock: break paxos_acquire wait loop
sanlock: increase log line to 512 bytes
sanlock: change a log_error to log_debug
sanlock: fail host_id when corrupted
sanlock: release 1.5
sanlock: release 1.6
sanlock: handle colon escaping in path strings
wdmd: add option for high priority
wdmd: use accept4 with SOCK_NONBLOCK
wdmd: tidy sun_addr snprintf
wdmd: pid and sock file changes
wdmd: add man page
wdmd: disable test scripts
sanlock: use accept4 with SOCK_NONBLOCK
sanlock: tidy sun_addr snprintf
sanlock: add explicit -luuid
sanlock: pid and sock file changes
sanlock: add man page
sanlock/wdmd: improve mkdir of run dir
wdmd: new build flags
sanlock: new build flags
sanlock/wdmd: use monotonic time
sanlock: build with pie
sanlock/wdmd: nonblocking listening/accept
sanlock: add missing monotime files
sanlock: update man page
sanlock: man page update
sanlock: update man page and help text
sanlock: print connections limit
release: sanlock 1.7
makefile: install mode for man pages
sanlock: read align_size in renewal
sanlock: check other host_id leases
sanlock: minor fixes and cleanups
sanlock: add request api/cmd
sanlock: crc code tidying
sanlock/wdmd: add license header to files
sanlock: create libsanlock_client
sanlock: move client code
remove COPYING file
sanlock: remove internal header from sanlock_sock
libsanlock_client: use LGPLv2+
libwdmd: use LGPLv2+
sanlock_rv.h: switch to LGPLv2+
README.license: document licenses
sanlock: client align and init
sanlock: write request record
sanlock: request struct magic and version numbers
sanlock: set bitmap for requests
sanlock: examine resource requests
sanlock: update man page
sanlock: remove old comment
sanlock: renaming functions
sanlock: optimize paxos wait
sanlock: use flags in struct resource
sanlock: restructure lockspace checks
sanlock: remove BLOCK_WD force mode
libsanlock: fix function stubs
sanlock: new status and host_status
sanlock: improve status output
sanlock: status output sorting
sanlock: SIGTERM and SIGKILL for REQ_KILL_PID
sanlock: add flag RESTRICT_SIGKILL
sanlock: setup fewer aio events
sanlock: move cmd processing
sanlock: suppress log messages
sanlock: rename source files
sanlock: improve killing pids
sanlock: fix log_dump
version 1.8
Fabio M. Di Nitto (5):
build: sanlock should link with libsanlock
build: install shared lib header files and fix DESTDIR usage
build: drop rpm target
spec file: do first cut for total spec file
build: fix linking with libsanlock and install target
Federico Simoncelli (33):
rpm: sync specfile with fedora
rpm: add sanlock_admin.h header
rpm: add the lib package, install the initscripts
python: remove unused python code
python: add python binding
python: release the gil during sanlock operations
python: wrap sanlock extension with a module
rpm: add python binding package
python: pass a lockspace copy to str_to_lockspace
makefile: fix install typo
rpm: add sanlock_direct header
python: add sanlock init functions and exception
direct: close disks after initialization
python: register process only once
daemon: configurable socket permissions
rpm: add sanlock user and group
python: exceptions must contain the errno
rpm: add missing libaio-devel dependency
rpm: add daemon options in the init file
python: add missing aio library
python: add get_alignment function
libs: include libsanlock_direct into libsanlock
python: align num_hosts and max_hosts defaults
python: expose sanlock file descriptor
python: improve error reporting
python: parse lockspaces and resources natively
python: add usage example
python: initial support for sanlock errors
python: document the sanlock module
python: module cleanup
build: fix documentation install path
client: return appropriate errno on failure
python: expose errno in the exception
Saggi Mizrahi (21):
Added the begining of the testing and debugging tools
Better handling of max hosts
sync_manager: Updated tests to work with new lease struct
sync_manager: fixed skipping first arg in command
sync_manager: acquire and release actions
sync_manager: minor fixes
sync_manager: renamed stuff
sync_manager: made acquire synchronous again
sync_manager: added set_host_id action
sync_manager: use kill(0) for secondary pid check
sync_manager: make rpm and install
sync_manager: spec file update
sync_manager: Allow longer resource names
sync_manager: allow repeated set_host_id
sync_manager: Added escaping for the leases arg
sync_manager: Created the python bindings for sync_manager
sync_manager: listener socket permissions
sync_manager: Updated python binding and tests
sync_manager: Made 'token' a const in log_level
sync_manager: refactor messaging system
sync_manager: use getsockopt PEERCRED
12 years, 2 months
src/main.c wdmd/main.c
by David Teigland
src/main.c | 2 +-
wdmd/main.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
New commits:
commit 68aa2ca480d93737b46b0464490f85316835fc39
Author: David Teigland <teigland(a)redhat.com>
Date: Fri Sep 16 13:12:37 2011 -0500
version 1.8
diff --git a/src/main.c b/src/main.c
index e5aa9c6..e2d0c0b 100644
--- a/src/main.c
+++ b/src/main.c
@@ -1317,7 +1317,7 @@ static int parse_arg_resource(char *arg)
return 0;
}
-#define RELEASE_VERSION "1.7"
+#define RELEASE_VERSION "1.8"
/*
* daemon: acquires leases for the local host_id, associates them with a local
diff --git a/wdmd/main.c b/wdmd/main.c
index b31d9f6..b3e9d67 100644
--- a/wdmd/main.c
+++ b/wdmd/main.c
@@ -888,8 +888,8 @@ static void print_usage(void)
would be for wdmd to fail starting if it found a pid file left over from
its previous run. */
-#define RELEASE_VERSION "1.1"
-
+#define RELEASE_VERSION "1.8"
+
int main(int argc, char *argv[])
{
char optchar;
12 years, 2 months
src/client_cmd.c src/client_cmd.h src/cmd.c src/log.c src/log.h src/main.c src/sanlock_internal.h
by David Teigland
src/client_cmd.c | 38 ++++++++++++++++++++++++++------------
src/client_cmd.h | 2 +-
src/cmd.c | 17 +++++++++++++----
src/log.c | 42 ++++++++++++++++++++++++++++++------------
src/log.h | 2 +-
src/main.c | 2 +-
src/sanlock_internal.h | 2 +-
7 files changed, 73 insertions(+), 32 deletions(-)
New commits:
commit 0b2e562ff5f5a719cbb320868d8a7629dac4cc0e
Author: David Teigland <teigland(a)redhat.com>
Date: Fri Sep 16 13:08:10 2011 -0500
sanlock: fix log_dump
and make it more efficient
diff --git a/src/client_cmd.c b/src/client_cmd.c
index 119149f..2662122 100644
--- a/src/client_cmd.c
+++ b/src/client_cmd.c
@@ -470,15 +470,22 @@ int sanlock_host_status(int debug, char *lockspace_name)
return rv;
}
-int sanlock_log_dump(void)
+int sanlock_log_dump(int max_size)
{
struct sm_header h;
- char buf[4096];
+ char *buf;
int fd, rv;
+ buf = malloc(max_size);
+ if (!buf)
+ return -ENOMEM;
+ memset(buf, 0, max_size);
+
fd = send_command(SM_CMD_LOG_DUMP, 0);
- if (fd < 0)
+ if (fd < 0) {
+ free(buf);
return fd;
+ }
memset(&h, 0, sizeof(h));
@@ -492,20 +499,27 @@ int sanlock_log_dump(void)
goto out;
}
+ if (h.data <= 0 || h.data > max_size)
+ goto out;
- while (1) {
- memset(buf, 0, sizeof(buf));
-
- rv = recv(fd, buf, sizeof(buf) - 1, MSG_WAITALL);
-
- if (rv > 0)
- printf("%s", buf);
- else
- break;
+ rv = recv(fd, buf, h.data, MSG_WAITALL);
+ if (rv < 0) {
+ rv = -errno;
+ goto out;
}
+ if (!rv) {
+ rv = -1;
+ goto out;
+ }
+
+ printf("%s", buf);
printf("\n");
+
+ if (rv != h.data)
+ printf("partial dump %d of %d\n", rv, h.data);
out:
close(fd);
+ free(buf);
return rv;
}
diff --git a/src/client_cmd.h b/src/client_cmd.h
index dceb7ca..6759c93 100644
--- a/src/client_cmd.h
+++ b/src/client_cmd.h
@@ -11,7 +11,7 @@
int sanlock_status(int debug, char sort_arg);
int sanlock_host_status(int debug, char *lockspace_name);
-int sanlock_log_dump(void);
+int sanlock_log_dump(int max_size);
int sanlock_shutdown(void);
#endif
diff --git a/src/cmd.c b/src/cmd.c
index bbab319..97c4eba 100644
--- a/src/cmd.c
+++ b/src/cmd.c
@@ -1196,13 +1196,15 @@ static int print_state_daemon(char *str)
"io_timeout=%d "
"id_renewal=%d "
"id_renewal_fail=%d "
- "id_renewal_warn=%d",
+ "id_renewal_warn=%d "
+ "monotime=%llu",
our_host_name_global,
main_task.use_aio,
main_task.io_timeout_seconds,
main_task.id_renewal_seconds,
main_task.id_renewal_fail_seconds,
- main_task.id_renewal_warn_seconds);
+ main_task.id_renewal_warn_seconds,
+ (unsigned long long)monotime());
return strlen(str) + 1;
}
@@ -1537,11 +1539,18 @@ static void cmd_host_status(int fd, struct sm_header *h_recv)
free(status);
}
+static char send_log_dump[LOG_DUMP_SIZE];
+
static void cmd_log_dump(int fd, struct sm_header *h_recv)
{
- send(fd, h_recv, sizeof(struct sm_header), MSG_DONTWAIT);
+ int len;
+
+ copy_log_dump(send_log_dump, &len);
+
+ h_recv->data = len;
- write_log_dump(fd);
+ send(fd, h_recv, sizeof(struct sm_header), MSG_NOSIGNAL);
+ send(fd, send_log_dump, len, MSG_NOSIGNAL);
}
static void cmd_restrict(int ci, int fd, struct sm_header *h_recv)
diff --git a/src/log.c b/src/log.c
index 72b424f..7513152 100644
--- a/src/log.c
+++ b/src/log.c
@@ -33,7 +33,7 @@ static pthread_t thread_handle;
static pthread_mutex_t log_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t log_cond = PTHREAD_COND_INITIALIZER;
-static char log_dump[SM_LOG_DUMP_SIZE];
+static char log_dump[LOG_DUMP_SIZE];
static unsigned int log_point;
static unsigned int log_wrap;
@@ -42,9 +42,9 @@ struct entry {
char str[LOG_STR_LEN];
};
-#define SM_LOG_DEFAULT_ENTRIES 4096
+#define LOG_DEFAULT_ENTRIES 4096
static struct entry *log_ents;
-static unsigned int log_num_ents = SM_LOG_DEFAULT_ENTRIES;
+static unsigned int log_num_ents = LOG_DEFAULT_ENTRIES;
static unsigned int log_head_ent; /* add at head */
static unsigned int log_tail_ent; /* remove from tail */
static unsigned int log_dropped;
@@ -62,10 +62,21 @@ static void _log_save_dump(int level GNUC_UNUSED, int len)
{
int i;
+ if (len < LOG_DUMP_SIZE - log_point) {
+ memcpy(log_dump+log_point, log_str, len);
+ log_point += len;
+
+ if (log_point == LOG_DUMP_SIZE) {
+ log_point = 0;
+ log_wrap = 1;
+ }
+ return;
+ }
+
for (i = 0; i < len; i++) {
log_dump[log_point++] = log_str[i];
- if (log_point == SM_LOG_DUMP_SIZE) {
+ if (log_point == LOG_DUMP_SIZE) {
log_point = 0;
log_wrap = 1;
}
@@ -175,17 +186,24 @@ static void write_dropped(int level, int num)
write_entry(level, str);
}
-void write_log_dump(int fd)
+void copy_log_dump(char *buf, int *len)
{
- pthread_mutex_lock(&log_mutex);
-
- if (log_wrap)
- send(fd, log_dump + log_point, SM_LOG_DUMP_SIZE - log_point, MSG_DONTWAIT);
+ int tail_len;
- log_dump[log_point] = '\0';
-
- send(fd, log_dump, log_point, MSG_DONTWAIT);
+ pthread_mutex_lock(&log_mutex);
+ if (!log_wrap && !log_point) {
+ *len = 0;
+ } else if (log_wrap) {
+ tail_len = LOG_DUMP_SIZE - log_point;
+ memcpy(buf, log_dump+log_point, tail_len);
+ if (log_point)
+ memcpy(buf+tail_len, log_dump, log_point);
+ *len = LOG_DUMP_SIZE;
+ } else {
+ memcpy(buf, log_dump, log_point-1);
+ *len = log_point-1;
+ }
pthread_mutex_unlock(&log_mutex);
}
diff --git a/src/log.h b/src/log.h
index 8661eb0..63168eb 100644
--- a/src/log.h
+++ b/src/log.h
@@ -14,7 +14,7 @@ void log_level(uint32_t space_id, uint32_t token_id, char *name_in, int level, c
int setup_logging(void);
void close_logging(void);
-void write_log_dump(int fd);
+void copy_log_dump(char *buf, int *len);
#define log_debug(fmt, args...) log_level(0, 0, NULL, LOG_DEBUG, fmt, ##args)
#define log_space(space, fmt, args...) log_level(space->space_id, 0, NULL, LOG_DEBUG, fmt, ##args)
diff --git a/src/main.c b/src/main.c
index 3034f47..e5aa9c6 100644
--- a/src/main.c
+++ b/src/main.c
@@ -1708,7 +1708,7 @@ static int do_client(void)
break;
case ACT_LOG_DUMP:
- rv = sanlock_log_dump();
+ rv = sanlock_log_dump(LOG_DUMP_SIZE);
break;
case ACT_SHUTDOWN:
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index 24ceb2f..46e2b36 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -35,7 +35,7 @@
#define DEFAULT_MAX_HOSTS 2000
-#define SM_LOG_DUMP_SIZE (1024*1024)
+#define LOG_DUMP_SIZE (1024*1024)
/* this is just the path to the executable, not full command line */
12 years, 2 months
src/cmd.c src/main.c src/sanlock_internal.h src/sanlock_resource.h src/task.c
by David Teigland
src/cmd.c | 6 +-
src/main.c | 129 +++++++++++++++++++++++++------------------------
src/sanlock_internal.h | 7 +-
src/sanlock_resource.h | 1
src/task.c | 14 ++++-
5 files changed, 88 insertions(+), 69 deletions(-)
New commits:
commit ba4b71a6cd7b27ea3adcd011916b4b93b6da253e
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Sep 15 16:23:18 2011 -0500
sanlock: improve killing pids
try to kill a pid once a second for a configured
max number of attempts (60), after which we give up.
The first 10 attempts use SIGTERM (if not restricted),
after which we use SIGKILL (if not restricted).
diff --git a/src/cmd.c b/src/cmd.c
index 4810876..bbab319 100644
--- a/src/cmd.c
+++ b/src/cmd.c
@@ -1219,7 +1219,8 @@ static int print_state_client(struct client *cl, int ci, char *str)
"cmd_active=%d "
"cmd_last=%d "
"pid_dead=%d "
- "killing=%d "
+ "kill_count=%d "
+ "kill_last=%llu "
"suspend=%d "
"need_free=%d",
ci,
@@ -1229,7 +1230,8 @@ static int print_state_client(struct client *cl, int ci, char *str)
cl->cmd_active,
cl->cmd_last,
cl->pid_dead,
- cl->killing,
+ cl->kill_count,
+ (unsigned long long)cl->kill_last,
cl->suspend,
cl->need_free);
diff --git a/src/main.c b/src/main.c
index 25f2b09..3034f47 100644
--- a/src/main.c
+++ b/src/main.c
@@ -150,7 +150,8 @@ static void _client_free(int ci)
cl->pid_dead = 0;
cl->suspend = 0;
cl->need_free = 0;
- cl->killing = 0;
+ cl->kill_count = 0;
+ cl->kill_last = 0;
cl->restrict = 0;
memset(cl->owner_name, 0, sizeof(cl->owner_name));
cl->workfn = NULL;
@@ -417,7 +418,7 @@ static int client_using_space(struct client *cl, struct space *sp)
if (strncmp(token->r.lockspace_name, sp->space_name, NAME_ID_SIZE))
continue;
- if (cl->killing < 10)
+ if (!cl->kill_count)
log_spoke(sp, token, "client_using_space pid %d", cl->pid);
token->space_dead = sp->space_dead;
rv = 1;
@@ -425,92 +426,89 @@ static int client_using_space(struct client *cl, struct space *sp)
return rv;
}
-/*
- * TODO: move to time-based kill tracking instead of repetition based, e.g.
- * kill pid once a second until a max pid_exit_wait after which we quit
- * trying to kill it. half way through pid_exit_wait shift from SIGTERM
- * to SIGKILL (if no restriction).
- *
- * sp->killing pids begins at 1
- * don't increment sp->killing pids each time
- * when all clients have gone past pid_exit_wait, set sp->killing_pids to 2
- * when sp->killing_pids is 2, then return immediately from kill_pids()
- *
- * Remove the usleep delay after a kill.
- */
+/* TODO: try killscript first if one is provided */
static void kill_pids(struct space *sp)
{
struct client *cl;
- int ci, pid;
- int sig;
- int found = 0;
-
- /* TODO: try killscript first if one is provided */
+ uint64_t now;
+ int ci, fd, pid, sig;
+ int do_kill;
- if (sp->killing_pids > 11)
+ /*
+ * all remaining pids using sp are stuck, we've made max attempts to
+ * kill all, don't bother cycling through them
+ */
+ if (sp->killing_pids > 1)
return;
- log_space(sp, "kill_pids %d", sp->killing_pids);
-
- if (sp->killing_pids > 10) {
- sp->killing_pids++;
- goto do_dump;
- }
-
- sp->killing_pids++;
+ now = monotime();
for (ci = 0; ci <= client_maxi; ci++) {
- pid = -1;
+ do_kill = 0;
cl = &client[ci];
pthread_mutex_lock(&cl->mutex);
if (!cl->used)
goto unlock;
+
if (cl->pid <= 0)
goto unlock;
+
+ /* NB this cl may not be using sp, but trying to
+ avoid the expensive client_using_space check */
+
+ if (cl->kill_count >= main_task.kill_count_max)
+ goto unlock;
+
+ if (cl->kill_count && (now - cl->kill_last < 1))
+ goto unlock;
+
if (!client_using_space(cl, sp))
goto unlock;
- if (!(cl->restrict & SANLK_RESTRICT_SIGKILL) && cl->killing > 1)
+ cl->kill_last = now;
+ cl->kill_count++;
+
+ fd = cl->fd;
+ pid = cl->pid;
+
+ if (cl->restrict & SANLK_RESTRICT_SIGKILL)
+ sig = SIGTERM;
+ else if (cl->restrict & SANLK_RESTRICT_SIGTERM)
sig = SIGKILL;
- else
+ else if (cl->kill_count <= main_task.kill_count_term)
sig = SIGTERM;
+ else
+ sig = SIGKILL;
- pid = cl->pid;
- cl->killing++;
- found++;
+ do_kill = 1;
unlock:
pthread_mutex_unlock(&cl->mutex);
- if (pid > 0)
- kill(pid, sig);
- }
-
- if (found) {
- log_space(sp, "kill_pids %d found %d pids", sig, found);
- usleep(500000);
- }
-
- return;
+ if (!do_kill)
+ continue;
- do_dump:
- for (ci = 0; ci <= client_maxi; ci++) {
- if (client[ci].pid && client[ci].killing) {
- log_error("kill_pids %d stuck", client[ci].pid);
+ if (cl->kill_count == main_task.kill_count_max) {
+ log_erros(sp, "kill %d,%d,%d sig %d count %d final attempt",
+ ci, fd, pid, sig, cl->kill_count);
+ } else {
+ log_space(sp, "kill %d,%d,%d sig %d count %d",
+ ci, fd, pid, sig, cl->kill_count);
}
+
+ kill(pid, sig);
}
}
static int all_pids_dead(struct space *sp)
{
struct client *cl;
- int ci, pid;
+ int stuck = 0, check = 0;
+ int ci;
for (ci = 0; ci <= client_maxi; ci++) {
- pid = -1;
-
cl = &client[ci];
pthread_mutex_lock(&cl->mutex);
@@ -521,18 +519,23 @@ static int all_pids_dead(struct space *sp)
if (!client_using_space(cl, sp))
goto unlock;
- pid = cl->pid;
+ if (cl->kill_count >= main_task.kill_count_max)
+ stuck++;
+ else
+ check++;
unlock:
pthread_mutex_unlock(&cl->mutex);
+ }
- if (pid > 0) {
- if (cl->killing < 10) {
- log_space(sp, "used by pid %d killing %d",
- pid, cl->killing);
- }
- return 0;
- }
+ if (stuck && !check && sp->killing_pids < 2) {
+ log_erros(sp, "killing pids stuck %d", stuck);
+ /* cause kill_pids to give up */
+ sp->killing_pids = 2;
}
+
+ if (stuck || check)
+ return 0;
+
log_space(sp, "used by no pids");
return 1;
}
@@ -887,9 +890,9 @@ static void process_cmd_thread_registered(int ci_in, struct sm_header *h_recv)
goto out;
}
- if (cl->killing) {
- log_error("cmd %d %d,%d,%d killing",
- h_recv->cmd, ci_target, cl->fd, cl->pid);
+ if (cl->kill_count) {
+ log_error("cmd %d %d,%d,%d kill_count %d",
+ h_recv->cmd, ci_target, cl->fd, cl->pid, cl->kill_count);
result = -EBUSY;
goto out;
}
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index fb13afb..24ceb2f 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -450,10 +450,10 @@ struct task {
int id_renewal_seconds; /* configured */
int id_renewal_fail_seconds; /* configured */
int id_renewal_warn_seconds; /* configured */
-
int host_dead_seconds; /* calculated */
-
int request_finish_seconds; /* calculated */
+ int kill_count_term; /* constant */
+ int kill_count_max; /* constant */
unsigned int io_count; /* stats */
unsigned int to_count; /* stats */
@@ -477,8 +477,9 @@ struct client {
int pid_dead;
int suspend;
int need_free;
- int killing;
+ int kill_count;
uint32_t restrict;
+ uint64_t kill_last;
char owner_name[SANLK_NAME_LEN+1];
pthread_mutex_t mutex;
void *workfn;
diff --git a/src/sanlock_resource.h b/src/sanlock_resource.h
index ae8dfa4..b86bde0 100644
--- a/src/sanlock_resource.h
+++ b/src/sanlock_resource.h
@@ -23,6 +23,7 @@
/* restrict flags */
#define SANLK_RESTRICT_ALL 0x00000001
#define SANLK_RESTRICT_SIGKILL 0x00000002
+#define SANLK_RESTRICT_SIGTERM 0x00000004
/* release flags */
#define SANLK_REL_ALL 0x00000001
diff --git a/src/task.c b/src/task.c
index 9635f92..e06a6fa 100644
--- a/src/task.c
+++ b/src/task.c
@@ -53,13 +53,24 @@ void setup_task_timeouts(struct task *task, int io_timeout_arg)
int paxos_acquire_held_min = host_dead_seconds;
int paxos_acquire_free_max = 6 * io_timeout_seconds;
int paxos_acquire_free_min = 0;
+ int request_finish_seconds = 3 * id_renewal_seconds; /* random */
task->io_timeout_seconds = io_timeout_seconds;
task->id_renewal_seconds = id_renewal_seconds;
task->id_renewal_fail_seconds = id_renewal_fail_seconds;
task->id_renewal_warn_seconds = id_renewal_warn_seconds;
task->host_dead_seconds = host_dead_seconds;
- task->request_finish_seconds = 3 * id_renewal_seconds; /* random */
+ task->request_finish_seconds = request_finish_seconds;
+
+ /* interval between each kill count is approx 1 sec, so we
+ spend about 10 seconds sending 10 SIGTERMs to a pid,
+ then send SIGKILLs to it. after 60 attempts the watchdog
+ should have fired if the kills are due to failed renewal;
+ otherwise we just give up at that point */
+
+ task->kill_count_term = 10;
+ task->kill_count_max = 60;
+
/* the rest are calculated as needed in place */
/* hack to make just main thread log this info */
@@ -84,6 +95,7 @@ void setup_task_timeouts(struct task *task, int io_timeout_arg)
log_debug("paxos_acquire_held_min %d", paxos_acquire_held_min);
log_debug("paxos_acquire_free_max %d", paxos_acquire_free_max);
log_debug("paxos_acquire_free_min %d", paxos_acquire_free_min);
+ log_debug("request_finish_seconds %d", request_finish_seconds);
}
void setup_task_aio(struct task *task, int use_aio, int cb_size)
12 years, 2 months
2 commits - src/main.c src/sanlock_internal.h src/sanlock_resource.h src/token_manager.c src/token_manager.h tests/devcount.c
by David Teigland
src/main.c | 23 +++++++++-----
src/sanlock_internal.h | 11 +++---
src/sanlock_resource.h | 1
src/token_manager.c | 80 ++++++++++++++++++++++++++++++++-----------------
src/token_manager.h | 2 -
tests/devcount.c | 25 +++++++++++++++
6 files changed, 101 insertions(+), 41 deletions(-)
New commits:
commit e3077bf925432aa852f6eb4f3eb552c21a09d353
Author: David Teigland <teigland(a)redhat.com>
Date: Wed Sep 14 13:50:33 2011 -0500
sanlock: setup fewer aio events
since there is a system limit on these, we'd unnecessarily
cause io_setup to fail and not use aio at all. We really
shouldn't need many anyway.
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index 0bd3d4c..6e7aae6 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -431,11 +431,11 @@ struct space {
* The delta lease algorithm expects real io timeouts.
*/
-#define HOSTID_AIO_CB_SIZE 64
-#define WORKER_AIO_CB_SIZE 8
-#define DIRECT_AIO_CB_SIZE 8
-#define RESOURCE_AIO_CB_SIZE 64
-#define LIB_AIO_CB_SIZE 8
+#define HOSTID_AIO_CB_SIZE 4
+#define WORKER_AIO_CB_SIZE 2
+#define DIRECT_AIO_CB_SIZE 1
+#define RESOURCE_AIO_CB_SIZE 2
+#define LIB_AIO_CB_SIZE 1
struct aicb {
int used;
commit dd6791b3844e16144fbe81330371f5d4f4847757
Author: David Teigland <teigland(a)redhat.com>
Date: Wed Sep 14 13:24:51 2011 -0500
sanlock: add flag RESTRICT_SIGKILL
which causes sanlock to only use SIGTERM to kill pid,
not to escalate to SIGKILL. A process may be holding
a lease based on state outside itself; the fact that
the pid exits does not imply it's safe to release
the lease. The pid would choose to ignore the SIGTERM
if it's not safe to release the lease, and using
SIGKILL makes it impossible for the pid to make that
choice.
diff --git a/src/main.c b/src/main.c
index d3d0c4f..2d6eeee 100644
--- a/src/main.c
+++ b/src/main.c
@@ -66,7 +66,7 @@ struct client {
int suspend;
int need_free;
int killing;
- uint32_t cmd_flags;
+ uint32_t restrict;
char owner_name[SANLK_NAME_LEN+1];
pthread_mutex_t mutex;
void *workfn;
@@ -183,7 +183,7 @@ static void _client_free(int ci)
cl->suspend = 0;
cl->need_free = 0;
cl->killing = 0;
- cl->cmd_flags = 0;
+ cl->restrict = 0;
memset(cl->owner_name, 0, sizeof(cl->owner_name));
cl->workfn = NULL;
cl->deadfn = NULL;
@@ -432,7 +432,7 @@ static void kill_pids(struct space *sp)
{
struct client *cl;
int ci, pid;
- int sig = SIGTERM;
+ int sig;
int found = 0;
log_space(sp, "kill_pids %d", sp->killing_pids);
@@ -447,8 +447,6 @@ static void kill_pids(struct space *sp)
goto do_dump;
}
- if (sp->killing_pids > 1)
- sig = SIGKILL;
sp->killing_pids++;
for (ci = 0; ci <= client_maxi; ci++) {
@@ -464,6 +462,11 @@ static void kill_pids(struct space *sp)
if (!client_using_space(cl, sp))
goto unlock;
+ if (!(cl->restrict & SANLK_RESTRICT_SIGKILL) && cl->killing > 1)
+ sig = SIGKILL;
+ else
+ sig = SIGTERM;
+
pid = cl->pid;
cl->killing++;
found++;
@@ -777,6 +780,7 @@ static void cmd_acquire(struct task *task, struct cmd_args *ca)
char *opt_str;
uint64_t acquire_lver = 0;
uint32_t new_num_hosts = 0;
+ uint32_t cl_restrict;
int token_len, disks_len;
int fd, rv, i, j, empty_slots;
int alloc_count = 0, add_count = 0, acquire_count = 0;
@@ -809,6 +813,7 @@ static void cmd_acquire(struct task *task, struct cmd_args *ca)
pthread_mutex_unlock(&cl->mutex);
goto done;
}
+ cl_restrict = cl->restrict;
empty_slots = 0;
for (i = 0; i < SANLK_MAX_RESOURCES; i++) {
@@ -962,7 +967,7 @@ static void cmd_acquire(struct task *task, struct cmd_args *ca)
for (i = 0; i < new_tokens_count; i++) {
token = new_tokens[i];
- rv = add_resource(token, cl_pid);
+ rv = add_resource(token, cl_pid, cl_restrict);
if (rv < 0) {
if (!com.quiet_fail)
log_errot(token, "cmd_acquire %d,%d,%d add_resource %d",
@@ -1967,6 +1972,7 @@ static int print_state_client(struct client *cl, int ci, char *str)
"ci=%d "
"fd=%d "
"pid=%d "
+ "restrict=%x "
"cmd_active=%d "
"cmd_last=%d "
"pid_dead=%d "
@@ -1976,6 +1982,7 @@ static int print_state_client(struct client *cl, int ci, char *str)
ci,
cl->fd,
cl->pid,
+ cl->restrict,
cl->cmd_active,
cl->cmd_last,
cl->pid_dead,
@@ -2297,7 +2304,7 @@ static void cmd_restrict(int ci, int fd, struct sm_header *h_recv)
log_debug("cmd_restrict ci %d fd %d pid %d flags %x",
ci, fd, client[ci].pid, h_recv->cmd_flags);
- client[ci].cmd_flags = h_recv->cmd_flags;
+ client[ci].restrict = h_recv->cmd_flags;
send_result(fd, h_recv, 0);
}
@@ -2532,7 +2539,7 @@ static void process_connection(int ci)
ci, rv, h.magic, SM_MAGIC);
goto dead;
}
- if (client[ci].cmd_flags & SANLK_RESTRICT_ALL) {
+ if (client[ci].restrict & SANLK_RESTRICT_ALL) {
log_error("ci %d fd %d pid %d cmd %d restrict all",
ci, client[ci].fd, client[ci].pid, h.cmd);
goto dead;
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index 0b39934..0bd3d4c 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -90,6 +90,7 @@ struct token {
};
#define R_EXAMINE 0x00000001
+#define R_NO_SIGKILL 0x00000002
struct resource {
struct list_head list;
diff --git a/src/sanlock_resource.h b/src/sanlock_resource.h
index 770ebfc..ae8dfa4 100644
--- a/src/sanlock_resource.h
+++ b/src/sanlock_resource.h
@@ -22,6 +22,7 @@
/* restrict flags */
#define SANLK_RESTRICT_ALL 0x00000001
+#define SANLK_RESTRICT_SIGKILL 0x00000002
/* release flags */
#define SANLK_REL_ALL 0x00000001
diff --git a/src/token_manager.c b/src/token_manager.c
index 33de66a..824a202 100644
--- a/src/token_manager.c
+++ b/src/token_manager.c
@@ -98,7 +98,7 @@ static void save_resource_lver(struct token *token, uint64_t lver)
}
-int add_resource(struct token *token, int pid)
+int add_resource(struct token *token, int pid, uint32_t cl_restrict)
{
struct resource *r;
int rv, disks_len, r_len;
@@ -135,6 +135,8 @@ int add_resource(struct token *token, int pid)
r->token_id = token->token_id;
r->token = token;
r->pid = pid;
+ if (cl_restrict & SANLK_RESTRICT_SIGKILL)
+ r->flags |= R_NO_SIGKILL;
list_add_tail(&r->list, &resources);
rv = 0;
out:
@@ -343,6 +345,55 @@ static int examine_token(struct task *task, struct token *token,
return rv;
}
+static void do_req_kill_pid(struct token *tt, int pid)
+{
+ struct resource *r;
+ uint32_t flags;
+ int found = 0;
+
+ pthread_mutex_lock(&resource_mutex);
+ r = find_resource(tt, &resources);
+ if (r && r->pid == pid) {
+ found = 1;
+ flags = r->flags;
+ }
+ pthread_mutex_unlock(&resource_mutex);
+
+ if (!found) {
+ log_error("req pid %d %.48s:%.48s not found",
+ pid, tt->r.lockspace_name, tt->r.name);
+ return;
+ }
+
+ log_debug("do_req_kill_pid %d flags %x %.48s:%.48s",
+ pid, flags, tt->r.lockspace_name, tt->r.name);
+
+ /* TODO: share code with kill_pids() to gradually
+ * escalate from killscript, SIGTERM, SIGKILL */
+
+ kill(pid, SIGTERM);
+
+ if (flags & R_NO_SIGKILL)
+ return;
+
+ sleep(1);
+ kill(pid, SIGKILL);
+}
+
+/*
+ * TODO? add force_mode SANLK_REQ_KILL_PID_OR_RESET
+ * which would attempt to kill the pid like KILL_PID,
+ * but if the pid doesn't exit will block watchdog
+ * updates to reset the host.
+ * Here set r->block_wd_time = now + pid_exit_time,
+ * In renewal check for any r in resources with
+ * block_wd_time <= now, and if found will not
+ * update the watchdog. If the pid continues to
+ * not exit, the wd will fire and reset the machine.
+ * If the pid exits before pid_exit_time, no wd
+ * updates will be skipped.
+ */
+
/*
* - releases tokens of pid's that die
* - examines request blocks of resources
@@ -440,33 +491,8 @@ static void *resource_thread(void *arg GNUC_UNUSED)
continue;
}
- /*
- * TODO: add force_mode SANLK_REQ_KILL_PID_OR_RESET
- * which would attempt to kill the pid like KILL_PID,
- * but if the pid doesn't exit will block watchdog
- * updates to reset the host.
- * Here set r->block_wd_time = now + pid_exit_time,
- * In renewal check for any r in resources with
- * block_wd_time <= now, and if found will not
- * update the watchdog. If the pid continues to
- * not exit, the wd will fire and reset the machine.
- * If the pid exits before pid_exit_time, no wd
- * updates will be skipped.
- */
-
if (req.force_mode == SANLK_REQ_KILL_PID) {
- /* look up r again? verify it exists and pid same */
-
- log_debug("req force_mode %u pid %d %.48s:%.48s",
- req.force_mode, pid,
- tt->r.lockspace_name, tt->r.name);
-
- /* TODO: share code with kill_pids() to gradually
- * escalate from killscript, SIGTERM, SIGKILL */
-
- kill(pid, SIGTERM);
- sleep(1);
- kill(pid, SIGKILL);
+ do_req_kill_pid(tt, pid);
} else {
log_error("req force_mode %u unknown", req.force_mode);
}
diff --git a/src/token_manager.h b/src/token_manager.h
index da34baa..751b636 100644
--- a/src/token_manager.h
+++ b/src/token_manager.h
@@ -19,7 +19,7 @@ void release_token_async(struct token *token);
int request_token(struct task *task, struct token *token, uint32_t force_mode,
uint64_t *owner_id);
-int add_resource(struct token *token, int pid);
+int add_resource(struct token *token, int pid, uint32_t cl_restrict);
void del_resource(struct token *token);
int set_resource_examine(char *space_name, char *res_name);
diff --git a/tests/devcount.c b/tests/devcount.c
index 9db5a43..6c51c6c 100644
--- a/tests/devcount.c
+++ b/tests/devcount.c
@@ -58,6 +58,18 @@ do { \
printf("ERROR %llu " fmt "\n", (unsigned long long)time(NULL), ##args); \
} while (0)
+static void sigterm_handler(int sig)
+{
+ log_debug("sigterm_handler %d", sig);
+}
+
+static void setup_sigterm(void)
+{
+ struct sigaction act;
+ memset(&act, 0, sizeof(act));
+ act.sa_handler = sigterm_handler;
+ sigaction(SIGTERM, &act, NULL);
+}
static int kill_pid(int pid)
{
@@ -836,6 +848,13 @@ static int do_wrap(int argc, char *argv[])
exit(-1);
}
+ rv = sanlock_restrict(sock, SANLK_RESTRICT_SIGKILL);
+ if (rv < 0) {
+ log_error("%s c %d sanlock_restrict error %d",
+ count_path, pid, sock);
+ exit(-1);
+ }
+
rv = sanlock_acquire(sock, -1, 0, 1, &res, NULL);
if (rv < 0) {
log_error("%s c %d sanlock_acquire error %d",
@@ -1489,6 +1508,12 @@ int main(int argc, char *argv[])
else if (!strcmp(argv[1], "rw") || !strcmp(argv[1], "wr"))
rv = do_count(argc, argv);
+ else if (!strcmp(argv[1], "rwsig")) {
+ setup_sigterm();
+ argv[1] = "rw";
+ rv = do_count(argc, argv);
+ }
+
else if (!strcmp(argv[1], "lock"))
rv = do_lock(argc, argv);
12 years, 2 months
src/host_id.c src/token_manager.c
by David Teigland
src/host_id.c | 5 ++++-
src/token_manager.c | 28 ++++++++++++++++++++++++----
2 files changed, 28 insertions(+), 5 deletions(-)
New commits:
commit 3d14379e4aea8eef07cfd0ff7dee586a89ef4b9d
Author: David Teigland <teigland(a)redhat.com>
Date: Tue Sep 13 14:40:34 2011 -0500
sanlock: SIGTERM and SIGKILL for REQ_KILL_PID
and add some comments about another req mode
diff --git a/src/host_id.c b/src/host_id.c
index 4d39d82..18ef28d 100644
--- a/src/host_id.c
+++ b/src/host_id.c
@@ -519,6 +519,7 @@ static void *lockspace_thread(void *arg_in)
/*
* pet the watchdog
+ * (don't update on thread_stop because it's probably unlinked)
*/
if (delta_result == SANLK_OK && !sp->thread_stop)
@@ -543,7 +544,9 @@ static void *lockspace_thread(void *arg_in)
}
}
- /* unlink called below to get it done ASAP */
+ /* watchdog unlink was done in main_loop when thread_stop was set, to
+ get it done as quickly as possible in case the wd is about to fire. */
+
close_watchdog_file(sp);
out:
if (delta_result == SANLK_OK)
diff --git a/src/token_manager.c b/src/token_manager.c
index 6568af3..33de66a 100644
--- a/src/token_manager.c
+++ b/src/token_manager.c
@@ -440,15 +440,35 @@ static void *resource_thread(void *arg GNUC_UNUSED)
continue;
}
+ /*
+ * TODO: add force_mode SANLK_REQ_KILL_PID_OR_RESET
+ * which would attempt to kill the pid like KILL_PID,
+ * but if the pid doesn't exit will block watchdog
+ * updates to reset the host.
+ * Here set r->block_wd_time = now + pid_exit_time,
+ * In renewal check for any r in resources with
+ * block_wd_time <= now, and if found will not
+ * update the watchdog. If the pid continues to
+ * not exit, the wd will fire and reset the machine.
+ * If the pid exits before pid_exit_time, no wd
+ * updates will be skipped.
+ */
+
if (req.force_mode == SANLK_REQ_KILL_PID) {
- /* look up r again to check it still exists and
- pid is same? */
+ /* look up r again? verify it exists and pid same */
- log_error("req_kill_pid %d %.48s:%.48s", pid,
+ log_debug("req force_mode %u pid %d %.48s:%.48s",
+ req.force_mode, pid,
tt->r.lockspace_name, tt->r.name);
+
+ /* TODO: share code with kill_pids() to gradually
+ * escalate from killscript, SIGTERM, SIGKILL */
+
+ kill(pid, SIGTERM);
+ sleep(1);
kill(pid, SIGKILL);
} else {
- log_error("req force_mode unknown %u", req.force_mode);
+ log_error("req force_mode %u unknown", req.force_mode);
}
}
}
12 years, 2 months
2 commits - src/client_cmd.c src/client_cmd.h src/main.c src/sanlock.8 src/sanlock_internal.h
by David Teigland
src/client_cmd.c | 336 +++++++++++++++++++++++++++++++++++++++++--------
src/client_cmd.h | 2
src/main.c | 32 +++-
src/sanlock.8 | 1
src/sanlock_internal.h | 1
5 files changed, 311 insertions(+), 61 deletions(-)
New commits:
commit 758e7c83b6bdd3e8a8203de6b541c38385ad9196
Author: David Teigland <teigland(a)redhat.com>
Date: Tue Sep 13 11:39:23 2011 -0500
sanlock: status output sorting
-o p|s to sort resources by pid/lockspace.
Also fix host_status bug when -s not found.
diff --git a/src/client_cmd.c b/src/client_cmd.c
index 73404bf..119149f 100644
--- a/src/client_cmd.c
+++ b/src/client_cmd.c
@@ -56,16 +56,15 @@ static void print_debug(char *str, int len)
printf(" %s\n", p);
}
-static void status_daemon(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
+static void status_daemon(struct sanlk_state *st, char *str, int debug)
{
printf("daemon %.48s\n", st->name);
- if (st->str_len && debug) {
+ if (st->str_len && debug)
print_debug(str, st->str_len);
- }
}
-static void status_client(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
+static void status_client(struct sanlk_state *st, char *str, int debug)
{
printf("p %d ", st->data32);
printf("%.48s\n", st->name);
@@ -74,38 +73,33 @@ static void status_client(int fd GNUC_UNUSED, struct sanlk_state *st, char *str,
print_debug(str, st->str_len);
}
-static void status_lockspace(int fd, struct sanlk_state *st, char *str, int debug)
+static void status_lockspace(struct sanlk_state *st, char *str, char *bin, int debug)
{
- struct sanlk_lockspace lockspace;
- int rv;
-
- rv = recv(fd, &lockspace, sizeof(lockspace), MSG_WAITALL);
+ struct sanlk_lockspace *ls = (struct sanlk_lockspace *)bin;
printf("s %.48s:%llu:%s:%llu\n",
- lockspace.name,
- (unsigned long long)lockspace.host_id,
- lockspace.host_id_disk.path,
- (unsigned long long)lockspace.host_id_disk.offset);
+ ls->name,
+ (unsigned long long)ls->host_id,
+ ls->host_id_disk.path,
+ (unsigned long long)ls->host_id_disk.offset);
if (st->str_len && debug)
print_debug(str, st->str_len);
}
-static void status_resource(int fd, struct sanlk_state *st, char *str, int debug)
+static void status_resource(struct sanlk_state *st, char *str, char *bin, int debug)
{
- struct sanlk_resource resource;
- struct sanlk_disk disk;
- int i, rv;
-
- rv = recv(fd, &resource, sizeof(resource), MSG_WAITALL);
+ struct sanlk_resource *res = (struct sanlk_resource *)bin;
+ struct sanlk_disk *disk;
+ int i;
- printf("r %.48s:%.48s", resource.lockspace_name, resource.name);
+ printf("r %.48s:%.48s", res->lockspace_name, res->name);
- for (i = 0; i < resource.num_disks; i++) {
- rv = recv(fd, &disk, sizeof(disk), MSG_WAITALL);
+ for (i = 0; i < res->num_disks; i++) {
+ disk = (struct sanlk_disk *)(bin + sizeof(struct sanlk_resource) + i * sizeof(struct sanlk_disk));
printf(":%s:%llu",
- disk.path, (unsigned long long)disk.offset);
+ disk->path, (unsigned long long)disk->offset);
}
printf(":%llu p %u\n", (unsigned long long)st->data64, st->data32);
@@ -113,7 +107,7 @@ static void status_resource(int fd, struct sanlk_state *st, char *str, int debug
print_debug(str, st->str_len);
}
-static void status_host(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
+static void status_host(struct sanlk_state *st, char *str, int debug)
{
printf("%u timestamp %llu\n", st->data32,
(unsigned long long)st->data64);
@@ -122,12 +116,213 @@ static void status_host(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, i
print_debug(str, st->str_len);
}
-int sanlock_status(int debug)
+static void print_st(struct sanlk_state *st, char *str, char *bin, int debug)
+{
+ switch (st->type) {
+ case SANLK_STATE_DAEMON:
+ status_daemon(st, str, debug);
+ break;
+ case SANLK_STATE_CLIENT:
+ status_client(st, str, debug);
+ break;
+ case SANLK_STATE_LOCKSPACE:
+ status_lockspace(st, str, bin, debug);
+ break;
+ case SANLK_STATE_RESOURCE:
+ status_resource(st, str, bin, debug);
+ break;
+ }
+}
+
+#define MAX_SORT_ENTRIES 1024
+static char *sort_bufs[MAX_SORT_ENTRIES];
+static int sort_count;
+static int sort_done;
+
+static void print_type(int type, int debug)
+{
+ struct sanlk_state *st;
+ char *buf, *str, *bin;
+ int i;
+
+ for (i = 0; i < sort_count; i++) {
+ buf = sort_bufs[i];
+ if (!buf)
+ continue;
+ st = (struct sanlk_state *)buf;
+ str = buf + sizeof(struct sanlk_state);
+ bin = buf + sizeof(struct sanlk_state) + SANLK_STATE_MAXSTR;
+
+ if (!type || st->type == type) {
+ print_st(st, str, bin, debug);
+ free(buf);
+ sort_bufs[i] = NULL;
+ sort_done++;
+ }
+ }
+}
+
+static void print_p(int p, int debug)
+{
+ struct sanlk_state *st;
+ char *buf, *str, *bin;
+ int i;
+
+ for (i = 0; i < sort_count; i++) {
+ buf = sort_bufs[i];
+ if (!buf)
+ continue;
+ st = (struct sanlk_state *)buf;
+ str = buf + sizeof(struct sanlk_state);
+ bin = buf + sizeof(struct sanlk_state) + SANLK_STATE_MAXSTR;
+
+ if (st->type != SANLK_STATE_CLIENT)
+ continue;
+
+ if (st->data32 == p) {
+ print_st(st, str, bin, debug);
+ free(buf);
+ sort_bufs[i] = NULL;
+ sort_done++;
+ }
+ }
+}
+
+static int find_type(int type, int *sort_index)
+{
+ struct sanlk_state *st;
+ char *buf;
+ int i;
+
+ for (i = 0; i < sort_count; i++) {
+ buf = sort_bufs[i];
+ if (!buf)
+ continue;
+ st = (struct sanlk_state *)buf;
+
+ if (st->type == type) {
+ *sort_index = i;
+ return 0;
+ }
+ }
+ return -1;
+}
+
+static void print_r(int p, char *s, int debug)
+{
+ struct sanlk_resource *res;
+ struct sanlk_state *st;
+ char *buf, *str, *bin;
+ int i;
+
+ for (i = 0; i < sort_count; i++) {
+ buf = sort_bufs[i];
+ if (!buf)
+ continue;
+ st = (struct sanlk_state *)buf;
+ str = buf + sizeof(struct sanlk_state);
+ bin = buf + sizeof(struct sanlk_state) + SANLK_STATE_MAXSTR;
+
+ if (st->type != SANLK_STATE_RESOURCE)
+ continue;
+
+ res = (struct sanlk_resource *)bin;
+
+ if ((p && st->data32 == p) ||
+ (s && !strncmp(s, res->lockspace_name, SANLK_NAME_LEN))) {
+ print_st(st, str, bin, debug);
+ free(buf);
+ sort_bufs[i] = NULL;
+ sort_done++;
+ }
+ }
+}
+
+static void print_r_by_p(int debug)
+{
+ struct sanlk_state *st;
+ char *buf, *str, *bin;
+ int rv, i;
+
+ while (1) {
+ rv = find_type(SANLK_STATE_CLIENT, &i);
+ if (rv < 0)
+ return;
+
+ buf = sort_bufs[i];
+ st = (struct sanlk_state *)buf;
+ str = buf + sizeof(struct sanlk_state);
+ bin = buf + sizeof(struct sanlk_state) + SANLK_STATE_MAXSTR;
+
+ print_st(st, str, bin, debug);
+
+ print_r(st->data32, NULL, debug);
+
+ free(buf);
+ sort_bufs[i] = NULL;
+ sort_done++;
+ }
+}
+
+static void print_r_by_s(int debug)
+{
+ struct sanlk_state *st;
+ char *buf, *str, *bin;
+ int rv, i;
+
+ while (1) {
+ rv = find_type(SANLK_STATE_LOCKSPACE, &i);
+ if (rv < 0)
+ return;
+
+ buf = sort_bufs[i];
+ st = (struct sanlk_state *)buf;
+ str = buf + sizeof(struct sanlk_state);
+ bin = buf + sizeof(struct sanlk_state) + SANLK_STATE_MAXSTR;
+
+ print_st(st, str, bin, debug);
+
+ print_r(0, st->name, debug);
+
+ free(buf);
+ sort_bufs[i] = NULL;
+ sort_done++;
+ }
+}
+
+static void recv_bin(int fd, struct sanlk_state *st, char *bin)
+{
+ struct sanlk_resource *res;
+
+ if (st->type == SANLK_STATE_LOCKSPACE) {
+ recv(fd, bin, sizeof(struct sanlk_lockspace), MSG_WAITALL);
+
+ } else if (st->type == SANLK_STATE_RESOURCE) {
+ recv(fd, bin, sizeof(struct sanlk_resource), MSG_WAITALL);
+
+ res = (struct sanlk_resource *)bin;
+
+ recv(fd, bin+sizeof(struct sanlk_resource),
+ res->num_disks * sizeof(struct sanlk_disk),
+ MSG_WAITALL);
+ }
+}
+
+int sanlock_status(int debug, char sort_arg)
{
struct sm_header h;
- struct sanlk_state st;
- char str[SANLK_STATE_MAXSTR];
- int fd, rv;
+ struct sanlk_state state;
+ char maxstr[SANLK_STATE_MAXSTR];
+ char maxbin[SANLK_STATE_MAXSTR];
+ struct sanlk_state *st;
+ char *buf, *str, *bin;
+ int fd, rv, len;
+ int sort_p = 0, sort_s = 0;
+
+ if (sort_arg == 'p')
+ sort_p = 1;
+ else if (sort_arg == 's')
+ sort_s = 1;
fd = send_command(SM_CMD_STATUS, 0);
if (fd < 0)
@@ -143,34 +338,74 @@ int sanlock_status(int debug)
goto out;
}
+ st = &state;
+ str = maxstr;
+ bin = maxbin;
+
while (1) {
- rv = recv(fd, &st, sizeof(st), MSG_WAITALL);
+ if (sort_p || sort_s) {
+ len = sizeof(struct sanlk_state) + SANLK_STATE_MAXSTR*4;
+ buf = malloc(len);
+ if (!buf)
+ return -ENOMEM;
+ memset(buf, 0, len);
+ st = (struct sanlk_state *)buf;
+ str = buf + sizeof(struct sanlk_state);
+ bin = buf + sizeof(struct sanlk_state) + SANLK_STATE_MAXSTR;
+ } else {
+ memset(&state, 0, sizeof(state));
+ memset(maxstr, 0, sizeof(maxstr));
+ memset(maxbin, 0, sizeof(maxbin));
+ }
+
+ rv = recv(fd, st, sizeof(struct sanlk_state), MSG_WAITALL);
if (!rv)
break;
- if (rv != sizeof(st))
+ if (rv != sizeof(struct sanlk_state))
break;
- if (st.str_len) {
- rv = recv(fd, str, st.str_len, MSG_WAITALL);
- if (rv != st.str_len)
+ if (st->str_len) {
+ rv = recv(fd, str, st->str_len, MSG_WAITALL);
+ if (rv != st->str_len)
break;
}
- switch (st.type) {
- case SANLK_STATE_DAEMON:
- status_daemon(fd, &st, str, debug);
- break;
- case SANLK_STATE_CLIENT:
- status_client(fd, &st, str, debug);
- break;
- case SANLK_STATE_LOCKSPACE:
- status_lockspace(fd, &st, str, debug);
- break;
- case SANLK_STATE_RESOURCE:
- status_resource(fd, &st, str, debug);
- break;
+ recv_bin(fd, st, bin);
+
+ if (sort_p || sort_s) {
+ if (sort_count == MAX_SORT_ENTRIES) {
+ printf("cannot sort over %d\n", MAX_SORT_ENTRIES);
+ goto out;
+ }
+ sort_bufs[sort_count++] = buf;
+ continue;
}
+
+ /* no sorting, print as received */
+
+ print_st(st, str, bin, debug);
}
+
+ if (sort_p) {
+ print_type(SANLK_STATE_DAEMON, debug);
+ print_p(-1, debug);
+ print_type(SANLK_STATE_LOCKSPACE, debug);
+ print_r_by_p(debug);
+ if (sort_done < sort_count) {
+ printf("-\n");
+ print_type(0, debug);
+ }
+ } else if (sort_s) {
+ print_type(SANLK_STATE_DAEMON, debug);
+ print_p(-1, debug);
+ print_type(SANLK_STATE_CLIENT, debug);
+ print_r_by_s(debug);
+ if (sort_done < sort_count) {
+ printf("-\n");
+ print_type(0, debug);
+ }
+ }
+
rv = 0;
out:
close(fd);
@@ -224,12 +459,12 @@ int sanlock_host_status(int debug, char *lockspace_name)
switch (st.type) {
case SANLK_STATE_HOST:
- status_host(fd, &st, str, debug);
+ status_host(&st, str, debug);
break;
}
}
- rv = 0;
+ rv = h.data;
out:
close(fd);
return rv;
diff --git a/src/client_cmd.h b/src/client_cmd.h
index 3599659..dceb7ca 100644
--- a/src/client_cmd.h
+++ b/src/client_cmd.h
@@ -9,7 +9,7 @@
#ifndef __CLIENT_CMD_H__
#define __CLIENT_CMD_H__
-int sanlock_status(int debug);
+int sanlock_status(int debug, char sort_arg);
int sanlock_host_status(int debug, char *lockspace_name);
int sanlock_log_dump(void);
int sanlock_shutdown(void);
diff --git a/src/main.c b/src/main.c
index 5b7477d..d3d0c4f 100644
--- a/src/main.c
+++ b/src/main.c
@@ -2027,7 +2027,7 @@ static int print_state_resource(struct resource *r, char *str, const char *list_
"list=%s "
"flags=%x "
"lver=%llu "
- "token_id=%u ",
+ "token_id=%u",
list_name,
r->flags,
(unsigned long long)r->lver,
@@ -2046,7 +2046,7 @@ static int print_state_host(struct host_status *hs, char *str)
"last_req=%llu "
"owner_id=%llu "
"owner_generation=%llu "
- "timestamp=%llu ",
+ "timestamp=%llu",
(unsigned long long)hs->last_check,
(unsigned long long)hs->last_live,
(unsigned long long)hs->last_req,
@@ -2255,13 +2255,19 @@ static void cmd_host_status(int fd, struct sm_header *h_recv)
goto fail;
}
- send(fd, &h, sizeof(h), MSG_NOSIGNAL);
-
pthread_mutex_lock(&spaces_mutex);
sp = find_lockspace(lockspace.name);
- memcpy(status, &sp->host_status, status_len);
+ if (sp)
+ memcpy(status, &sp->host_status, status_len);
pthread_mutex_unlock(&spaces_mutex);
+ if (!sp) {
+ h.data = -ENOSPC;
+ goto fail;
+ }
+
+ send(fd, &h, sizeof(h), MSG_NOSIGNAL);
+
for (i = 0; i < DEFAULT_MAX_HOSTS; i++) {
hs = &status[i];
if (!hs->last_live && !hs->owner_id)
@@ -3007,7 +3013,7 @@ static void print_usage(void)
printf(" -o 0|1 io timeout in seconds (%d)\n", DEFAULT_IO_TIMEOUT);
printf("\n");
printf("sanlock client <action> [options]\n");
- printf("sanlock client status [-D]\n");
+ printf("sanlock client status [-D] [-o p|s]\n");
printf("sanlock client host_status -s LOCKSPACE [-D]\n");
printf("sanlock client log_dump\n");
printf("sanlock client shutdown\n");
@@ -3231,9 +3237,13 @@ static int read_command_line(int argc, char *argv[])
com.high_priority = atoi(optionarg);
break;
case 'o':
- com.io_timeout_arg = atoi(optionarg);
- if (!com.io_timeout_arg)
- com.io_timeout_arg = DEFAULT_IO_TIMEOUT;
+ if (com.action == ACT_STATUS) {
+ com.sort_arg = *optionarg;
+ } else {
+ com.io_timeout_arg = atoi(optionarg);
+ if (!com.io_timeout_arg)
+ com.io_timeout_arg = DEFAULT_IO_TIMEOUT;
+ }
break;
case 'n':
com.num_hosts = atoi(optionarg);
@@ -3344,7 +3354,7 @@ static int do_client(void)
switch (com.action) {
case ACT_STATUS:
- rv = sanlock_status(com.debug);
+ rv = sanlock_status(com.debug, com.sort_arg);
break;
case ACT_HOST_STATUS:
diff --git a/src/sanlock.8 b/src/sanlock.8
index 4462280..891ab86 100644
--- a/src/sanlock.8
+++ b/src/sanlock.8
@@ -247,6 +247,7 @@ io timeout in seconds
Print processes, lockspaces, and resources being manged by the sanlock
daemon. Add -D to show extra internal daemon status for debugging.
+Add -o p to show resources by pid, or -o s to show resources by lockspace.
.BR "sanlock client host_status -s" " LOCKSPACE"
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index 2621317..0b39934 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -492,6 +492,7 @@ struct command_line {
int uid; /* -U */
int gid; /* -G */
int pid; /* -p */
+ char sort_arg;
uint64_t local_host_id; /* -i */
uint64_t local_host_generation; /* -g */
int num_hosts; /* -n */
commit ad80edea0471fc32221da1d2610d3197f9e31275
Author: David Teigland <teigland(a)redhat.com>
Date: Mon Sep 12 14:26:58 2011 -0500
sanlock: improve status output
diff --git a/src/client_cmd.c b/src/client_cmd.c
index a3949d1..73404bf 100644
--- a/src/client_cmd.c
+++ b/src/client_cmd.c
@@ -58,15 +58,16 @@ static void print_debug(char *str, int len)
static void status_daemon(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
{
- printf("daemon\n");
+ printf("daemon %.48s\n", st->name);
- if (st->str_len && debug)
+ if (st->str_len && debug) {
print_debug(str, st->str_len);
+ }
}
static void status_client(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
{
- printf("pid %d ", st->data32);
+ printf("p %d ", st->data32);
printf("%.48s\n", st->name);
if (st->str_len && debug)
@@ -80,7 +81,7 @@ static void status_lockspace(int fd, struct sanlk_state *st, char *str, int debu
rv = recv(fd, &lockspace, sizeof(lockspace), MSG_WAITALL);
- printf("lockspace %.48s:%llu:%s:%llu\n",
+ printf("s %.48s:%llu:%s:%llu\n",
lockspace.name,
(unsigned long long)lockspace.host_id,
lockspace.host_id_disk.path,
@@ -98,7 +99,7 @@ static void status_resource(int fd, struct sanlk_state *st, char *str, int debug
rv = recv(fd, &resource, sizeof(resource), MSG_WAITALL);
- printf("pid %d resource %.48s:%.48s", st->data32, resource.lockspace_name, resource.name);
+ printf("r %.48s:%.48s", resource.lockspace_name, resource.name);
for (i = 0; i < resource.num_disks; i++) {
rv = recv(fd, &disk, sizeof(disk), MSG_WAITALL);
@@ -106,7 +107,7 @@ static void status_resource(int fd, struct sanlk_state *st, char *str, int debug
printf(":%s:%llu",
disk.path, (unsigned long long)disk.offset);
}
- printf("\n");
+ printf(":%llu p %u\n", (unsigned long long)st->data64, st->data32);
if (st->str_len && debug)
print_debug(str, st->str_len);
diff --git a/src/main.c b/src/main.c
index b7cfe91..5b7477d 100644
--- a/src/main.c
+++ b/src/main.c
@@ -2064,6 +2064,7 @@ static void send_state_daemon(int fd)
int str_len;
memset(&st, 0, sizeof(st));
+ strncpy(st.name, our_host_name_global, NAME_ID_SIZE);
st.type = SANLK_STATE_DAEMON;
@@ -2137,6 +2138,7 @@ static void send_state_resource(int fd, struct resource *r, const char *list_nam
st.type = SANLK_STATE_RESOURCE;
st.data32 = r->pid;
+ st.data64 = r->lver;
strncpy(st.name, r->r.name, NAME_ID_SIZE);
str_len = print_state_resource(r, str, list_name);
12 years, 2 months
src/Makefile src/client.c src/client_cmd.c src/client_cmd.h src/direct_lib.c src/host_id.c src/host_id.h src/log.c src/log.h src/main.c src/sanlock.8 src/sanlock_admin.h src/sanlock_internal.h src/sanlock_sock.h src/token_manager.c
by David Teigland
src/Makefile | 1
src/client.c | 172 -------------------
src/client_cmd.c | 287 ++++++++++++++++++++++++++++++++
src/client_cmd.h | 17 +
src/direct_lib.c | 4
src/host_id.c | 56 +-----
src/host_id.h | 5
src/log.c | 2
src/log.h | 2
src/main.c | 434 +++++++++++++++++++++++++++++++++----------------
src/sanlock.8 | 6
src/sanlock_admin.h | 8
src/sanlock_internal.h | 27 ++-
src/sanlock_sock.h | 6
src/token_manager.c | 22 --
15 files changed, 652 insertions(+), 397 deletions(-)
New commits:
commit 241606a1ae1d4db45cff28902e052e00d541b9e6
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Sep 8 17:27:13 2011 -0500
sanlock: new status and host_status
Change the client status command to operate on global resource
lists instead of per client tokens.
Add new client host_status command to print a lockspace's
saved info about hosts.
diff --git a/src/Makefile b/src/Makefile
index 8afbd8a..5d710e4 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -30,6 +30,7 @@ CMD_SOURCE = \
token_manager.c \
watchdog.c \
monotime.c \
+ client_cmd.c \
sanlock_sock.c
LIB_ENTIRE_SOURCE = \
diff --git a/src/client.c b/src/client.c
index 4987867..ea25832 100644
--- a/src/client.c
+++ b/src/client.c
@@ -81,7 +81,9 @@ static int send_header(int sock, int cmd, uint32_t cmd_flags, int datalen,
return 0;
}
-static int send_command(int cmd, uint32_t data)
+int send_command(int cmd, uint32_t data);
+
+int send_command(int cmd, uint32_t data)
{
int rv, sock;
@@ -114,174 +116,6 @@ static int recv_result(int fd)
return (int)h.data;
}
-int sanlock_shutdown(void)
-{
- int fd;
-
- fd = send_command(SM_CMD_SHUTDOWN, 0);
- if (fd < 0)
- return fd;
-
- close(fd);
- return 0;
-}
-
-int sanlock_log_dump(void)
-{
- struct sm_header h;
- char buf[4096];
- int fd, rv;
-
- fd = send_command(SM_CMD_LOG_DUMP, 0);
- if (fd < 0)
- return fd;
-
- memset(&h, 0, sizeof(h));
-
- rv = recv(fd, &h, sizeof(h), MSG_WAITALL);
- if (rv != sizeof(h)) {
- rv = -1;
- goto out;
- }
-
- while (1) {
- memset(buf, 0, sizeof(buf));
-
- rv = recv(fd, buf, sizeof(buf) - 1, MSG_WAITALL);
-
- if (rv > 0)
- printf("%s", buf);
- else
- break;
- }
- printf("\n");
- out:
- close(fd);
- return rv;
-}
-
-static void print_debug(char *str, int len)
-{
- char *p;
- int i;
-
- p = &str[0];
- for (i = 0; i < len-1; i++) {
- if (str[i] == ' ') {
- str[i] = '\0';
- printf(" %s\n", p);
- p = &str[i+1];
- }
- }
-
- if (p)
- printf(" %s\n", p);
-}
-
-static void status_daemon(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
-{
- printf("daemon\n");
-
- if (st->str_len && debug)
- print_debug(str, st->str_len);
-}
-
-static void status_lockspace(int fd, struct sanlk_state *st, char *str, int debug)
-{
- struct sanlk_lockspace lockspace;
- int rv;
-
- rv = recv(fd, &lockspace, sizeof(lockspace), MSG_WAITALL);
-
- printf(" lockspace %.48s:%llu:%s:%llu\n",
- lockspace.name,
- (unsigned long long)lockspace.host_id,
- lockspace.host_id_disk.path,
- (unsigned long long)lockspace.host_id_disk.offset);
-
- if (st->str_len && debug)
- print_debug(str, st->str_len);
-}
-
-static void status_client(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
-{
- printf("pid %d ", st->data32);
- printf("%.48s\n", st->name);
-
- if (st->str_len && debug)
- print_debug(str, st->str_len);
-}
-
-static void status_resource(int fd, struct sanlk_state *st, char *str, int debug)
-{
- struct sanlk_resource resource;
- struct sanlk_disk disk;
- int i, rv;
-
- rv = recv(fd, &resource, sizeof(resource), MSG_WAITALL);
-
- printf(" resource %.48s:%.48s", resource.lockspace_name, resource.name);
-
- for (i = 0; i < resource.num_disks; i++) {
- rv = recv(fd, &disk, sizeof(disk), MSG_WAITALL);
-
- printf(":%s:%llu",
- disk.path, (unsigned long long)disk.offset);
- }
- printf("\n");
-
- if (st->str_len && debug)
- print_debug(str, st->str_len);
-}
-
-int sanlock_status(int debug)
-{
- struct sm_header h;
- struct sanlk_state st;
- char str[SANLK_STATE_MAXSTR];
- int fd, rv;
-
- fd = send_command(SM_CMD_STATUS, 0);
- if (fd < 0)
- return fd;
-
- rv = recv(fd, &h, sizeof(h), MSG_WAITALL);
- if (rv != sizeof(h))
- return -1;
-
-
- while (1) {
- rv = recv(fd, &st, sizeof(st), MSG_WAITALL);
- if (!rv)
- break;
- if (rv != sizeof(st))
- return -1;
-
- if (st.str_len) {
- rv = recv(fd, str, st.str_len, MSG_WAITALL);
- if (rv != st.str_len)
- return -1;
- }
-
- switch (st.type) {
- case SANLK_STATE_DAEMON:
- status_daemon(fd, &st, str, debug);
- break;
- case SANLK_STATE_LOCKSPACE:
- status_lockspace(fd, &st, str, debug);
- break;
- case SANLK_STATE_CLIENT:
- status_client(fd, &st, str, debug);
- break;
- case SANLK_STATE_RESOURCE:
- status_resource(fd, &st, str, debug);
- break;
- }
- }
-
- return 0;
-}
-
static int cmd_lockspace(int cmd, struct sanlk_lockspace *ls, uint32_t flags)
{
int rv, fd;
diff --git a/src/client_cmd.c b/src/client_cmd.c
new file mode 100644
index 0000000..a3949d1
--- /dev/null
+++ b/src/client_cmd.c
@@ -0,0 +1,287 @@
+/*
+ * Copyright (C) 2010-2011 Red Hat, Inc. All rights reserved.
+ *
+ * This copyrighted material is made available to anyone wishing to use,
+ * modify, copy, or redistribute it subject to the terms and conditions
+ * of the GNU General Public License v.2.
+ */
+
+#include <inttypes.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <stddef.h>
+#include <fcntl.h>
+#include <string.h>
+#include <errno.h>
+#include <limits.h>
+#include <time.h>
+#include <syslog.h>
+#include <pthread.h>
+#include <poll.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <sys/socket.h>
+#include <sys/time.h>
+#include <sys/un.h>
+
+#include "sanlock.h"
+#include "sanlock_sock.h"
+
+#include "client_cmd.h"
+
+#ifndef GNUC_UNUSED
+#define GNUC_UNUSED __attribute__((__unused__))
+#endif
+
+extern int send_command(int cmd, uint32_t data);
+
+static void print_debug(char *str, int len)
+{
+ char *p;
+ int i;
+
+ p = &str[0];
+ for (i = 0; i < len-1; i++) {
+ if (str[i] == ' ') {
+ str[i] = '\0';
+ printf(" %s\n", p);
+ p = &str[i+1];
+ }
+ }
+
+ if (p)
+ printf(" %s\n", p);
+}
+
+static void status_daemon(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
+{
+ printf("daemon\n");
+
+ if (st->str_len && debug)
+ print_debug(str, st->str_len);
+}
+
+static void status_client(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
+{
+ printf("pid %d ", st->data32);
+ printf("%.48s\n", st->name);
+
+ if (st->str_len && debug)
+ print_debug(str, st->str_len);
+}
+
+static void status_lockspace(int fd, struct sanlk_state *st, char *str, int debug)
+{
+ struct sanlk_lockspace lockspace;
+ int rv;
+
+ rv = recv(fd, &lockspace, sizeof(lockspace), MSG_WAITALL);
+
+ printf("lockspace %.48s:%llu:%s:%llu\n",
+ lockspace.name,
+ (unsigned long long)lockspace.host_id,
+ lockspace.host_id_disk.path,
+ (unsigned long long)lockspace.host_id_disk.offset);
+
+ if (st->str_len && debug)
+ print_debug(str, st->str_len);
+}
+
+static void status_resource(int fd, struct sanlk_state *st, char *str, int debug)
+{
+ struct sanlk_resource resource;
+ struct sanlk_disk disk;
+ int i, rv;
+
+ rv = recv(fd, &resource, sizeof(resource), MSG_WAITALL);
+
+ printf("pid %d resource %.48s:%.48s", st->data32, resource.lockspace_name, resource.name);
+
+ for (i = 0; i < resource.num_disks; i++) {
+ rv = recv(fd, &disk, sizeof(disk), MSG_WAITALL);
+
+ printf(":%s:%llu",
+ disk.path, (unsigned long long)disk.offset);
+ }
+ printf("\n");
+
+ if (st->str_len && debug)
+ print_debug(str, st->str_len);
+}
+
+static void status_host(int fd GNUC_UNUSED, struct sanlk_state *st, char *str, int debug)
+{
+ printf("%u timestamp %llu\n", st->data32,
+ (unsigned long long)st->data64);
+
+ if (st->str_len && debug)
+ print_debug(str, st->str_len);
+}
+
+int sanlock_status(int debug)
+{
+ struct sm_header h;
+ struct sanlk_state st;
+ char str[SANLK_STATE_MAXSTR];
+ int fd, rv;
+
+ fd = send_command(SM_CMD_STATUS, 0);
+ if (fd < 0)
+ return fd;
+
+ rv = recv(fd, &h, sizeof(h), MSG_WAITALL);
+ if (rv < 0) {
+ rv = -errno;
+ goto out;
+ }
+ if (rv != sizeof(h)) {
+ rv = -1;
+ goto out;
+ }
+
+ while (1) {
+ rv = recv(fd, &st, sizeof(st), MSG_WAITALL);
+ if (!rv)
+ break;
+ if (rv != sizeof(st))
+ break;
+
+ if (st.str_len) {
+ rv = recv(fd, str, st.str_len, MSG_WAITALL);
+ if (rv != st.str_len)
+ break;
+ }
+
+ switch (st.type) {
+ case SANLK_STATE_DAEMON:
+ status_daemon(fd, &st, str, debug);
+ break;
+ case SANLK_STATE_CLIENT:
+ status_client(fd, &st, str, debug);
+ break;
+ case SANLK_STATE_LOCKSPACE:
+ status_lockspace(fd, &st, str, debug);
+ break;
+ case SANLK_STATE_RESOURCE:
+ status_resource(fd, &st, str, debug);
+ break;
+ }
+ }
+ rv = 0;
+ out:
+ close(fd);
+ return rv;
+}
+
+int sanlock_host_status(int debug, char *lockspace_name)
+{
+ struct sm_header h;
+ struct sanlk_state st;
+ struct sanlk_lockspace lockspace;
+ char str[SANLK_STATE_MAXSTR];
+ int fd, rv;
+
+ if (!lockspace_name || !lockspace_name[0])
+ return -1;
+
+ fd = send_command(SM_CMD_HOST_STATUS, 0);
+ if (fd < 0)
+ return fd;
+
+ memset(&lockspace, 0, sizeof(lockspace));
+ snprintf(lockspace.name, SANLK_NAME_LEN, "%s", lockspace_name);
+
+ rv = send(fd, &lockspace, sizeof(lockspace), 0);
+ if (rv < 0)
+ goto out;
+
+ rv = recv(fd, &h, sizeof(h), MSG_WAITALL);
+ if (rv < 0) {
+ rv = -errno;
+ goto out;
+ }
+ if (rv != sizeof(h)) {
+ rv = -1;
+ goto out;
+ }
+
+ while (1) {
+ rv = recv(fd, &st, sizeof(st), MSG_WAITALL);
+ if (!rv)
+ break;
+ if (rv != sizeof(st))
+ break;
+
+ if (st.str_len) {
+ rv = recv(fd, str, st.str_len, MSG_WAITALL);
+ if (rv != st.str_len)
+ break;
+ }
+
+ switch (st.type) {
+ case SANLK_STATE_HOST:
+ status_host(fd, &st, str, debug);
+ break;
+ }
+ }
+
+ rv = 0;
+ out:
+ close(fd);
+ return rv;
+}
+
+int sanlock_log_dump(void)
+{
+ struct sm_header h;
+ char buf[4096];
+ int fd, rv;
+
+ fd = send_command(SM_CMD_LOG_DUMP, 0);
+ if (fd < 0)
+ return fd;
+
+ memset(&h, 0, sizeof(h));
+
+ rv = recv(fd, &h, sizeof(h), MSG_WAITALL);
+ if (rv < 0) {
+ rv = -errno;
+ goto out;
+ }
+ if (rv != sizeof(h)) {
+ rv = -1;
+ goto out;
+ }
+
+
+ while (1) {
+ memset(buf, 0, sizeof(buf));
+
+ rv = recv(fd, buf, sizeof(buf) - 1, MSG_WAITALL);
+
+ if (rv > 0)
+ printf("%s", buf);
+ else
+ break;
+ }
+ printf("\n");
+ out:
+ close(fd);
+ return rv;
+}
+
+int sanlock_shutdown(void)
+{
+ int fd;
+
+ fd = send_command(SM_CMD_SHUTDOWN, 0);
+ if (fd < 0)
+ return fd;
+
+ close(fd);
+ return 0;
+}
+
diff --git a/src/client_cmd.h b/src/client_cmd.h
new file mode 100644
index 0000000..3599659
--- /dev/null
+++ b/src/client_cmd.h
@@ -0,0 +1,17 @@
+/*
+ * Copyright (C) 2010-2011 Red Hat, Inc. All rights reserved.
+ *
+ * This copyrighted material is made available to anyone wishing to use,
+ * modify, copy, or redistribute it subject to the terms and conditions
+ * of the GNU General Public License v.2.
+ */
+
+#ifndef __CLIENT_CMD_H__
+#define __CLIENT_CMD_H__
+
+int sanlock_status(int debug);
+int sanlock_host_status(int debug, char *lockspace_name);
+int sanlock_log_dump(void);
+int sanlock_shutdown(void);
+
+#endif
diff --git a/src/direct_lib.c b/src/direct_lib.c
index 7d7f9a9..59513ab 100644
--- a/src/direct_lib.c
+++ b/src/direct_lib.c
@@ -21,11 +21,11 @@
#include "direct.h"
#include "task.h"
-void log_level(int space_id GNUC_UNUSED, int token_id GNUC_UNUSED,
+void log_level(uint32_t space_id GNUC_UNUSED, uint32_t token_id GNUC_UNUSED,
char *name GNUC_UNUSED,
int level GNUC_UNUSED, const char *fmt GNUC_UNUSED, ...);
-void log_level(int space_id GNUC_UNUSED, int token_id GNUC_UNUSED,
+void log_level(uint32_t space_id GNUC_UNUSED, uint32_t token_id GNUC_UNUSED,
char *name GNUC_UNUSED,
int level GNUC_UNUSED, const char *fmt GNUC_UNUSED, ...)
{
diff --git a/src/host_id.c b/src/host_id.c
index 86df60b..4d39d82 100644
--- a/src/host_id.c
+++ b/src/host_id.c
@@ -34,43 +34,7 @@
#include "direct.h"
#include "token_manager.h"
-static unsigned int space_id_counter = 1;
-
-struct list_head spaces;
-struct list_head spaces_add;
-struct list_head spaces_rem;
-pthread_mutex_t spaces_mutex = PTHREAD_MUTEX_INITIALIZER;
-
-int print_space_state(struct space *sp, char *str)
-{
- memset(str, 0, SANLK_STATE_MAXSTR);
-
- snprintf(str, SANLK_STATE_MAXSTR-1,
- "space_id=%u "
- "host_generation=%llu "
- "space_dead=%d "
- "killing_pids=%d "
- "corrupt_result=%d "
- "acquire_last_result=%d "
- "renewal_last_result=%d "
- "acquire_last_attempt=%llu "
- "acquire_last_success=%llu "
- "renewal_last_attempt=%llu "
- "renewal_last_success=%llu",
- sp->space_id,
- (unsigned long long)sp->host_generation,
- sp->space_dead,
- sp->killing_pids,
- sp->lease_status.corrupt_result,
- sp->lease_status.acquire_last_result,
- sp->lease_status.renewal_last_result,
- (unsigned long long)sp->lease_status.acquire_last_attempt,
- (unsigned long long)sp->lease_status.acquire_last_success,
- (unsigned long long)sp->lease_status.renewal_last_attempt,
- (unsigned long long)sp->lease_status.renewal_last_success);
-
- return strlen(str) + 1;
-}
+static uint32_t space_id_counter = 1;
static struct space *_search_space(char *name,
struct sync_disk *disk,
@@ -123,6 +87,11 @@ static struct space *_search_space(char *name,
return NULL;
}
+struct space *find_lockspace(char *name)
+{
+ return _search_space(name, NULL, 0, &spaces, &spaces_rem, &spaces_add);
+}
+
int _lockspace_info(char *space_name, struct space *sp_out)
{
struct space *sp;
@@ -290,9 +259,6 @@ void check_other_leases(struct task *task, struct space *sp, char *buf)
new = 0;
for (i = 0; i < DEFAULT_MAX_HOSTS; i++) {
- if (i+1 == sp->host_id)
- continue;
-
hs = &sp->host_status[i];
hs->last_check = now;
@@ -309,6 +275,9 @@ void check_other_leases(struct task *task, struct space *sp, char *buf)
hs->timestamp = leader->timestamp;
hs->last_live = now;
+ if (i+1 == sp->host_id)
+ continue;
+
bitmap = (char *)leader + HOSTID_BITMAP_OFFSET;
if (!test_id_bit(sp->host_id, bitmap))
@@ -834,10 +803,3 @@ void free_lockspaces(int wait)
pthread_mutex_unlock(&spaces_mutex);
}
-void setup_spaces(void)
-{
- INIT_LIST_HEAD(&spaces);
- INIT_LIST_HEAD(&spaces_add);
- INIT_LIST_HEAD(&spaces_rem);
-}
-
diff --git a/src/host_id.h b/src/host_id.h
index fba2d03..00dbcf7 100644
--- a/src/host_id.h
+++ b/src/host_id.h
@@ -9,20 +9,17 @@
#ifndef __HOST_ID_H__
#define __HOST_ID__H__
-int print_space_state(struct space *sp, char *str);
+struct space *find_lockspace(char *name);
int _lockspace_info(char *space_name, struct space *sp_out);
int lockspace_info(char *space_name, struct space *sp_out);
int lockspace_disk(char *space_name, struct sync_disk *disk);
-void block_watchdog_updates(char *space_name);
int host_info(char *space_name, uint64_t host_id, struct host_status *hs_out);
int host_status_set_bit(char *space_name, uint64_t host_id);
-int host_status_clear_bit(char *space_name, uint64_t host_id);
int test_id_bit(int host_id, char *bitmap);
int check_our_lease(struct task *task, struct space *sp, int *check_all, char *check_buf);
void check_other_leases(struct task *task, struct space *sp, char *buf);
int add_lockspace(struct sanlk_lockspace *ls);
int rem_lockspace(struct sanlk_lockspace *ls);
void free_lockspaces(int wait);
-void setup_spaces(void);
#endif
diff --git a/src/log.c b/src/log.c
index 3cb4064..72b424f 100644
--- a/src/log.c
+++ b/src/log.c
@@ -100,7 +100,7 @@ static void _log_save_ent(int level, int len)
* logfile and/or syslog (so callers don't block writing messages to files)
*/
-void log_level(int space_id, int token_id, char *name_in, int level, const char *fmt, ...)
+void log_level(uint32_t space_id, uint32_t token_id, char *name_in, int level, const char *fmt, ...)
{
va_list ap;
char name[NAME_ID_SIZE + 1];
diff --git a/src/log.h b/src/log.h
index eacb91b..8661eb0 100644
--- a/src/log.h
+++ b/src/log.h
@@ -9,7 +9,7 @@
#ifndef __LOG_H__
#define __LOG_H__
-void log_level(int space_id, int token_id, char *name_in, int level, const char *fmt, ...)
+void log_level(uint32_t space_id, uint32_t token_id, char *name_in, int level, const char *fmt, ...)
__attribute__((format(printf, 5, 6)));
int setup_logging(void);
diff --git a/src/main.c b/src/main.c
index c249ace..b7cfe91 100644
--- a/src/main.c
+++ b/src/main.c
@@ -49,6 +49,7 @@
#include "lockfile.h"
#include "watchdog.h"
#include "task.h"
+#include "client_cmd.h"
/* priorities are LOG_* from syslog.h */
int log_logfile_priority = LOG_WARNING;
@@ -82,7 +83,7 @@ static struct pollfd *pollfd = NULL;
static char command[COMMAND_MAX];
static int cmd_argc;
static char **cmd_argv;
-static unsigned int token_id_counter = 1;
+static uint32_t token_id_counter = 1;
struct cmd_args {
struct list_head list; /* thread_pool data */
@@ -106,10 +107,6 @@ struct thread_pool {
static struct thread_pool pool;
-extern struct list_head spaces;
-extern struct list_head spaces_rem;
-extern pthread_mutex_t spaces_mutex;
-
static struct random_data rand_data;
static char rand_state[32];
static pthread_mutex_t rand_mutex = PTHREAD_MUTEX_INITIALIZER;
@@ -1921,7 +1918,27 @@ static int thread_pool_create(int min_workers, int max_workers)
return rv;
}
-static int print_daemon_state(char *str)
+/*
+ * sanlock client status
+ *
+ * 1. send_state_daemon
+ *
+ * 2. for each cl in clients
+ * send_state_client() [sanlk_state + str_len]
+ *
+ * 3. for each sp in spaces, spaces_add, spaces_rem
+ * send_state_lockspace() [sanlk_state + str_len + sanlk_lockspace]
+ *
+ * 4. for each r in resources, dispose_resources
+ * send_state_resource() [sanlk_state + str_len + sanlk_resource + sanlk_disk * num_disks]
+ *
+ * sanlock client host_status <lockspace_name>
+ *
+ * 1. for each hs in sp->host_status
+ * send_state_host()
+ */
+
+static int print_state_daemon(char *str)
{
memset(str, 0, SANLK_STATE_MAXSTR);
@@ -1942,7 +1959,7 @@ static int print_daemon_state(char *str)
return strlen(str) + 1;
}
-static int print_client_state(struct client *cl, int ci, char *str)
+static int print_state_client(struct client *cl, int ci, char *str)
{
memset(str, 0, SANLK_STATE_MAXSTR);
@@ -1969,180 +1986,295 @@ static int print_client_state(struct client *cl, int ci, char *str)
return strlen(str) + 1;
}
-static int print_token_state(struct token *t, char *str)
+static int print_state_lockspace(struct space *sp, char *str, const char *list_name)
{
memset(str, 0, SANLK_STATE_MAXSTR);
snprintf(str, SANLK_STATE_MAXSTR-1,
- "token_id=%u "
- "acquire_result=%d "
- "release_result=%d "
- "leader.lver=%llu "
- "leader.timestamp=%llu "
- "leader.owner_id=%llu "
- "leader.owner_generation=%llu",
- t->token_id,
- t->acquire_result,
- t->release_result,
- (unsigned long long)t->leader.lver,
- (unsigned long long)t->leader.timestamp,
- (unsigned long long)t->leader.owner_id,
- (unsigned long long)t->leader.owner_generation);
+ "list=%s "
+ "space_id=%u "
+ "host_generation=%llu "
+ "space_dead=%d "
+ "killing_pids=%d "
+ "corrupt_result=%d "
+ "acquire_last_result=%d "
+ "renewal_last_result=%d "
+ "acquire_last_attempt=%llu "
+ "acquire_last_success=%llu "
+ "renewal_last_attempt=%llu "
+ "renewal_last_success=%llu",
+ list_name,
+ sp->space_id,
+ (unsigned long long)sp->host_generation,
+ sp->space_dead,
+ sp->killing_pids,
+ sp->lease_status.corrupt_result,
+ sp->lease_status.acquire_last_result,
+ sp->lease_status.renewal_last_result,
+ (unsigned long long)sp->lease_status.acquire_last_attempt,
+ (unsigned long long)sp->lease_status.acquire_last_success,
+ (unsigned long long)sp->lease_status.renewal_last_attempt,
+ (unsigned long long)sp->lease_status.renewal_last_success);
return strlen(str) + 1;
}
-/*
- * 0. header
- * 1. dst (sanlk_state DAEMON)
- * 2. dst.str (dst.len)
- * 3. lst (sanlk_state LOCKSPACE)
- * 4. lst.str (lst.len) print_space_state()
- * 5. lockspace (sanlk_lockspace)
- * 6. [repeat 3-5 for each space]
- * 7. cst (sanlk_state CLIENT)
- * 8. cst.str (cst.len) print_client_state()
- * 9. rst (sanlk_state RESOURCE)
- * 10. rst.str (rst.len) print_token_state()
- * 11. resource (sanlk_resource)
- * 12. disks (sanlk_disk * resource.num_disks)
- * 13. [repeat 9-12 for each token]
- * 14. [repeat 7-13 for each client]
- */
+static int print_state_resource(struct resource *r, char *str, const char *list_name)
+{
+ memset(str, 0, SANLK_STATE_MAXSTR);
-/*
- * TODO:
- * increase max transfer size
- * send spaces/spaces_add/spaces_rem instead of just spaces
- * use resources/dispose_resources to send resource state instead of cl->tokens
- *
- * . daemon
- * . clients
- * . lockspaces from spaces/spaces_add/spaces_rem
- * . resources from resources/dispose_resources
- *
- * in print function,
- * for each client pid, go through resources and print any for that pid, clear the res
- * print any remaining detached resources (orphan, dispose)
- *
- * sanlock client host_status <lockspace_name>
- * send sp->host_status[] for the named lockspace
- */
+ snprintf(str, SANLK_STATE_MAXSTR-1,
+ "list=%s "
+ "flags=%x "
+ "lver=%llu "
+ "token_id=%u ",
+ list_name,
+ r->flags,
+ (unsigned long long)r->lver,
+ r->token_id);
-static void cmd_status(int fd, struct sm_header *h_recv)
+ return strlen(str) + 1;
+}
+
+static int print_state_host(struct host_status *hs, char *str)
{
- struct sm_header h;
- struct sanlk_state dst;
- struct sanlk_state lst;
- struct sanlk_state cst;
- struct sanlk_state rst;
- struct sanlk_lockspace lockspace;
+ memset(str, 0, SANLK_STATE_MAXSTR);
+
+ snprintf(str, SANLK_STATE_MAXSTR-1,
+ "last_check=%llu "
+ "last_live=%llu "
+ "last_req=%llu "
+ "owner_id=%llu "
+ "owner_generation=%llu "
+ "timestamp=%llu ",
+ (unsigned long long)hs->last_check,
+ (unsigned long long)hs->last_live,
+ (unsigned long long)hs->last_req,
+ (unsigned long long)hs->owner_id,
+ (unsigned long long)hs->owner_generation,
+ (unsigned long long)hs->timestamp);
+
+ return strlen(str) + 1;
+}
+
+static void send_state_daemon(int fd)
+{
+ struct sanlk_state st;
char str[SANLK_STATE_MAXSTR];
- struct token *token;
- struct space *sp;
- struct client *cl;
- int ci, i, j, str_len;
+ int str_len;
- /*
- * send header: h
- */
+ memset(&st, 0, sizeof(st));
- memset(&h, 0, sizeof(h));
- memcpy(&h, h_recv, sizeof(struct sm_header));
- h.length = sizeof(h);
- h.data = 0;
+ st.type = SANLK_STATE_DAEMON;
- send(fd, &h, sizeof(h), MSG_NOSIGNAL);
+ str_len = print_state_daemon(str);
- /*
- * send daemon state: dst, dst.str
- */
+ st.str_len = str_len;
- str_len = print_daemon_state(str);
- memset(&dst, 0, sizeof(dst));
- dst.type = SANLK_STATE_DAEMON;
- dst.str_len = str_len;
+ send(fd, &st, sizeof(st), MSG_NOSIGNAL);
+ if (str_len)
+ send(fd, str, str_len, MSG_NOSIGNAL);
+}
- send(fd, &dst, sizeof(dst), MSG_NOSIGNAL);
+static void send_state_client(int fd, struct client *cl, int ci)
+{
+ struct sanlk_state st;
+ char str[SANLK_STATE_MAXSTR];
+ int str_len;
+
+ memset(&st, 0, sizeof(st));
+
+ st.type = SANLK_STATE_CLIENT;
+ st.data32 = cl->pid;
+ strncpy(st.name, cl->owner_name, NAME_ID_SIZE);
+
+ str_len = print_state_client(cl, ci, str);
+
+ st.str_len = str_len;
+
+ send(fd, &st, sizeof(st), MSG_NOSIGNAL);
if (str_len)
send(fd, str, str_len, MSG_NOSIGNAL);
+}
- if (h_recv->data == SANLK_STATE_DAEMON)
- return;
+static void send_state_lockspace(int fd, struct space *sp, const char *list_name)
+{
+ struct sanlk_state st;
+ struct sanlk_lockspace lockspace;
+ char str[SANLK_STATE_MAXSTR];
+ int str_len;
- /*
- * send lockspace state: lst, lst.str, sanlk_lockspace
- */
+ memset(&st, 0, sizeof(st));
- pthread_mutex_lock(&spaces_mutex);
- list_for_each_entry(sp, &spaces, list) {
- str_len = print_space_state(sp, str);
- memset(&lst, 0, sizeof(lst));
- lst.type = SANLK_STATE_LOCKSPACE;
- lst.data64 = sp->host_id;
- strncpy(lst.name, sp->space_name, NAME_ID_SIZE);
- lst.str_len = str_len;
+ st.type = SANLK_STATE_LOCKSPACE;
+ st.data64 = sp->host_id;
+ strncpy(st.name, sp->space_name, NAME_ID_SIZE);
+
+ str_len = print_state_lockspace(sp, str, list_name);
+
+ st.str_len = str_len;
+
+ send(fd, &st, sizeof(st), MSG_NOSIGNAL);
+ if (str_len)
+ send(fd, str, str_len, MSG_NOSIGNAL);
- send(fd, &lst, sizeof(lst), MSG_NOSIGNAL);
- if (str_len)
- send(fd, str, str_len, MSG_NOSIGNAL);
+ memset(&lockspace, 0, sizeof(struct sanlk_lockspace));
+ strncpy(lockspace.name, sp->space_name, NAME_ID_SIZE);
+ lockspace.host_id = sp->host_id;
+ memcpy(&lockspace.host_id_disk, &sp->host_id_disk, sizeof(struct sanlk_disk));
- memset(&lockspace, 0, sizeof(struct sanlk_lockspace));
- strncpy(lockspace.name, sp->space_name, NAME_ID_SIZE);
- lockspace.host_id = sp->host_id;
- memcpy(&lockspace.host_id_disk, &sp->host_id_disk, sizeof(struct sanlk_disk));
+ send(fd, &lockspace, sizeof(lockspace), MSG_NOSIGNAL);
+}
+
+static void send_state_resource(int fd, struct resource *r, const char *list_name)
+{
+ struct sanlk_state st;
+ char str[SANLK_STATE_MAXSTR];
+ int str_len;
+ int i;
- send(fd, &lockspace, sizeof(lockspace), MSG_NOSIGNAL);
+ memset(&st, 0, sizeof(st));
+
+ st.type = SANLK_STATE_RESOURCE;
+ st.data32 = r->pid;
+ strncpy(st.name, r->r.name, NAME_ID_SIZE);
+
+ str_len = print_state_resource(r, str, list_name);
+
+ st.str_len = str_len;
+
+ send(fd, &st, sizeof(st), MSG_NOSIGNAL);
+ if (str_len)
+ send(fd, str, str_len, MSG_NOSIGNAL);
+
+ send(fd, &r->r, sizeof(struct sanlk_resource), MSG_NOSIGNAL);
+
+ for (i = 0; i < r->r.num_disks; i++) {
+ send(fd, &r->r.disks[i], sizeof(struct sanlk_disk), MSG_NOSIGNAL);
}
- pthread_mutex_unlock(&spaces_mutex);
+}
- if (h_recv->data == SANLK_STATE_LOCKSPACE)
- return;
+static void send_state_host(int fd, struct host_status *hs, int host_id)
+{
+ struct sanlk_state st;
+ char str[SANLK_STATE_MAXSTR];
+ int str_len;
- /*
- * send client and resource state:
- * cst, cst.str, (rst, rst.str, resource, disk*N)*M
- */
+ memset(&st, 0, sizeof(st));
+
+ st.type = SANLK_STATE_HOST;
+ st.data32 = host_id;
+ st.data64 = hs->timestamp;
+
+ str_len = print_state_host(hs, str);
+
+ st.str_len = str_len;
+
+ send(fd, &st, sizeof(st), MSG_NOSIGNAL);
+ if (str_len)
+ send(fd, str, str_len, MSG_NOSIGNAL);
+}
+
+static void cmd_status(int fd, struct sm_header *h_recv)
+{
+ struct sm_header h;
+ struct client *cl;
+ struct space *sp;
+ struct resource *r;
+ int ci;
+
+ memset(&h, 0, sizeof(h));
+ memcpy(&h, h_recv, sizeof(struct sm_header));
+ h.length = sizeof(h);
+ h.data = 0;
+
+ send(fd, &h, sizeof(h), MSG_NOSIGNAL);
+
+ send_state_daemon(fd);
+
+ if (h_recv->data == SANLK_STATE_DAEMON)
+ return;
for (ci = 0; ci <= client_maxi; ci++) {
cl = &client[ci];
-
if (!cl->used)
continue;
+ send_state_client(fd, cl, ci);
+ }
- str_len = print_client_state(cl, ci, str);
- memset(&cst, 0, sizeof(cst));
- cst.type = SANLK_STATE_CLIENT;
- cst.data32 = cl->pid;
- strncpy(cst.name, cl->owner_name, NAME_ID_SIZE);
- cst.str_len = str_len;
+ if (h_recv->data == SANLK_STATE_CLIENT)
+ return;
- send(fd, &cst, sizeof(cst), MSG_NOSIGNAL);
- if (str_len)
- send(fd, str, str_len, MSG_NOSIGNAL);
+ pthread_mutex_lock(&spaces_mutex);
+ list_for_each_entry(sp, &spaces, list)
+ send_state_lockspace(fd, sp, "spaces");
+ list_for_each_entry(sp, &spaces_rem, list)
+ send_state_lockspace(fd, sp, "spaces_rem");
+ list_for_each_entry(sp, &spaces_rem, list)
+ send_state_lockspace(fd, sp, "spaces_add");
+ pthread_mutex_unlock(&spaces_mutex);
- for (i = 0; i < SANLK_MAX_RESOURCES; i++) {
- token = cl->tokens[i];
- if (!token)
- continue;
+ if (h_recv->data == SANLK_STATE_LOCKSPACE)
+ return;
+
+ pthread_mutex_lock(&resource_mutex);
+ list_for_each_entry(r, &resources, list)
+ send_state_resource(fd, r, "resources");
+ list_for_each_entry(r, &dispose_resources, list)
+ send_state_resource(fd, r, "dispose_resources");
+ pthread_mutex_unlock(&resource_mutex);
+}
+
+static void cmd_host_status(int fd, struct sm_header *h_recv)
+{
+ struct sm_header h;
+ struct sanlk_lockspace lockspace;
+ struct space *sp;
+ struct host_status *hs, *status = NULL;
+ int status_len;
+ int i, rv;
- str_len = print_token_state(token, str);
- memset(&rst, 0, sizeof(rst));
- rst.type = SANLK_STATE_RESOURCE;
- strncpy(rst.name, token->r.name, NAME_ID_SIZE);
- rst.str_len = str_len;
+ memset(&h, 0, sizeof(h));
+ memcpy(&h, h_recv, sizeof(struct sm_header));
+ h.length = sizeof(h);
+ h.data = 0;
- send(fd, &rst, sizeof(rst), MSG_NOSIGNAL);
- if (str_len)
- send(fd, str, str_len, MSG_NOSIGNAL);
+ status_len = sizeof(struct host_status) * DEFAULT_MAX_HOSTS;
- send(fd, &token->r, sizeof(struct sanlk_resource), MSG_NOSIGNAL);
+ status = malloc(status_len);
+ if (!status) {
+ h.data = -ENOMEM;
+ goto fail;
+ }
- for (j = 0; j < token->r.num_disks; j++) {
- send(fd, &token->disks[j], sizeof(struct sanlk_disk), MSG_NOSIGNAL);
- }
- }
+ rv = recv(fd, &lockspace, sizeof(struct sanlk_lockspace), MSG_WAITALL);
+ if (rv != sizeof(struct sanlk_lockspace)) {
+ h.data = -ENOTCONN;
+ goto fail;
+ }
+
+ send(fd, &h, sizeof(h), MSG_NOSIGNAL);
+
+ pthread_mutex_lock(&spaces_mutex);
+ sp = find_lockspace(lockspace.name);
+ memcpy(status, &sp->host_status, status_len);
+ pthread_mutex_unlock(&spaces_mutex);
+
+ for (i = 0; i < DEFAULT_MAX_HOSTS; i++) {
+ hs = &status[i];
+ if (!hs->last_live && !hs->owner_id)
+ continue;
+ send_state_host(fd, hs, i+1);
}
+
+ if (status)
+ free(status);
+ return;
+ fail:
+ send(fd, &h, sizeof(h), MSG_NOSIGNAL);
+
+ if (status)
+ free(status);
}
static void cmd_log_dump(int fd, struct sm_header *h_recv)
@@ -2354,6 +2486,10 @@ static void process_cmd_daemon(int ci, struct sm_header *h_recv)
strcpy(client[ci].owner_name, "status");
cmd_status(fd, h_recv);
break;
+ case SM_CMD_HOST_STATUS:
+ strcpy(client[ci].owner_name, "host_status");
+ cmd_host_status(fd, h_recv);
+ break;
case SM_CMD_LOG_DUMP:
strcpy(client[ci].owner_name, "log_dump");
cmd_log_dump(fd, h_recv);
@@ -2401,6 +2537,7 @@ static void process_connection(int ci)
case SM_CMD_RESTRICT:
case SM_CMD_SHUTDOWN:
case SM_CMD_STATUS:
+ case SM_CMD_HOST_STATUS:
case SM_CMD_LOG_DUMP:
process_cmd_daemon(ci, &h);
break;
@@ -2644,8 +2781,6 @@ static int do_daemon(void)
if (rv < 0)
goto out_threads;
- setup_spaces();
-
main_loop();
close_token_manager();
@@ -2871,6 +3006,7 @@ static void print_usage(void)
printf("\n");
printf("sanlock client <action> [options]\n");
printf("sanlock client status [-D]\n");
+ printf("sanlock client host_status -s LOCKSPACE [-D]\n");
printf("sanlock client log_dump\n");
printf("sanlock client shutdown\n");
printf("sanlock client init -s LOCKSPACE | -r RESOURCE\n");
@@ -2968,6 +3104,8 @@ static int read_command_line(int argc, char *argv[])
case COM_CLIENT:
if (!strcmp(act, "status"))
com.action = ACT_STATUS;
+ else if (!strcmp(act, "host_status"))
+ com.action = ACT_HOST_STATUS;
else if (!strcmp(act, "log_dump"))
com.action = ACT_LOG_DUMP;
else if (!strcmp(act, "shutdown"))
@@ -3207,6 +3345,10 @@ static int do_client(void)
rv = sanlock_status(com.debug);
break;
+ case ACT_HOST_STATUS:
+ rv = sanlock_host_status(com.debug, com.lockspace.name);
+ break;
+
case ACT_LOG_DUMP:
rv = sanlock_log_dump();
break;
@@ -3468,6 +3610,16 @@ int main(int argc, char *argv[])
BUILD_BUG_ON(sizeof(struct sanlk_disk) != sizeof(struct sync_disk));
BUILD_BUG_ON(sizeof(struct leader_record) > LEADER_RECORD_MAX);
+
+ /* initialize global variables */
+ pthread_mutex_init(&spaces_mutex, NULL);
+ pthread_mutex_init(&resource_mutex, NULL);
+ pthread_cond_init(&resource_cond, NULL);
+ INIT_LIST_HEAD(&spaces);
+ INIT_LIST_HEAD(&spaces_rem);
+ INIT_LIST_HEAD(&spaces_add);
+ INIT_LIST_HEAD(&resources);
+ INIT_LIST_HEAD(&dispose_resources);
memset(&com, 0, sizeof(com));
com.use_watchdog = DEFAULT_USE_WATCHDOG;
diff --git a/src/sanlock.8 b/src/sanlock.8
index e3196e9..4462280 100644
--- a/src/sanlock.8
+++ b/src/sanlock.8
@@ -248,6 +248,12 @@ io timeout in seconds
Print processes, lockspaces, and resources being manged by the sanlock
daemon. Add -D to show extra internal daemon status for debugging.
+.BR "sanlock client host_status -s" " LOCKSPACE"
+
+Print state of host_id delta leases read during the last renewal.
+Only lockspace_name is used from the LOCKSPACE argument.
+Add -D to show extra internal daemon status for debugging.
+
.B sanlock client log_dump
Print the sanlock daemon internal debug log.
diff --git a/src/sanlock_admin.h b/src/sanlock_admin.h
index 3a64a2d..802beee 100644
--- a/src/sanlock_admin.h
+++ b/src/sanlock_admin.h
@@ -11,14 +11,6 @@
#define __SANLOCK_ADMIN_H__
/*
- * daemon admin/managment
- */
-
-int sanlock_status(int debug);
-int sanlock_log_dump(void);
-int sanlock_shutdown(void);
-
-/*
* add_lockspace returns:
* 0: the lockspace has been added successfully
* -EEXIST: the lockspace already exists
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index 7a26749..2621317 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -79,7 +79,7 @@ struct token {
uint64_t host_generation;
/* internal */
- int token_id; /* used to refer to this token instance in log messages */
+ uint32_t token_id; /* used to refer to this token instance in log messages */
int space_dead; /* don't bother trying to release if ls is dead */
int acquire_result;
int release_result;
@@ -89,6 +89,18 @@ struct token {
struct sanlk_resource r;
};
+#define R_EXAMINE 0x00000001
+
+struct resource {
+ struct list_head list;
+ struct token *token;
+ int pid;
+ uint32_t token_id;
+ uint32_t flags;
+ uint64_t lver;
+ struct sanlk_resource r;
+};
+
struct lease_status {
int corrupt_result;
int acquire_last_result;
@@ -116,11 +128,11 @@ struct host_status {
struct space {
struct list_head list;
char space_name[NAME_ID_SIZE];
+ uint32_t space_id; /* used to refer to this space instance in log messages */
uint64_t host_id;
uint64_t host_generation;
struct sync_disk host_id_disk;
int align_size;
- int space_id; /* used to refer to this space instance in log messages */
int space_dead;
int killing_pids;
int external_remove;
@@ -502,6 +514,7 @@ EXTERN struct command_line com;
enum {
ACT_STATUS = 1,
+ ACT_HOST_STATUS,
ACT_LOG_DUMP,
ACT_SHUTDOWN,
ACT_ADD_LOCKSPACE,
@@ -527,5 +540,15 @@ enum {
EXTERN int external_shutdown;
EXTERN char our_host_name_global[SANLK_NAME_LEN+1];
+EXTERN struct list_head spaces;
+EXTERN struct list_head spaces_rem;
+EXTERN struct list_head spaces_add;
+EXTERN pthread_mutex_t spaces_mutex;
+
+EXTERN struct list_head resources;
+EXTERN struct list_head dispose_resources;
+EXTERN pthread_mutex_t resource_mutex;
+EXTERN pthread_cond_t resource_cond;
+
#endif
diff --git a/src/sanlock_sock.h b/src/sanlock_sock.h
index 5c6a004..8279bd4 100644
--- a/src/sanlock_sock.h
+++ b/src/sanlock_sock.h
@@ -34,6 +34,7 @@ enum {
SM_CMD_INIT_RESOURCE = 14,
SM_CMD_EXAMINE_LOCKSPACE = 15,
SM_CMD_EXAMINE_RESOURCE = 16,
+ SM_CMD_HOST_STATUS = 17,
};
struct sm_header {
@@ -50,9 +51,10 @@ struct sm_header {
#define SANLK_STATE_MAXSTR 4096
#define SANLK_STATE_DAEMON 1
-#define SANLK_STATE_LOCKSPACE 2
-#define SANLK_STATE_CLIENT 3
+#define SANLK_STATE_CLIENT 2
+#define SANLK_STATE_LOCKSPACE 3
#define SANLK_STATE_RESOURCE 4
+#define SANLK_STATE_HOST 5
struct sanlk_state {
uint32_t type; /* SANLK_STATE_ */
diff --git a/src/token_manager.c b/src/token_manager.c
index a701cc3..6568af3 100644
--- a/src/token_manager.c
+++ b/src/token_manager.c
@@ -31,25 +31,10 @@
#include "task.h"
#include "host_id.h"
-static struct list_head resources;
-static struct list_head dispose_resources;
-static pthread_mutex_t resource_mutex = PTHREAD_MUTEX_INITIALIZER;
-static pthread_cond_t resource_cond = PTHREAD_COND_INITIALIZER;
static pthread_t resource_pt;
static int resource_thread_stop;
static int resource_examine;
-#define R_EXAMINE 0x00000001
-
-struct resource {
- struct list_head list;
- struct token *token;
- int pid;
- uint32_t flags;
- uint64_t lver;
- struct sanlk_resource r;
-};
-
int set_resource_examine(char *space_name, char *res_name)
{
struct resource *r;
@@ -147,6 +132,7 @@ int add_resource(struct token *token, int pid)
memset(r, 0, r_len);
memcpy(&r->r, &token->r, sizeof(struct sanlk_resource));
memcpy(&r->r.disks, &token->r.disks, disks_len);
+ r->token_id = token->token_id;
r->token = token;
r->pid = pid;
list_add_tail(&r->list, &resources);
@@ -208,10 +194,9 @@ int acquire_token(struct task *task, struct token *token,
if (rv < 0)
return rv;
- save_resource_lver(token, token->leader.lver);
-
memcpy(&token->leader, &leader_ret, sizeof(struct leader_record));
token->r.lver = token->leader.lver;
+ save_resource_lver(token, token->leader.lver);
return rv; /* SANLK_OK */
}
@@ -498,9 +483,6 @@ int setup_token_manager(void)
{
int rv;
- INIT_LIST_HEAD(&resources);
- INIT_LIST_HEAD(&dispose_resources);
-
rv = pthread_create(&resource_pt, NULL, resource_thread, NULL);
if (rv)
return -1;
12 years, 2 months
4 commits - src/direct_lib.c src/host_id.c src/main.c src/sanlock.8 src/sanlock_internal.h src/sanlock_resource.h src/token_manager.c
by David Teigland
src/direct_lib.c | 11 +++
src/host_id.c | 15 -----
src/main.c | 143 +++++++++++++++++++++++++++++++++++++------------
src/sanlock.8 | 7 --
src/sanlock_internal.h | 1
src/sanlock_resource.h | 1
src/token_manager.c | 17 +++--
7 files changed, 130 insertions(+), 65 deletions(-)
New commits:
commit dd9dfa9c9d544ff12684ab94b5e30fc165206052
Author: David Teigland <teigland(a)redhat.com>
Date: Wed Sep 7 16:49:13 2011 -0500
libsanlock: fix function stubs
would like to get rid of this ugliness
diff --git a/src/direct_lib.c b/src/direct_lib.c
index e2c7d7a..7d7f9a9 100644
--- a/src/direct_lib.c
+++ b/src/direct_lib.c
@@ -31,9 +31,16 @@ void log_level(int space_id GNUC_UNUSED, int token_id GNUC_UNUSED,
{
}
-int host_id_disk_info(char *name GNUC_UNUSED, struct sync_disk *disk GNUC_UNUSED);
+int lockspace_disk(char *space_name GNUC_UNUSED, struct sync_disk *disk GNUC_UNUSED);
-int host_id_disk_info(char *name GNUC_UNUSED, struct sync_disk *disk GNUC_UNUSED)
+int lockspace_disk(char *space_name GNUC_UNUSED, struct sync_disk *disk GNUC_UNUSED)
+{
+ return -1;
+}
+
+int host_info(char *space_name, uint64_t host_id, struct host_status *hs_out);
+
+int host_info(char *space_name GNUC_UNUSED, uint64_t host_id GNUC_UNUSED, struct host_status *hs_out GNUC_UNUSED)
{
return -1;
}
commit 4d1ed9632189bc2752ad6a684145ca38971ab959
Author: David Teigland <teigland(a)redhat.com>
Date: Wed Sep 7 15:54:53 2011 -0500
sanlock: remove BLOCK_WD force mode
don't include this option until we're ready to use it,
in case the usage specifics end up being different
diff --git a/src/host_id.c b/src/host_id.c
index 1b82eeb..86df60b 100644
--- a/src/host_id.c
+++ b/src/host_id.c
@@ -163,17 +163,6 @@ int lockspace_disk(char *space_name, struct sync_disk *disk)
return rv;
}
-void block_watchdog_updates(char *space_name)
-{
- struct space *sp;
-
- pthread_mutex_lock(&spaces_mutex);
- sp = _search_space(space_name, NULL, 0, &spaces, NULL, NULL);
- if (sp)
- sp->block_watchdog_updates = 1;
- pthread_mutex_unlock(&spaces_mutex);
-}
-
#if 0
static void clear_bit(int host_id, char *bitmap)
{
@@ -563,9 +552,7 @@ static void *lockspace_thread(void *arg_in)
* pet the watchdog
*/
- if (delta_result == SANLK_OK &&
- !sp->thread_stop &&
- !sp->block_watchdog_updates)
+ if (delta_result == SANLK_OK && !sp->thread_stop)
update_watchdog_file(sp, last_success);
pthread_mutex_unlock(&sp->mutex);
diff --git a/src/sanlock.8 b/src/sanlock.8
index 9a33990..e3196e9 100644
--- a/src/sanlock.8
+++ b/src/sanlock.8
@@ -477,11 +477,6 @@ take:
process has exited, the resource lease will be released, and can then be
acquired by anyone.
-\fB2\fP (BLOCK_WD): stop updating the watchdog (/dev/watchdog keepalive
-via wdmd_test_live) for the lockspace, which will lead to /dev/watchdog
-firing and reseting the host. The resource lease can be acquired after
-the timeout for a failed host.
-
.SH SEE ALSO
.BR wdmd (8)
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index 70c696d..7a26749 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -124,7 +124,6 @@ struct space {
int space_dead;
int killing_pids;
int external_remove;
- int block_watchdog_updates;
int thread_stop;
int wd_fd;
pthread_t thread;
diff --git a/src/sanlock_resource.h b/src/sanlock_resource.h
index 4998c16..770ebfc 100644
--- a/src/sanlock_resource.h
+++ b/src/sanlock_resource.h
@@ -28,7 +28,6 @@
/* request flags */
#define SANLK_REQ_KILL_PID 0x00000001
-#define SANLK_REQ_BLOCK_WD 0x00000002
int sanlock_register(void);
diff --git a/src/token_manager.c b/src/token_manager.c
index a15e47a..a701cc3 100644
--- a/src/token_manager.c
+++ b/src/token_manager.c
@@ -462,10 +462,8 @@ static void *resource_thread(void *arg GNUC_UNUSED)
log_error("req_kill_pid %d %.48s:%.48s", pid,
tt->r.lockspace_name, tt->r.name);
kill(pid, SIGKILL);
-
- } else if (req.force_mode == SANLK_REQ_BLOCK_WD) {
- log_error("req_block_wd %.48s", tt->r.lockspace_name);
- block_watchdog_updates(tt->r.lockspace_name);
+ } else {
+ log_error("req force_mode unknown %u", req.force_mode);
}
}
}
commit 0a356d94812872ebc7d8c39f79cac7cc88997ab1
Author: David Teigland <teigland(a)redhat.com>
Date: Wed Sep 7 12:12:34 2011 -0500
sanlock: restructure lockspace checks
shifts some code around, should be no functional change.
not checking sp after the list_move is better.
diff --git a/src/main.c b/src/main.c
index 51c1fe7..c249ace 100644
--- a/src/main.c
+++ b/src/main.c
@@ -424,10 +424,6 @@ static int client_using_space(struct client *cl, struct space *sp)
log_spoke(sp, token, "client_using_space pid %d", cl->pid);
token->space_dead = sp->space_dead;
rv = 1;
-
- /* we could break here after finding one if we didn't care
- * about setting token->space_dead which isn't really
- * necessary; it just avoids trying to release the token */
}
return rv;
}
@@ -586,9 +582,42 @@ static int main_loop(void)
last_check = now;
check_interval = STANDARD_CHECK_INTERVAL;
+ /*
+ * check the condition of each lockspace,
+ * if pids are being killed, have pids all exited?
+ * is its host_id being renewed?, if not kill pids
+ */
+
pthread_mutex_lock(&spaces_mutex);
list_for_each_entry_safe(sp, safe, &spaces, list) {
- check_all = 0;
+
+ if (sp->killing_pids && all_pids_dead(sp)) {
+ /*
+ * move sp to spaces_rem so main_loop
+ * will no longer see it.
+ */
+ log_space(sp, "set thread_stop");
+ pthread_mutex_lock(&sp->mutex);
+ sp->thread_stop = 1;
+ unlink_watchdog_file(sp);
+ pthread_mutex_unlock(&sp->mutex);
+ list_move(&sp->list, &spaces_rem);
+ continue;
+ }
+
+ if (sp->killing_pids) {
+ /*
+ * continue to kill the pids with increasing
+ * levels of severity until they all exit
+ */
+ kill_pids(sp);
+ check_interval = RECOVERY_CHECK_INTERVAL;
+ continue;
+ }
+
+ /*
+ * check host_id lease renewal
+ */
if (sp->align_size > check_buf_len) {
if (check_buf)
@@ -599,37 +628,21 @@ static int main_loop(void)
if (check_buf)
memset(check_buf, 0, check_buf_len);
- if (sp->killing_pids) {
- if (all_pids_dead(sp)) {
- log_space(sp, "set thread_stop");
- pthread_mutex_lock(&sp->mutex);
- sp->thread_stop = 1;
- unlink_watchdog_file(sp);
- pthread_mutex_unlock(&sp->mutex);
- list_move(&sp->list, &spaces_rem);
- } else {
- kill_pids(sp);
- }
- } else {
- rv = check_our_lease(&main_task, sp,
- &check_all, check_buf);
-
- if (rv || external_shutdown || sp->external_remove) {
- log_space(sp, "set killing_pids check %d "
- "shutdown %d remove %d",
- rv, external_shutdown,
- sp->external_remove);
- sp->space_dead = 1;
- sp->killing_pids = 1;
- kill_pids(sp);
- }
- }
+ check_all = 0;
- if (!sp->killing_pids && check_all)
- check_other_leases(&main_task, sp, check_buf);
+ rv = check_our_lease(&main_task, sp, &check_all, check_buf);
- if (sp->killing_pids)
+ if (rv || external_shutdown || sp->external_remove) {
+ log_space(sp, "set killing_pids check %d shutdown %d remove %d",
+ rv, external_shutdown, sp->external_remove);
+ sp->space_dead = 1;
+ sp->killing_pids = 1;
+ kill_pids(sp);
check_interval = RECOVERY_CHECK_INTERVAL;
+
+ } else if (check_all) {
+ check_other_leases(&main_task, sp, check_buf);
+ }
}
empty = list_empty(&spaces);
pthread_mutex_unlock(&spaces_mutex);
@@ -1516,6 +1529,51 @@ static void cmd_add_lockspace(struct cmd_args *ca)
client_resume(ca->ci_in);
}
+/*
+ * TODO: rem_lockspace works like a renewal failure would, and abandons
+ * resource leases (tokens) without releasing them. Unlike the renewal
+ * failure case, rem_lockspace most likely releases the host_id.
+ *
+ * What might be nice is an option where rem_lockspace would try to
+ * release resource leases before releasing the lockspace host_id.
+ * (We don't really want to be releasing tokens after we've released
+ * our host_id for the token's lockspace.)
+ *
+ * - kill all pids (by looking at struct resource pid?)
+ * - wait for all pids to exit
+ * o have us or other thread release their tokens/resources
+ * o wait for tokens/resources to be released, although the release
+ * may fail or time out, we don't want to wait too long
+ * - set sp->external_remove
+ * - main_loop sets sp->thread_stop (should find no pids)
+ * - main_loop unlinks watchdog
+ * - lockspace_thread releases host_id
+ *
+ * The aim is that we kill pids and wait for resources to be released
+ * before main_loop gets involved and before the lockspace_thread is
+ * told to stop.
+ *
+ * An alternative messy is to add another condition to the current
+ * main_loop checks:
+ *
+ * if (sp->killing_pids && all_pids_dead(sp) && all_tokens_released(sp)) {
+ * sp->thread_stop = 1;
+ * unlink_watchdog_file(sp);
+ * list_move(spaces_rem);
+ * }
+ *
+ * all_tokens_released would just return 1 in case we're not doing
+ * the releases
+ *
+ * release_token_async would need to learn to put the resources onto
+ * dispose list in this case
+ *
+ * consider using the resources/dispose_resources list for all_pids_dead
+ * and kill_pids? instead of the clients[].tokens[] loops? actually,
+ * could we remove tokens and cl->tokens altogether and just use the
+ * resources list?
+ */
+
static void cmd_rem_lockspace(struct cmd_args *ca)
{
struct sanlk_lockspace lockspace;
diff --git a/src/sanlock.8 b/src/sanlock.8
index f004159..9a33990 100644
--- a/src/sanlock.8
+++ b/src/sanlock.8
@@ -278,7 +278,7 @@ This will allow resources to be acquired in the lockspace.
Tell the sanlock daemon to release the specified host_id in the lockspace.
Any processes holding resource leases in this lockspace will be killed,
-and the leases released.
+and the resource leases not released.
.BR "sanlock client command -r" " RESOURCE " \
\fB-c\fP " " \fIpath\fP " " \fIargs\fP
commit 65de7ce8f6f67bcc54803d56e499dc611668aad0
Author: David Teigland <teigland(a)redhat.com>
Date: Wed Sep 7 11:16:29 2011 -0500
sanlock: use flags in struct resource
diff --git a/src/main.c b/src/main.c
index f30960b..51c1fe7 100644
--- a/src/main.c
+++ b/src/main.c
@@ -1952,6 +1952,25 @@ static int print_token_state(struct token *t, char *str)
* 14. [repeat 7-13 for each client]
*/
+/*
+ * TODO:
+ * increase max transfer size
+ * send spaces/spaces_add/spaces_rem instead of just spaces
+ * use resources/dispose_resources to send resource state instead of cl->tokens
+ *
+ * . daemon
+ * . clients
+ * . lockspaces from spaces/spaces_add/spaces_rem
+ * . resources from resources/dispose_resources
+ *
+ * in print function,
+ * for each client pid, go through resources and print any for that pid, clear the res
+ * print any remaining detached resources (orphan, dispose)
+ *
+ * sanlock client host_status <lockspace_name>
+ * send sp->host_status[] for the named lockspace
+ */
+
static void cmd_status(int fd, struct sm_header *h_recv)
{
struct sm_header h;
diff --git a/src/token_manager.c b/src/token_manager.c
index 4d447a6..a15e47a 100644
--- a/src/token_manager.c
+++ b/src/token_manager.c
@@ -39,11 +39,13 @@ static pthread_t resource_pt;
static int resource_thread_stop;
static int resource_examine;
+#define R_EXAMINE 0x00000001
+
struct resource {
struct list_head list;
struct token *token;
int pid;
- int examine;
+ uint32_t flags;
uint64_t lver;
struct sanlk_resource r;
};
@@ -59,7 +61,7 @@ int set_resource_examine(char *space_name, char *res_name)
continue;
if (res_name && strncmp(r->r.name, res_name, NAME_ID_SIZE))
continue;
- r->examine = 1;
+ r->flags |= R_EXAMINE;
resource_examine = 1;
count++;
}
@@ -75,7 +77,7 @@ static struct resource *find_resource_examine(void)
struct resource *r;
list_for_each_entry(r, &resources, list) {
- if (r->examine)
+ if (r->flags & R_EXAMINE)
return r;
}
return NULL;
@@ -421,7 +423,7 @@ static void *resource_thread(void *arg GNUC_UNUSED)
pthread_mutex_unlock(&resource_mutex);
continue;
}
- r->examine = 0;
+ r->flags &= ~R_EXAMINE;
/* we can't safely access r->token here, and
r may be freed after we release mutex, so copy
@@ -482,6 +484,7 @@ void release_token_async(struct token *token)
r = find_resource(token, &resources);
if (r) {
/* assert r->token == token ? */
+
if (token->space_dead || (token->acquire_result != SANLK_OK)) {
_del_resource(r);
free(token);
12 years, 2 months
4 commits - src/host_id.c src/host_id.h src/main.c src/paxos_lease.c src/sanlock.8 src/sanlock_internal.h
by David Teigland
src/host_id.c | 100 +++++++++++++++++++---------------
src/host_id.h | 11 ++-
src/main.c | 6 +-
src/paxos_lease.c | 144 +++++++++++++++++++++++++++----------------------
src/sanlock.8 | 96 +++++++++++++++++++++++++++++++-
src/sanlock_internal.h | 6 +-
6 files changed, 241 insertions(+), 122 deletions(-)
New commits:
commit 63faee433069cbb9e3c16a85c3a66c2be0d33e75
Author: David Teigland <teigland(a)redhat.com>
Date: Fri Sep 2 16:17:21 2011 -0500
sanlock: optimize paxos wait
When paxos_lease_acquire waits for host_dead_seconds
for a delta lease to expire, the start of our waiting
time can be the last timestamp update we saved from
the host during our host_id renewal. This avoids
always waiting for the full host_dead_seconds and
could often mean waiting no additional time.
diff --git a/src/host_id.c b/src/host_id.c
index ecab8fd..1b82eeb 100644
--- a/src/host_id.c
+++ b/src/host_id.c
@@ -236,6 +236,29 @@ int host_status_set_bit(char *space_name, uint64_t host_id)
return 0;
}
+int host_info(char *space_name, uint64_t host_id, struct host_status *hs_out)
+{
+ struct space *sp;
+ int found = 0;
+
+ if (!host_id || host_id > DEFAULT_MAX_HOSTS)
+ return -EINVAL;
+
+ pthread_mutex_lock(&spaces_mutex);
+ list_for_each_entry(sp, &spaces, list) {
+ if (strncmp(sp->space_name, space_name, NAME_ID_SIZE))
+ continue;
+ memcpy(hs_out, &sp->host_status[host_id-1], sizeof(struct host_status));
+ found = 1;
+ break;
+ }
+ pthread_mutex_unlock(&spaces_mutex);
+
+ if (!found)
+ return -ENOSPC;
+ return 0;
+}
+
static void create_bitmap(struct task *task, struct space *sp, char *bitmap)
{
uint64_t now;
@@ -263,17 +286,6 @@ static void create_bitmap(struct task *task, struct space *sp, char *bitmap)
pthread_mutex_unlock(&sp->mutex);
}
-/*
- * when entering the monitor loop in paxos_lease, once
- * last_check - last_live > host_dead_seconds, it's expired
- *
- * at local time t=last_live, we read timstamp=X
- * at local time t=last_check, we read timestamp=X
- * so once the difference between last_live and last_check
- * is > host_dead_seconds, the host has not renewed it's
- * timestamp in host_dead_seconds.
- */
-
void check_other_leases(struct task *task, struct space *sp, char *buf)
{
struct leader_record *leader;
diff --git a/src/host_id.h b/src/host_id.h
index 5fab69d..fba2d03 100644
--- a/src/host_id.h
+++ b/src/host_id.h
@@ -14,6 +14,7 @@ int _lockspace_info(char *space_name, struct space *sp_out);
int lockspace_info(char *space_name, struct space *sp_out);
int lockspace_disk(char *space_name, struct sync_disk *disk);
void block_watchdog_updates(char *space_name);
+int host_info(char *space_name, uint64_t host_id, struct host_status *hs_out);
int host_status_set_bit(char *space_name, uint64_t host_id);
int host_status_clear_bit(char *space_name, uint64_t host_id);
int test_id_bit(int host_id, char *bitmap);
diff --git a/src/paxos_lease.c b/src/paxos_lease.c
index 7323196..e7494e8 100644
--- a/src/paxos_lease.c
+++ b/src/paxos_lease.c
@@ -1011,15 +1011,16 @@ int paxos_lease_acquire(struct task *task,
struct leader_record new_leader;
struct paxos_dblock our_dblock;
struct paxos_dblock dblock;
- time_t start;
+ struct host_status hs;
+ uint64_t wait_start, now;
+ uint64_t last_timestamp;
uint64_t next_lver;
uint64_t our_mbal = 0;
- uint64_t last_timestamp = 0;
int copy_cur_leader = 0;
int disk_open = 0;
int error, rv, us;
- log_token(token, "paxos_acquire begin acquire_lver %llu flags %x",
+ log_token(token, "paxos_acquire begin lver %llu flags %x",
(unsigned long long)acquire_lver, flags);
restart:
@@ -1063,10 +1064,6 @@ int paxos_lease_acquire(struct task *task,
* its watchdog has triggered and we can go for the paxos lease.
*/
- log_token(token, "paxos_acquire check owner_id %llu gen %llu",
- (unsigned long long)cur_leader.owner_id,
- (unsigned long long)cur_leader.owner_generation);
-
if (!disk_open) {
memset(&host_id_disk, 0, sizeof(host_id_disk));
@@ -1085,17 +1082,28 @@ int paxos_lease_acquire(struct task *task,
error = SANLK_ACQUIRE_IDDISK;
goto out;
}
+ }
- log_token(token, "paxos_acquire lockspace %.48s "
- "path %s offset %llu sector_size %u fd %d",
- cur_leader.space_name,
- host_id_disk.path,
- (unsigned long long)host_id_disk.offset,
- host_id_disk.sector_size,
- host_id_disk.fd);
+ rv = host_info(cur_leader.space_name, cur_leader.owner_id, &hs);
+ if (!rv && hs.last_check && hs.last_live &&
+ hs.owner_id == cur_leader.owner_id &&
+ hs.owner_generation == cur_leader.owner_generation) {
+ wait_start = hs.last_live;
+ last_timestamp = hs.timestamp;
+ } else {
+ wait_start = monotime();
+ last_timestamp = 0;
}
- start = monotime();
+ log_token(token, "paxos_acquire owner %llu %llu %llu "
+ "host_status %llu %llu %llu wait_start %llu",
+ (unsigned long long)cur_leader.owner_id,
+ (unsigned long long)cur_leader.owner_generation,
+ (unsigned long long)cur_leader.timestamp,
+ (unsigned long long)hs.owner_id,
+ (unsigned long long)hs.owner_generation,
+ (unsigned long long)hs.timestamp,
+ (unsigned long long)wait_start);
while (1) {
error = delta_lease_leader_read(task, &host_id_disk,
@@ -1104,9 +1112,14 @@ int paxos_lease_acquire(struct task *task,
&host_id_leader,
"paxos_acquire");
if (error < 0) {
- log_errot(token, "paxos_acquire host_id %llu read %d",
+ log_errot(token, "paxos_acquire owner %llu %llu %llu "
+ "delta read %d fd %d path %s off %llu ss %u",
(unsigned long long)cur_leader.owner_id,
- error);
+ (unsigned long long)cur_leader.owner_generation,
+ (unsigned long long)cur_leader.timestamp,
+ error, host_id_disk.fd, host_id_disk.path,
+ (unsigned long long)host_id_disk.offset,
+ host_id_disk.sector_size);
goto out;
}
@@ -1116,62 +1129,82 @@ int paxos_lease_acquire(struct task *task,
and acquiring cannot take less than host_dead_seconds */
if (host_id_leader.timestamp == LEASE_FREE) {
- log_token(token, "paxos_acquire host_id %llu free",
+ log_token(token, "paxos_acquire owner %llu delta free",
(unsigned long long)cur_leader.owner_id);
goto run;
}
/* another host has acquired the host_id of the host that
owned this paxos lease; acquiring a host_id also cannot be
- done in less than host_dead_seconds */
-
- if (host_id_leader.owner_id != cur_leader.owner_id) {
- log_token(token, "paxos_acquire host_id %llu owner %llu",
- (unsigned long long)cur_leader.owner_id,
- (unsigned long long)host_id_leader.owner_id);
- goto run;
- }
+ done in less than host_dead_seconds, or
- /* the host_id that owns this lease may be alive, but it
+ the host_id that owns this lease may be alive, but it
owned the lease in a previous generation without freeing it,
and no longer owns it */
- if (host_id_leader.owner_generation > cur_leader.owner_generation) {
- log_token(token, "paxos_acquire host_id %llu "
- "generation now %llu old %llu",
+ if (host_id_leader.owner_id != cur_leader.owner_id ||
+ host_id_leader.owner_generation > cur_leader.owner_generation) {
+ log_errot(token, "paxos_acquire owner %llu %llu %llu "
+ "delta %llu %llu %llu mismatch",
(unsigned long long)cur_leader.owner_id,
+ (unsigned long long)cur_leader.owner_generation,
+ (unsigned long long)cur_leader.timestamp,
+ (unsigned long long)host_id_leader.owner_id,
(unsigned long long)host_id_leader.owner_generation,
- (unsigned long long)cur_leader.owner_generation);
+ (unsigned long long)host_id_leader.timestamp);
goto run;
}
- /* if the owner hasn't renewed its host_id lease for
- host_dead_seconds then its watchdog should have fired
- by now */
-
- if (monotime() - start > task->host_dead_seconds) {
- log_token(token, "paxos_acquire host_id %llu expired %llu",
- (unsigned long long)cur_leader.owner_id,
- (unsigned long long)host_id_leader.timestamp);
- goto run;
+ if (!last_timestamp) {
+ last_timestamp = host_id_leader.timestamp;
+ goto skip_live_check;
}
/* the owner is renewing its host_id so it's alive */
- if (last_timestamp && (host_id_leader.timestamp != last_timestamp)) {
+ if (host_id_leader.timestamp != last_timestamp) {
if (flags & PAXOS_ACQUIRE_QUIET_FAIL) {
- log_token(token, "paxos_acquire host_id %llu alive",
- (unsigned long long)cur_leader.owner_id);
+ log_token(token, "paxos_acquire owner %llu "
+ "delta %llu %llu %llu alive",
+ (unsigned long long)cur_leader.owner_id,
+ (unsigned long long)host_id_leader.owner_id,
+ (unsigned long long)host_id_leader.owner_generation,
+ (unsigned long long)host_id_leader.timestamp);
} else {
- log_errot(token, "paxos_acquire host_id %llu alive",
- (unsigned long long)cur_leader.owner_id);
+ log_errot(token, "paxos_acquire owner %llu "
+ "delta %llu %llu %llu alive",
+ (unsigned long long)cur_leader.owner_id,
+ (unsigned long long)host_id_leader.owner_id,
+ (unsigned long long)host_id_leader.owner_generation,
+ (unsigned long long)host_id_leader.timestamp);
}
error = SANLK_ACQUIRE_IDLIVE;
goto out;
}
- last_timestamp = host_id_leader.timestamp;
+ /* if the owner hasn't renewed its host_id lease for
+ host_dead_seconds then its watchdog should have fired
+ by now */
+
+ now = monotime();
+
+ if (now - wait_start > task->host_dead_seconds) {
+ log_token(token, "paxos_acquire owner %llu %llu %llu "
+ "delta %llu %llu %llu dead %llu-%llu>%d",
+ (unsigned long long)cur_leader.owner_id,
+ (unsigned long long)cur_leader.owner_generation,
+ (unsigned long long)cur_leader.timestamp,
+ (unsigned long long)host_id_leader.owner_id,
+ (unsigned long long)host_id_leader.owner_generation,
+ (unsigned long long)host_id_leader.timestamp,
+ (unsigned long long)now,
+ (unsigned long long)wait_start,
+ task->host_dead_seconds);
+ goto run;
+ }
+
+ skip_live_check:
/* TODO: test with sleep(2) here */
sleep(1);
commit af80a0f1ef076010ad98c9fb57e9188918269d61
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Sep 1 17:28:32 2011 -0500
sanlock: renaming functions
get_space_info() -> lockspace_info()
struct host_info -> struct host_status
diff --git a/src/host_id.c b/src/host_id.c
index badccd9..ecab8fd 100644
--- a/src/host_id.c
+++ b/src/host_id.c
@@ -123,7 +123,7 @@ static struct space *_search_space(char *name,
return NULL;
}
-int _get_space_info(char *space_name, struct space *sp_out)
+int _lockspace_info(char *space_name, struct space *sp_out)
{
struct space *sp;
@@ -136,35 +136,24 @@ int _get_space_info(char *space_name, struct space *sp_out)
return -1;
}
-int get_space_info(char *space_name, struct space *sp_out)
+int lockspace_info(char *space_name, struct space *sp_out)
{
int rv;
pthread_mutex_lock(&spaces_mutex);
- rv = _get_space_info(space_name, sp_out);
+ rv = _lockspace_info(space_name, sp_out);
pthread_mutex_unlock(&spaces_mutex);
return rv;
}
-void block_watchdog_updates(char *space_name)
-{
- struct space *sp;
-
- pthread_mutex_lock(&spaces_mutex);
- sp = _search_space(space_name, NULL, 0, &spaces, NULL, NULL);
- if (sp)
- sp->block_watchdog_updates = 1;
- pthread_mutex_unlock(&spaces_mutex);
-}
-
-int host_id_disk_info(char *name, struct sync_disk *disk)
+int lockspace_disk(char *space_name, struct sync_disk *disk)
{
struct space space;
int rv;
pthread_mutex_lock(&spaces_mutex);
- rv = _get_space_info(name, &space);
+ rv = _lockspace_info(space_name, &space);
if (!rv) {
memcpy(disk, &space.host_id_disk, sizeof(struct sync_disk));
disk->fd = -1;
@@ -174,6 +163,17 @@ int host_id_disk_info(char *name, struct sync_disk *disk)
return rv;
}
+void block_watchdog_updates(char *space_name)
+{
+ struct space *sp;
+
+ pthread_mutex_lock(&spaces_mutex);
+ sp = _search_space(space_name, NULL, 0, &spaces, NULL, NULL);
+ if (sp)
+ sp->block_watchdog_updates = 1;
+ pthread_mutex_unlock(&spaces_mutex);
+}
+
#if 0
static void clear_bit(int host_id, char *bitmap)
{
@@ -210,7 +210,7 @@ int test_id_bit(int host_id, char *bitmap)
return (*byte & mask);
}
-int host_info_set_bit(char *space_name, uint64_t host_id)
+int host_status_set_bit(char *space_name, uint64_t host_id)
{
struct space *sp;
int found = 0;
@@ -231,7 +231,7 @@ int host_info_set_bit(char *space_name, uint64_t host_id)
return -ENOSPC;
pthread_mutex_lock(&sp->mutex);
- sp->host_info[host_id-1].set_bit_time = monotime();
+ sp->host_status[host_id-1].set_bit_time = monotime();
pthread_mutex_unlock(&sp->mutex);
return 0;
}
@@ -249,12 +249,12 @@ static void create_bitmap(struct task *task, struct space *sp, char *bitmap)
if (i+1 == sp->host_id)
continue;
- if (!sp->host_info[i].set_bit_time)
+ if (!sp->host_status[i].set_bit_time)
continue;
- if (now - sp->host_info[i].set_bit_time > task->request_finish_seconds) {
+ if (now - sp->host_status[i].set_bit_time > task->request_finish_seconds) {
log_space(sp, "bitmap clear host_id %d", i+1);
- sp->host_info[i].set_bit_time = 0;
+ sp->host_status[i].set_bit_time = 0;
} else {
set_id_bit(i+1, bitmap, &c);
log_space(sp, "bitmap set host_id %d byte %x", i+1, c);
@@ -278,7 +278,7 @@ void check_other_leases(struct task *task, struct space *sp, char *buf)
{
struct leader_record *leader;
struct sync_disk *disk;
- struct host_info *info;
+ struct host_status *hs;
char *bitmap;
uint64_t now;
int i, new;
@@ -292,21 +292,21 @@ void check_other_leases(struct task *task, struct space *sp, char *buf)
if (i+1 == sp->host_id)
continue;
- info = &sp->host_info[i];
- info->last_check = now;
+ hs = &sp->host_status[i];
+ hs->last_check = now;
leader = (struct leader_record *)(buf + (i * disk->sector_size));
- if (info->owner_id == leader->owner_id &&
- info->owner_generation == leader->owner_generation &&
- info->timestamp == leader->timestamp) {
+ if (hs->owner_id == leader->owner_id &&
+ hs->owner_generation == leader->owner_generation &&
+ hs->timestamp == leader->timestamp) {
continue;
}
- info->owner_id = leader->owner_id;
- info->owner_generation = leader->owner_generation;
- info->timestamp = leader->timestamp;
- info->last_live = now;
+ hs->owner_id = leader->owner_id;
+ hs->owner_generation = leader->owner_generation;
+ hs->timestamp = leader->timestamp;
+ hs->last_live = now;
bitmap = (char *)leader + HOSTID_BITMAP_OFFSET;
@@ -316,11 +316,11 @@ void check_other_leases(struct task *task, struct space *sp, char *buf)
/* this host has made a request for us, we won't take a new
request from this host for another request_finish_seconds */
- if (now - info->last_req < task->request_finish_seconds)
+ if (now - hs->last_req < task->request_finish_seconds)
continue;
log_space(sp, "request from host_id %d", i+1);
- info->last_req = now;
+ hs->last_req = now;
new = 1;
}
diff --git a/src/host_id.h b/src/host_id.h
index ea5c225..5fab69d 100644
--- a/src/host_id.h
+++ b/src/host_id.h
@@ -10,12 +10,12 @@
#define __HOST_ID__H__
int print_space_state(struct space *sp, char *str);
-int _get_space_info(char *space_name, struct space *sp_out);
-int get_space_info(char *space_name, struct space *sp_out);
+int _lockspace_info(char *space_name, struct space *sp_out);
+int lockspace_info(char *space_name, struct space *sp_out);
+int lockspace_disk(char *space_name, struct sync_disk *disk);
void block_watchdog_updates(char *space_name);
-int host_id_disk_info(char *name, struct sync_disk *disk);
-int host_info_set_bit(char *space_name, uint64_t host_id);
-int host_info_clear_bit(char *space_name, uint64_t host_id);
+int host_status_set_bit(char *space_name, uint64_t host_id);
+int host_status_clear_bit(char *space_name, uint64_t host_id);
int test_id_bit(int host_id, char *bitmap);
int check_our_lease(struct task *task, struct space *sp, int *check_all, char *check_buf);
void check_other_leases(struct task *task, struct space *sp, char *buf);
diff --git a/src/main.c b/src/main.c
index ef98a45..f30960b 100644
--- a/src/main.c
+++ b/src/main.c
@@ -734,7 +734,7 @@ static int check_new_tokens_space(struct client *cl,
for (i = 0; i < new_tokens_count; i++) {
token = new_tokens[i];
- rv = _get_space_info(token->r.lockspace_name, &space);
+ rv = _lockspace_info(token->r.lockspace_name, &space);
if (!rv && !space.killing_pids && space.host_id == token->host_id)
continue;
@@ -928,7 +928,7 @@ static void cmd_acquire(struct task *task, struct cmd_args *ca)
for (i = 0; i < new_tokens_count; i++) {
token = new_tokens[i];
- rv = get_space_info(token->r.lockspace_name, &space);
+ rv = lockspace_info(token->r.lockspace_name, &space);
if (rv < 0 || space.killing_pids) {
log_errot(token, "cmd_acquire %d,%d,%d invalid lockspace "
"found %d failed %d name %.48s",
@@ -1429,7 +1429,7 @@ static void cmd_request(struct task *task, struct cmd_args *ca)
goto reply;
if (owner_id)
- host_info_set_bit(token->r.lockspace_name, owner_id);
+ host_status_set_bit(token->r.lockspace_name, owner_id);
reply:
free(token);
log_debug("cmd_request %d,%d done %d", ca->ci_in, fd, result);
diff --git a/src/paxos_lease.c b/src/paxos_lease.c
index 1d2edf0..7323196 100644
--- a/src/paxos_lease.c
+++ b/src/paxos_lease.c
@@ -1070,7 +1070,7 @@ int paxos_lease_acquire(struct task *task,
if (!disk_open) {
memset(&host_id_disk, 0, sizeof(host_id_disk));
- rv = host_id_disk_info(cur_leader.space_name, &host_id_disk);
+ rv = lockspace_disk(cur_leader.space_name, &host_id_disk);
if (rv < 0) {
log_errot(token, "paxos_acquire no lockspace info %.48s",
cur_leader.space_name);
diff --git a/src/sanlock_internal.h b/src/sanlock_internal.h
index 3d78a0c..70c696d 100644
--- a/src/sanlock_internal.h
+++ b/src/sanlock_internal.h
@@ -103,7 +103,7 @@ struct lease_status {
char *renewal_read_buf;
};
-struct host_info {
+struct host_status {
uint64_t last_check; /* local monotime */
uint64_t last_live; /* local monotime */
uint64_t last_req; /* local monotime */
@@ -126,11 +126,11 @@ struct space {
int external_remove;
int block_watchdog_updates;
int thread_stop;
+ int wd_fd;
pthread_t thread;
pthread_mutex_t mutex; /* protects lease_status, thread_stop */
struct lease_status lease_status;
- int wd_fd;
- struct host_info host_info[DEFAULT_MAX_HOSTS];
+ struct host_status host_status[DEFAULT_MAX_HOSTS];
};
/*
commit b6e7633e8c5a276be7acd8da5e9188c93bd18ce2
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Sep 1 17:10:00 2011 -0500
sanlock: remove old comment
We don't use real time timestamps any longer, only monotonic
timestamps. The comment about comparing real timestamps
among nodes being a problem due to out of sync date/time
settings is not relevant. monotonic timestamp comparisons
among nodes is even more impossible.
diff --git a/src/paxos_lease.c b/src/paxos_lease.c
index b5a7282..1d2edf0 100644
--- a/src/paxos_lease.c
+++ b/src/paxos_lease.c
@@ -1147,16 +1147,7 @@ int paxos_lease_acquire(struct task *task,
/* if the owner hasn't renewed its host_id lease for
host_dead_seconds then its watchdog should have fired
- by now
-
- if we trust that the clocks are in sync among hosts, then this
- check could be: if (time() - host_id_leader.timestamp >
- task->host_dead_seconds), but if the clocks are out of sync,
- this check would easily give two hosts the lease.
-
- N.B. we need to be careful about ever comparing local time
- to a time value we read off disk from another node that may
- have different time. */
+ by now */
if (monotime() - start > task->host_dead_seconds) {
log_token(token, "paxos_acquire host_id %llu expired %llu",
@@ -1164,14 +1155,6 @@ int paxos_lease_acquire(struct task *task,
(unsigned long long)host_id_leader.timestamp);
goto run;
}
-#if 0
- if (time(NULL) - host_id_leader.timestamp > task->host_dead_seconds) {
- log_token(token, "paxos_acquire host_id %llu expired %llu",
- (unsigned long long)cur_leader.owner_id,
- (unsigned long long)host_id_leader.timestamp);
- goto run;
- }
-#endif
/* the owner is renewing its host_id so it's alive */
commit 5859f32c8503e21db9bece4024341b4111dbc1c9
Author: David Teigland <teigland(a)redhat.com>
Date: Thu Sep 1 16:30:40 2011 -0500
sanlock: update man page
diff --git a/src/sanlock.8 b/src/sanlock.8
index 6a9787a..f004159 100644
--- a/src/sanlock.8
+++ b/src/sanlock.8
@@ -257,6 +257,18 @@ Print the sanlock daemon internal debug log.
Tell the sanlock daemon to exit. Any registered processes will be killed,
their resource leases released, and lockspaces removed.
+.BR "sanlock client init -s" " LOCKSPACE"
+.br
+.BR "sanlock client init -r" " RESOURCE"
+
+Tell the sanlock daemon to initialize storage for lease areas.
+(See sanlock direct init.)
+
+.BR "sanlock client align -s" " LOCKSPACE"
+
+Tell the sanlock daemon to report the required lease alignment for a storage
+path. Only path is used from the LOCKSPACE argument.
+
.BR "sanlock client add_lockspace -s" " LOCKSPACE"
Tell the sanlock daemon to acquire the specified host_id in the lockspace.
@@ -292,6 +304,23 @@ Print the resource leases held the given pid. The format is a versioned
RESOURCE string "RESOURCE:lver" where lver is the version of the lease
held.
+.BR "sanlock client request -r" " RESOURCE " \
+\fB-f\fP " " \fIforce_mode\fP
+
+Request the owner of a resource do something specified by force_mode. A
+versioned RESOURCE:lver string must be used with a greater version than is
+presently held. Zero lver and force_mode clears the request.
+
+.BR "sanlock client examine -r" " RESOURCE"
+
+Examine the request record for the currently held resource lease and carry
+out the action specified by the requested force_mode.
+
+.BR "sanlock client examine -s" " LOCKSPACE"
+
+Examine requests for all resource leases currently held in the named
+lockspace. Only lockspace_name is used from the LOCKSPACE argument.
+
.B "sanlock direct"
.I action
[options]
@@ -353,7 +382,9 @@ sanlock daemon's conclusion unless the configured timeouts match.)
.BI "sanlock direct dump" " path" \
\fR[\fP\fB:\fP\fIoffset\fP\fR]\fP
-Read disk sectors and print leader records for delta or paxos leases.
+Read disk sectors and print leader records for delta or paxos leases. Add
+-f 1 to print the request record values for paxos leases, and host_ids set
+in delta lease bitmaps.
.SS
LOCKSPACE option string
@@ -381,8 +412,13 @@ RESOURCE option string
.IR path " path to storage reserved for leases"
.br
.IR offset " offset on path (bytes)"
-.br
-.IR lver " optional leader version"
+
+.SS
+RESOURCE option string with version
+
+.BR \-r " " \fIlockspace_name\fP:\fIresource_name\fP:\fIpath\fP:\fIoffset\fP:\fIlver\fP
+.P
+.IR lver " leader version"
.SS Defaults
@@ -392,6 +428,60 @@ shows the default values for the options above.
.B sanlock version
shows the build version.
+.SH USAGE
+
+.SS Request/Examine
+
+The first part of making a request for a resource is writing the request
+record of the resource (the sector following the leader record). To make
+a successful request:
+.IP \(bu 3
+RESOURCE:lver must be greater than the lver presently held by the other
+host. This implies the leader record must be read to discover the lver,
+prior to making a request.
+.IP \(bu 3
+RESOURCE:lver must be greater than or equal to the lver presently
+written to the request record. Two hosts may write a new request at the
+same time for the same lver, in which case both would succeed, but the
+force_mode from the last would win.
+.IP \(bu 3
+The force_mode must be greater than zero.
+.IP \(bu 3
+To unconditionally clear the request record (set both lver and
+force_mode to 0), make request with RESOURCE:0 and force_mode 0.
+.PP
+
+The owner of the requested resource will not know of the request unless it
+is explicitly told to examine its resources via the "examine" api/command,
+or otherwise notfied.
+
+The second part of making a request is notifying the resource lease owner
+that it should examine the request records of its resource leases. The
+notification will cause the lease owner to automatically run the
+equivalent of "sanlock client examine -s LOCKSPACE" for the lockspace of
+the requested resource.
+
+The notification is made using a bitmap in each host_id delta lease. Each
+bit represents each of the possible host_ids (1-2000). If host A wants to
+notify host B to examine its resources, A sets the bit in its own bitmap
+that corresponds to the host_id of B. When B next renews its delta lease,
+it reads the delta leases for all hosts and checks each bitmap to see if
+its own host_id has been set. It finds the bit for its own host_id set in
+A's bitmap, and examines its resource request records. (The bit remains
+set in A's bitmap for request_finish_seconds.)
+
+\fIforce_mode\fP determines the action the resource lease owner should
+take:
+
+\fB1\fP (KILL_PID): kill the process holding the resource lease. When the
+process has exited, the resource lease will be released, and can then be
+acquired by anyone.
+
+\fB2\fP (BLOCK_WD): stop updating the watchdog (/dev/watchdog keepalive
+via wdmd_test_live) for the lockspace, which will lead to /dev/watchdog
+firing and reseting the host. The resource lease can be acquired after
+the timeout for a failed host.
+
.SH SEE ALSO
.BR wdmd (8)
12 years, 3 months