vgs stucked on acquire VGLK, ballot abort1 larger lver.

[ChangLimin]

Hi,
Sanlock version is 3.8.5.
1. vgs command on many hosts are stucked.
2. sanlock status command on many hosts show "r lvm_KVM01_QS_LUN01:VGLK:/dev/mapper/KVM01_QS_LUN01-lvmlock:553648128:SH p 41595 ADD"; NO host hold the VGLK lock.
3. sanlock direct dump show VGLK owner_id is host 3, write_id is 0x14, lver is 0x024c93
VGLK leader record:
  magic       version      flags       sector_size
21000000  10 20 15 06 04 00 06 00  80 00 00 00 00 10 00 00  |. ..............|
  num_hosts                max_hosts
21000010  d0 07 00 00 00 00 00 00  d0 07 00 00 00 00 00 00  |................|
  owner_id            owner_generation
21000020  03 00 00 00 00 00 00 00  04 00 00 00 00 00 00 00  |................|
  lver            space_name
21000030  93 4c 02 00 00 00 00 00  6c 76 6d 5f 4b 56 4d 30  |.L......lvm_KVM0|
21000040  31 5f 51 53 5f 4c 55 4e  30 31 00 00 00 00 00 00  |1_QS_LUN01......|
   resource_name
21000060  00 00 00 00 00 00 00 00  56 47 4c 4b 00 00 00 00  |........VGLK....|
   timestamp
21000090  00 00 00 00 00 00 00 00  05 fd c6 00 00 00 00 00  |................|
  unused1    checksum    unused2 io_timeout
210000a0  00 00 00 00 00 00 00 00  03 8b d5 c2 00 00 00 00  |................|
  write_id            write_generation
210000b0  14 00 00 00 00 00 00 00  03 00 00 00 00 00 00 00  |................|
  write_timestamp
210000c0  f3 c2 c5 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

4. host 8 dblock lver is 0x024c95 , other host dblock is equal or less than 0x024c94
21009000  58 31 1d 1b 00 00 00 00  58 31 1d 1b 00 00 00 00  |X1......X1......|
21009010  08 00 00 00 00 00 00 00  03 00 00 00 00 00 00 00  |................|
                                                           lver
21009020  b9 e5 c6 00 00 00 00 00  95 4c 02 00 00 00 00 00  |.........L......|
21009030  2d 23 98 f9 01 00 00 00  00 00 00 00 00 00 00 00  |-#..............|

5. host 3 dblock flags is 0, not set DBLOCK_FL_RELEASED in flags
uint64_t mbal uint64_t bal
21004000  e3 3b 81 ab 02 00 00 00  00 00 00 00 00 00 00 00  |.;..............|
uint64_t inp;   /* host_id */
21004020  00 00 00 00 00 00 00 00  94 4c 02 00 00 00 00 00  |.........L......|
checksum  flags
21004030  3b e8 3f a4 00 00 00 00  00 00 00 00 00 00 00 00  |;.?.............|

6. sanlock.log on many hosts show "abort1 larger lver in bk[7]", sorry I have not download the sanlock.log.

Q1. Leader record show host 3 is the owner, why sanlock status not show holding the lock?
Q2. Leader record lver is 0x024c93, Why host 8 dblock lver is 0x024c95 instead of 0x024c94?

Thanks and Best Regards.

Chang Limin