00010000:00010000:3.0F:1715206680.079660:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:2.0F:1715206680.359547:0:4326:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=1b6f0f0 00800000:00000080:2.0:1715206680.359591:0:4326:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a92c5860 uarg=0000000000000000 00000002:00000080:2.0:1715206680.359599:0:4326:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a92c5860 uarg=0000000000000000 00000100:00100000:2.0:1715206680.359635:0:4326:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:4326:1798523504932096:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715206680.359695:0:4326:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8fb40 going to sleep for 143 seconds 00000100:00100000:2.0:1715206680.361020:0:4326:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:4326:1798523504932096:192.168.201.123@tcp:60:lhsmtool_posix.0 00000002:00000080:2.0:1715206680.361033:0:4326:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a92c5860 uarg=0000000000000000 00000100:00100000:2.0:1715206680.361046:0:4326:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:4326:1798523504932160:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715206680.361088:0:4326:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8fb40 going to sleep for 143 seconds 00000100:00100000:2.0:1715206680.361663:0:4326:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:4326:1798523504932160:192.168.201.123@tcp:60:lhsmtool_posix.0 00000080:00200008:2.0:1715206680.361696:0:4326:0:(file.c:430:ll_file_release()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:2.0:1715206680.361722:0:4326:0:(mdc_request.c:927:mdc_close()) @@@ matched open req@ffff8800a967b100 x1798523504931648/t4294967388(4294967388) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206826 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000020:00100000:2.0:1715206680.361739:0:4326:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1715206680.361743:0:4326:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800abe2f800 x1798523504932288/t0(0) o35->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:23/10 lens 392/840 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00000100:00100000:2.0:1715206680.361754:0:4326:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:4326:1798523504932288:192.168.201.123@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1715206680.361784:0:4326:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1715206680.362650:0:4326:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:4326:1798523504932288:192.168.201.123@tcp:35:lhsmtool_posix.0 00000002:00100000:3.0:1715206680.362663:0:4326:0:(mdc_request.c:837:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8800a967b100 x1798523504931648/t4294967388(4294967388) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206826 ref 1 fl Complete:RQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200008:3.0:1715206680.362684:0:4326:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200008:2.0:1715206680.365019:0:4326:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00010000:00010000:1.0F:1715206681.080699:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1715206682.082682:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1715206682.389790:0:5120:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12c: Restore a file with stripe of 2 == 18:18:02 (1715206682) 00000080:00200008:3.0:1715206682.499587:0:5163:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715206682.499599:0:5163:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715206682.499623:0:5163:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.499641:0:5163:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206682.499657:0:5163:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715206682.499669:0:5163:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715206682.499673:0:5163:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:3.0:1715206682.499680:0:5163:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a9673cc8 00010000:00010000:3.0:1715206682.499684:0:5163:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206682.499693:0:5163:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200008:3.0:1715206682.500312:0:5163:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715206682.500322:0:5163:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206682.500340:0:5163:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.500393:0:5163:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500423:0:5163:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500434:0:5163:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5163 initiator: MDT0 00000100:00100000:3.0:1715206682.500453:0:5163:0:(client.c:2542:ptlrpc_set_wait()) set ffff880137369480 going to sleep for 0 seconds 00000100:00100000:3.0:1715206682.500465:0:5163:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413100 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:04532c2b-e5e9-420d-abb0-4480aab1e378:5163:1798523504932352:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1715206682.500476:0:5163:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500484:0:5163:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500489:0:5163:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500495:0:5163:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500503:0:5163:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500509:0:5163:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54a640 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206682.500514:0:5163:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500520:0:5163:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500527:0:5163:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500542:0:5163:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206682.500548:0:5163:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5163 initiator: MDT0 00010000:00010000:3.0:1715206682.500554:0:5163:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a640/0x7561918bd556fd9b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5163 initiator: MDT0 00000080:00020000:3.0:1715206682.500605:0:5163:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:3.0:1715206682.500628:0:5163:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715206682.500633:0:5163:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715206682.500642:0:5163:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.500650:0:5163:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206682.500660:0:5163:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715206682.500667:0:5163:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715206682.500670:0:5163:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:3.0:1715206682.500674:0:5163:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a9673cc8 00010000:00010000:3.0:1715206682.500677:0:5163:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206682.500685:0:5163:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715206682.500773:0:5163:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715206682.500784:0:5163:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.500791:0:5163:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206682.500805:0:5163:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715206682.500813:0:5163:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715206682.500815:0:5163:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:3.0:1715206682.500819:0:5163:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a9673b80 00010000:00010000:3.0:1715206682.500823:0:5163:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206682.500830:0:5163:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715206682.500839:0:5163:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:3.0:1715206682.500846:0:5163:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715206682.501743:0:5164:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715206682.501764:0:5164:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206682.501778:0:5164:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206682.501793:0:5164:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715206682.501804:0:5164:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715206682.501807:0:5164:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:1.0:1715206682.501812:0:5164:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a921fb80 00010000:00010000:1.0:1715206682.501816:0:5164:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206682.501824:0:5164:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715206682.501835:0:5164:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:1.0:1715206682.501841:0:5164:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715206682.501869:0:5164:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715206682.501876:0:5164:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206682.501882:0:5164:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206682.501889:0:5164:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715206682.501897:0:5164:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715206682.501906:0:5164:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:1.0:1715206682.501910:0:5164:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a921fbb0 00010000:00010000:1.0:1715206682.501913:0:5164:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206682.501920:0:5164:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715206682.501928:0:5164:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715206682.501934:0:5164:0:(dcache.c:318:ll_revalidate_dentry()) VFS Op:name=.lustre, flags=17 00000080:00200000:1.0:1715206682.501939:0:5164:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x1:0x0](ffff880136f410c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715206682.501947:0:5164:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), file ffff8800b605fc80, open_flags 8000, mode 0 opened 0 00000080:00200000:1.0:1715206682.501954:0:5164:0:(namei.c:838:ll_lookup_it()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), intent=open 00000002:00010000:1.0:1715206682.501964:0:5164:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: fid,[0x0:0x0:0x0]) in obj [0x200000002:0x1:0x0], intent: open flags 0100001 00010000:00010000:1.0:1715206682.502007:0:5164:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.502019:0:5164:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 5164 initiator: MDT0 00000020:00100000:1.0:1715206682.502071:0:5164:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1715206682.502083:0:5164:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a967bb80 x1798523504932480/t0(0) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/66536 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00010000:00010000:1.0:1715206682.502099:0:5164:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 5164 initiator: MDT0 00000100:00100000:1.0:1715206682.502121:0:5164:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:5164:1798523504932480:192.168.201.123@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1715206682.502279:0:5164:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd840 going to sleep for 143 seconds 00000100:00100000:1.0:1715206682.504629:0:5164:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:5164:1798523504932480:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1715206682.504648:0:5164:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504656:0:5164:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504662:0:5164:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504668:0:5164:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504674:0:5164:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504679:0:5164:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa6698c0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715206682.504683:0:5164:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504689:0:5164:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504695:0:5164:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 5/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504700:0:5164:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206682.504704:0:5164:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 5164 initiator: MDT0 00010000:00010000:1.0:1715206682.504709:0:5164:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa6698c0/0x7561918bd556fda2 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 5164 initiator: MDT0 00000002:00100000:1.0:1715206682.504723:0:5164:0:(mdc_locks.c:838:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff8800a967bb80 x1798523504932480/t4294967390(4294967390) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206830 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000002:00100000:1.0:1715206682.504748:0:5164:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a967bb80 x1798523504932480/t4294967390(4294967390) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206830 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:1.0:1715206682.504771:0:5164:0:(llite_lib.c:2872:ll_update_inode()) inode=[0x200000002:0x2:0x0], updating i_size 4096 00000080:00200000:1.0:1715206682.504775:0:5164:0:(namei.c:157:ll_iget()) got inode: [0x200000002:0x2:0x0](ffff8800b02788c8): rc = 0 00000080:00010000:1.0:1715206682.504779:0:5164:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000002:0x2:0x0](ffff8800b02788c8) for lock 0x0 00000080:00200008:1.0:1715206682.504793:0:5164:0:(file.c:871:ll_file_open()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:1.0:1715206682.504806:0:5164:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a967bb80 x1798523504932480/t4294967390(4294967390) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206830 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:1.0:1715206682.504854:0:5164:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x2:0x0](ffff8800b02788c8), inode mode 4040 mask 44 00000080:00200080:1.0:1715206682.504905:0:5164:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=7270f0 00800000:00000080:1.0:1715206682.504916:0:5164:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff88012e1a8140 uarg=0000000000000000 00000002:00000080:1.0:1715206682.504925:0:5164:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff88012e1a8140 uarg=0000000000000000 00000100:00100000:1.0:1715206682.504949:0:5164:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:5164:1798523504932544:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1715206682.504999:0:5164:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd900 going to sleep for 143 seconds 00000100:00100000:1.0:1715206682.506484:0:5164:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:5164:1798523504932544:192.168.201.123@tcp:59:lhsmtool_posix.0 00000002:00000080:1.0:1715206682.506504:0:5164:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff88012e1a8140 uarg=0000000000000000 00000100:00100000:1.0:1715206682.506540:0:5164:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:5164:1798523504932608:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1715206682.506618:0:5164:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd900 going to sleep for 143 seconds 00000100:00100000:1.0:1715206682.507974:0:5164:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:5164:1798523504932608:192.168.201.123@tcp:59:lhsmtool_posix.0 00000080:00200000:3.0:1715206682.511375:0:5165:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206682.511394:0:5165:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.511433:0:5165:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511447:0:5165:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511461:0:5165:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00000100:00100000:3.0:1715206682.511478:0:5165:0:(client.c:2542:ptlrpc_set_wait()) set ffff880137369480 going to sleep for 0 seconds 00000100:00100000:3.0:1715206682.511489:0:5165:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413100 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:5165:1798523504932672:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715206682.511503:0:5165:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511529:0:5165:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511539:0:5165:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511549:0:5165:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511609:0:5165:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511617:0:5165:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54aac0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206682.511623:0:5165:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511633:0:5165:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511643:0:5165:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511652:0:5165:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206682.511658:0:5165:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511667:0:5165:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54aac0/0x7561918bd556fda9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5165 initiator: MDT0 00000080:00020000:3.0:1715206682.511697:0:5165:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715206682.511736:0:5165:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206682.511750:0:5165:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.511791:0:5165:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511803:0:5165:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511815:0:5165:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00000100:00100000:3.0:1715206682.511831:0:5165:0:(client.c:2542:ptlrpc_set_wait()) set ffff880137369480 going to sleep for 0 seconds 00000100:00100000:3.0:1715206682.511841:0:5165:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413100 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:5165:1798523504932736:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715206682.511853:0:5165:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511863:0:5165:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511872:0:5165:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511882:0:5165:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511893:0:5165:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511902:0:5165:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54af40 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206682.511908:0:5165:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511917:0:5165:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511927:0:5165:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511945:0:5165:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206682.511951:0:5165:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.511960:0:5165:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd556fdb0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5165 initiator: MDT0 00000080:00020000:3.0:1715206682.511986:0:5165:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:3.0:1715206682.512013:0:5165:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715206682.512021:0:5165:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206682.512062:0:5165:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.512091:0:5165:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512102:0:5165:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512113:0:5165:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00000100:00100000:3.0:1715206682.512137:0:5165:0:(client.c:2542:ptlrpc_set_wait()) set ffff880137369480 going to sleep for 0 seconds 00000100:00100000:3.0:1715206682.512146:0:5165:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413100 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:5165:1798523504932800:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715206682.512157:0:5165:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512166:0:5165:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512175:0:5165:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512200:0:5165:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512210:0:5165:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512227:0:5165:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54b600 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206682.512233:0:5165:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512241:0:5165:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512250:0:5165:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512259:0:5165:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206682.512265:0:5165:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5165 initiator: MDT0 00010000:00010000:3.0:1715206682.512275:0:5165:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b600/0x7561918bd556fdb7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5165 initiator: MDT0 00000080:00020000:3.0:1715206682.512302:0:5165:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206682.517877:0:5166:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715206682.517889:0:5166:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206682.517913:0:5166:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517925:0:5166:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517932:0:5166:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5166 initiator: MDT0 00000100:00100000:1.0:1715206682.517941:0:5166:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd900 going to sleep for 0 seconds 00000100:00100000:1.0:1715206682.517947:0:5166:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:5166:1798523504932864:192.168.201.123@tcp:101:lfs.0 00010000:00010000:1.0:1715206682.517955:0:5166:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517960:0:5166:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517965:0:5166:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517970:0:5166:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517975:0:5166:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517979:0:5166:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa66a1c0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715206682.517982:0:5166:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517986:0:5166:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.517991:0:5166:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.518003:0:5166:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206682.518007:0:5166:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5166 initiator: MDT0 00010000:00010000:1.0:1715206682.518011:0:5166:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a1c0/0x7561918bd556fdbe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5166 initiator: MDT0 00000080:00020000:1.0:1715206682.518047:0:5166:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715206682.528606:0:5170:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206682.528625:0:5170:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.528663:0:5170:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528673:0:5170:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528682:0:5170:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5170 initiator: MDT0 00000100:00100000:3.0:1715206682.528702:0:5170:0:(client.c:2542:ptlrpc_set_wait()) set ffff880137369480 going to sleep for 0 seconds 00000100:00100000:3.0:1715206682.528710:0:5170:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413100 pname:cluuid:pid:xid:nid:opc:job mkdir:04532c2b-e5e9-420d-abb0-4480aab1e378:5170:1798523504932928:192.168.201.123@tcp:101:mkdir.0 00010000:00010000:3.0:1715206682.528721:0:5170:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528728:0:5170:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528734:0:5170:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528740:0:5170:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528746:0:5170:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528752:0:5170:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54b840 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206682.528756:0:5170:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528761:0:5170:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528767:0:5170:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528773:0:5170:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206682.528777:0:5170:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5170 initiator: MDT0 00010000:00010000:3.0:1715206682.528783:0:5170:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fdc5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5170 initiator: MDT0 00000080:00020000:3.0:1715206682.528807:0:5170:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206682.532910:0:5171:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715206682.532927:0:5171:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206682.532957:0:5171:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.532966:0:5171:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.532975:0:5171:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5171 initiator: MDT0 00000100:00100000:1.0:1715206682.532987:0:5171:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd900 going to sleep for 0 seconds 00000100:00100000:1.0:1715206682.532995:0:5171:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job rm:04532c2b-e5e9-420d-abb0-4480aab1e378:5171:1798523504932992:192.168.201.123@tcp:101:rm.0 00010000:00010000:1.0:1715206682.533005:0:5171:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533011:0:5171:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533018:0:5171:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533049:0:5171:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533062:0:5171:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533068:0:5171:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa66a400 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715206682.533072:0:5171:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533077:0:5171:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533084:0:5171:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533089:0:5171:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206682.533093:0:5171:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533099:0:5171:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd556fdcc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5171 initiator: MDT0 00000080:00020000:1.0:1715206682.533119:0:5171:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206682.533165:0:5171:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715206682.533175:0:5171:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206682.533195:0:5171:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533202:0:5171:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533209:0:5171:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5171 initiator: MDT0 00000100:00100000:1.0:1715206682.533218:0:5171:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd900 going to sleep for 0 seconds 00000100:00100000:1.0:1715206682.533230:0:5171:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job rm:04532c2b-e5e9-420d-abb0-4480aab1e378:5171:1798523504933056:192.168.201.123@tcp:101:rm.0 00010000:00010000:1.0:1715206682.533239:0:5171:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533245:0:5171:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533251:0:5171:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533256:0:5171:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533262:0:5171:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533267:0:5171:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa669680 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715206682.533270:0:5171:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533276:0:5171:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533281:0:5171:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533286:0:5171:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206682.533289:0:5171:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5171 initiator: MDT0 00010000:00010000:1.0:1715206682.533294:0:5171:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd556fdd3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5171 initiator: MDT0 00000080:00020000:1.0:1715206682.533308:0:5171:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206682.537560:0:5172:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715206682.537616:0:5172:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206682.537653:0:5172:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537667:0:5172:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537679:0:5172:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5172 initiator: MDT0 00000100:00100000:1.0:1715206682.537696:0:5172:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd900 going to sleep for 0 seconds 00000100:00100000:1.0:1715206682.537707:0:5172:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job dd:04532c2b-e5e9-420d-abb0-4480aab1e378:5172:1798523504933120:192.168.201.123@tcp:101:dd.0 00010000:00010000:1.0:1715206682.537720:0:5172:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537730:0:5172:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537739:0:5172:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537749:0:5172:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537759:0:5172:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537768:0:5172:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa668480 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715206682.537773:0:5172:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537783:0:5172:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537809:0:5172:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537818:0:5172:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206682.537824:0:5172:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5172 initiator: MDT0 00010000:00010000:1.0:1715206682.537834:0:5172:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdda lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5172 initiator: MDT0 00000080:00020000:1.0:1715206682.537862:0:5172:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715206682.544168:0:5173:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206682.544179:0:5173:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206682.544199:0:5173:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544206:0:5173:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544212:0:5173:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5173 initiator: MDT0 00000100:00100000:3.0:1715206682.544219:0:5173:0:(client.c:2542:ptlrpc_set_wait()) set ffff880137369480 going to sleep for 0 seconds 00000100:00100000:3.0:1715206682.544225:0:5173:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413100 pname:cluuid:pid:xid:nid:opc:job md5sum:04532c2b-e5e9-420d-abb0-4480aab1e378:5173:1798523504933184:192.168.201.123@tcp:101:md5sum.0 00010000:00010000:3.0:1715206682.544232:0:5173:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544236:0:5173:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544241:0:5173:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544245:0:5173:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544249:0:5173:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544252:0:5173:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54b840 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206682.544255:0:5173:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544263:0:5173:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544267:0:5173:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544271:0:5173:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206682.544274:0:5173:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5173 initiator: MDT0 00010000:00010000:3.0:1715206682.544277:0:5173:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54b840/0x7561918bd556fde1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5173 initiator: MDT0 00000080:00020000:3.0:1715206682.544290:0:5173:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:2.0:1715206682.549327:0:5174:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:2.0:1715206682.549347:0:5174:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1715206682.549386:0:5174:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549400:0:5174:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549422:0:5174:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5174 initiator: MDT0 00000100:00100000:2.0:1715206682.549450:0:5174:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f180 going to sleep for 0 seconds 00000100:00100000:2.0:1715206682.549460:0:5174:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:5174:1798523504933248:192.168.201.123@tcp:101:lfs.0 00010000:00010000:2.0:1715206682.549472:0:5174:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549482:0:5174:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549491:0:5174:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549510:0:5174:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549520:0:5174:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549529:0:5174:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a923b600 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:2.0:1715206682.549534:0:5174:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549543:0:5174:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549551:0:5174:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549559:0:5174:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1715206682.549599:0:5174:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5174 initiator: MDT0 00010000:00010000:2.0:1715206682.549616:0:5174:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a923b600/0x7561918bd556fde8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5174 initiator: MDT0 00000080:00020000:2.0:1715206682.549643:0:5174:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206682.861370:0:5184:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715206682.861398:0:5184:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206682.861455:0:5184:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861469:0:5184:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861481:0:5184:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5184 initiator: MDT0 00000100:00100000:1.0:1715206682.861502:0:5184:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd180 going to sleep for 0 seconds 00000100:00100000:1.0:1715206682.861516:0:5184:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:5184:1798523504933312:192.168.201.123@tcp:101:lfs.0 00010000:00010000:1.0:1715206682.861530:0:5184:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861539:0:5184:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861547:0:5184:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861591:0:5184:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861602:0:5184:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861610:0:5184:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa668480 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715206682.861616:0:5184:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861624:0:5184:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861634:0:5184:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861642:0:5184:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206682.861648:0:5184:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5184 initiator: MDT0 00010000:00010000:1.0:1715206682.861656:0:5184:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa668480/0x7561918bd556fdef lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 5184 initiator: MDT0 00000080:00020000:1.0:1715206682.861685:0:5184:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000100:00080000:0.0F:1715206683.032683:0:12483:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1715206683.032694:0:12483:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:0.0:1715206683.032706:0:12483:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1715206683.032709:0:12483:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:1.0:1715206683.032710:0:2001:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a96a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2001:1798523504933376:192.168.201.123@tcp:400:kworker.0 00000100:00080000:0.0:1715206683.032712:0:12483:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1715206683.032715:0:12483:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:0.0:1715206683.032740:0:2000:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a96a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2000:1798523504933440:192.168.201.123@tcp:400:kworker.0 00000100:00100000:1.0:1715206683.032801:0:2001:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a96a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2001:1798523504933504:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715206683.034748:0:2001:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2001:1798523504933376:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715206683.034781:0:2001:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2001:1798523504933504:192.168.201.123@tcp:400:kworker.0 00000100:00100000:1.0:1715206683.034813:0:2000:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2000:1798523504933440:192.168.201.123@tcp:400:kworker.0 00010000:00010000:2.0:1715206683.084731:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1715206683.331992:0:5317:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12c: @@@@@@ FAIL: release /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm failed Debug log: 375 lines, 375 kept, 0 dropped, 0 bad.