00010000:00010000:1.0F:1715207115.948692:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:2.0F:1715207115.973954:0:14265:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=20690f0 00800000:00000080:2.0:1715207115.973981:0:14265:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a9739fa0 uarg=0000000000000000 00000002:00000080:2.0:1715207115.973990:0:14265:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a9739fa0 uarg=0000000000000000 00000100:00100000:2.0:1715207115.974040:0:14265:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:14265:1798523504965760:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715207115.974123:0:14265:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f180 going to sleep for 11 seconds 00000100:00100000:2.0:1715207115.976045:0:14265:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:14265:1798523504965760:192.168.201.123@tcp:60:lhsmtool_posix.0 00000002:00000080:2.0:1715207115.976065:0:14265:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a9739fa0 uarg=0000000000000000 00000100:00100000:2.0:1715207115.976089:0:14265:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:14265:1798523504965824:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715207115.976139:0:14265:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f180 going to sleep for 11 seconds 00000100:00100000:2.0:1715207115.977385:0:14265:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:14265:1798523504965824:192.168.201.123@tcp:60:lhsmtool_posix.0 00000080:00200008:2.0:1715207115.977429:0:14265:0:(file.c:430:ll_file_release()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:2.0:1715207115.977467:0:14265:0:(mdc_request.c:927:mdc_close()) @@@ matched open req@ffff8800a9679500 x1798523504964864/t4294967400(4294967400) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207128 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000020:00100000:2.0:1715207115.977491:0:14265:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1715207115.977497:0:14265:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800abe2ca80 x1798523504965952/t0(0) o35->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:23/10 lens 392/840 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00000100:00100000:2.0:1715207115.977513:0:14265:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:14265:1798523504965952:192.168.201.123@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1715207115.977595:0:14265:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f540 going to sleep for 11 seconds 00000100:00100000:2.0:1715207115.979224:0:14265:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:14265:1798523504965952:192.168.201.123@tcp:35:lhsmtool_posix.0 00000002:00100000:2.0:1715207115.979250:0:14265:0:(mdc_request.c:837:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8800a9679500 x1798523504964864/t4294967400(4294967400) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207128 ref 1 fl Complete:RQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200008:2.0:1715207115.979285:0:14265:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200008:3.0F:1715207115.982658:0:14265:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00010000:00010000:0.0F:1715207116.950832:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1715207117.952792:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1715207118.636408:0:15491:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12n: Import/implicit restore/release == 18:25:18 (1715207118) 00000100:00080000:1.0:1715207118.728677:0:5:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1715207118.728682:0:5:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:1.0:1715207118.728689:0:5:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1715207118.728692:0:5:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:0.0:1715207118.728697:0:2000:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2000:1798523504966016:192.168.201.123@tcp:400:kworker.0 00000100:00080000:1.0:1715207118.728701:0:5:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1715207118.728703:0:5:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:0.0:1715207118.728765:0:2000:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a967aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2000:1798523504966144:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.728781:0:2000:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9678e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2000:1798523504966272:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.728794:0:2000:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1715207118.728799:0:2000:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9679f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2000:1798523504966080:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.728811:0:2000:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9678700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2000:1798523504966208:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.730690:0:2000:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2000:1798523504966016:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.730719:0:2000:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9679f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2000:1798523504966080:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.730730:0:2000:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2000:1798523504966208:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.730740:0:2000:0:(client.c:2956:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8800a9618000: committing for last_committed 4294967401 gen 1 00000100:00100000:0.0:1715207118.730744:0:2000:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2000:1798523504966144:192.168.201.123@tcp:400:kworker.0 00000100:00100000:0.0:1715207118.730754:0:2000:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2000:1798523504966272:192.168.201.123@tcp:400:kworker.0 00000080:00200008:3.0:1715207118.744396:0:15534:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715207118.744406:0:15534:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715207118.744426:0:15534:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207118.744443:0:15534:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207118.744456:0:15534:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715207118.744466:0:15534:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715207118.744470:0:15534:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:3.0:1715207118.744477:0:15534:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a923fcc8 00010000:00010000:3.0:1715207118.744481:0:15534:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207118.744489:0:15534:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200008:3.0:1715207118.744989:0:15534:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715207118.744995:0:15534:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715207118.745008:0:15534:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207118.745044:0:15534:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745053:0:15534:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745061:0:15534:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15534 initiator: MDT0 00000100:00100000:3.0:1715207118.745083:0:15534:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012a421600 going to sleep for 0 seconds 00000100:00100000:3.0:1715207118.745091:0:15534:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:04532c2b-e5e9-420d-abb0-4480aab1e378:15534:1798523504966336:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1715207118.745099:0:15534:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745106:0:15534:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745111:0:15534:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745116:0:15534:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745122:0:15534:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745127:0:15534:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa548240 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715207118.745131:0:15534:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745137:0:15534:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745143:0:15534:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745148:0:15534:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715207118.745153:0:15534:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 15534 initiator: MDT0 00010000:00010000:3.0:1715207118.745158:0:15534:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548240/0x7561918bd556ff0e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 15534 initiator: MDT0 00000080:00020000:3.0:1715207118.745179:0:15534:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:3.0:1715207118.745196:0:15534:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715207118.745199:0:15534:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715207118.745207:0:15534:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207118.745214:0:15534:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207118.745222:0:15534:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715207118.745229:0:15534:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715207118.745231:0:15534:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:3.0:1715207118.745234:0:15534:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a923fcc8 00010000:00010000:3.0:1715207118.745237:0:15534:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207118.745243:0:15534:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715207118.745316:0:15534:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715207118.745324:0:15534:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207118.745330:0:15534:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207118.745338:0:15534:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715207118.745345:0:15534:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715207118.745351:0:15534:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:3.0:1715207118.745354:0:15534:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a923fb80 00010000:00010000:3.0:1715207118.745357:0:15534:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207118.745364:0:15534:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715207118.745372:0:15534:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:3.0:1715207118.745378:0:15534:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715207118.746144:0:15535:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715207118.746157:0:15535:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207118.746166:0:15535:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207118.746176:0:15535:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715207118.746184:0:15535:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715207118.746186:0:15535:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:1.0:1715207118.746190:0:15535:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a91e3b80 00010000:00010000:1.0:1715207118.746194:0:15535:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207118.746201:0:15535:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715207118.746210:0:15535:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:1.0:1715207118.746215:0:15535:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715207118.746244:0:15535:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715207118.746251:0:15535:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207118.746257:0:15535:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207118.746264:0:15535:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715207118.746271:0:15535:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715207118.746273:0:15535:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd556fafb 00000080:00010000:1.0:1715207118.746276:0:15535:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd556fafb from it ffff8800a91e3bb0 00010000:00010000:1.0:1715207118.746278:0:15535:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207118.746284:0:15535:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b3c0/0x7561918bd556fafb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed878917bb expref: -99 pid: 20655 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715207118.746291:0:15535:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715207118.746296:0:15535:0:(dcache.c:318:ll_revalidate_dentry()) VFS Op:name=.lustre, flags=17 00000080:00200000:1.0:1715207118.746300:0:15535:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x1:0x0](ffff880136f410c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715207118.746307:0:15535:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), file ffff88012ab3d800, open_flags 8000, mode 0 opened 0 00000080:00200000:1.0:1715207118.746313:0:15535:0:(namei.c:838:ll_lookup_it()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), intent=open 00000002:00010000:1.0:1715207118.746321:0:15535:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: fid,[0x0:0x0:0x0]) in obj [0x200000002:0x1:0x0], intent: open flags 0100001 00010000:00010000:1.0:1715207118.746348:0:15535:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.746354:0:15535:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 15535 initiator: MDT0 00000020:00100000:1.0:1715207118.746363:0:15535:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1715207118.746374:0:15535:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a9679180 x1798523504966464/t0(0) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/66536 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00010000:00010000:1.0:1715207118.746384:0:15535:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 15535 initiator: MDT0 00000100:00100000:1.0:1715207118.746397:0:15535:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9679180 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15535:1798523504966464:192.168.201.123@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1715207118.746530:0:15535:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a9056cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1715207118.748782:0:15535:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9679180 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15535:1798523504966464:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1715207118.748800:0:15535:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748814:0:15535:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748823:0:15535:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748833:0:15535:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748844:0:15535:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748853:0:15535:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa668d80 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715207118.748858:0:15535:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748868:0:15535:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748878:0:15535:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 5/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748897:0:15535:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715207118.748903:0:15535:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 15535 initiator: MDT0 00010000:00010000:1.0:1715207118.748913:0:15535:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa668d80/0x7561918bd556ff15 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 15535 initiator: MDT0 00000002:00100000:1.0:1715207118.748955:0:15535:0:(mdc_locks.c:838:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff8800a9679180 x1798523504966464/t4294967402(4294967402) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207134 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000002:00100000:1.0:1715207118.748968:0:15535:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a9679180 x1798523504966464/t4294967402(4294967402) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207134 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:1.0:1715207118.748993:0:15535:0:(llite_lib.c:2872:ll_update_inode()) inode=[0x200000002:0x2:0x0], updating i_size 4096 00000080:00200000:1.0:1715207118.748997:0:15535:0:(namei.c:157:ll_iget()) got inode: [0x200000002:0x2:0x0](ffff8800b02788c8): rc = 0 00000080:00010000:1.0:1715207118.749001:0:15535:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000002:0x2:0x0](ffff8800b02788c8) for lock 0x0 00000080:00200008:1.0:1715207118.749016:0:15535:0:(file.c:871:ll_file_open()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:1.0:1715207118.749048:0:15535:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a9679180 x1798523504966464/t4294967402(4294967402) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207134 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:1.0:1715207118.749094:0:15535:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x2:0x0](ffff8800b02788c8), inode mode 4040 mask 44 00000080:00200080:1.0:1715207118.749147:0:15535:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=26270f0 00800000:00000080:1.0:1715207118.749156:0:15535:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b621bce0 uarg=0000000000000000 00000002:00000080:1.0:1715207118.749165:0:15535:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b621bce0 uarg=0000000000000000 00000100:00100000:1.0:1715207118.749189:0:15535:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9678380 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15535:1798523504966528:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1715207118.749221:0:15535:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90569c0 going to sleep for 11 seconds 00000100:00100000:2.0:1715207118.750902:0:15535:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678380 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15535:1798523504966528:192.168.201.123@tcp:59:lhsmtool_posix.0 00000002:00000080:2.0:1715207118.750935:0:15535:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b621bce0 uarg=0000000000000000 00000100:00100000:2.0:1715207118.750970:0:15535:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15535:1798523504966592:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715207118.751023:0:15535:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f480 going to sleep for 11 seconds 00000100:00100000:2.0:1715207118.753303:0:15535:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:15535:1798523504966592:192.168.201.123@tcp:59:lhsmtool_posix.0 00000080:00200000:1.0:1715207118.754698:0:15536:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715207118.754713:0:15536:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207118.754745:0:15536:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754760:0:15536:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754770:0:15536:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15536 initiator: MDT0 00000100:00100000:1.0:1715207118.754781:0:15536:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a9056240 going to sleep for 0 seconds 00000100:00100000:1.0:1715207118.754789:0:15536:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678000 pname:cluuid:pid:xid:nid:opc:job mkdir:04532c2b-e5e9-420d-abb0-4480aab1e378:15536:1798523504966656:192.168.201.123@tcp:101:mkdir.0 00010000:00010000:1.0:1715207118.754798:0:15536:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754804:0:15536:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754810:0:15536:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754817:0:15536:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754823:0:15536:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754829:0:15536:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa669200 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715207118.754832:0:15536:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754838:0:15536:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754844:0:15536:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754859:0:15536:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715207118.754864:0:15536:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 15536 initiator: MDT0 00010000:00010000:1.0:1715207118.754870:0:15536:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669200/0x7561918bd556ff1c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 15536 initiator: MDT0 00000080:00020000:1.0:1715207118.754887:0:15536:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715207118.892396:0:15573:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715207118.892424:0:15573:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207118.892479:0:15573:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892490:0:15573:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892500:0:15573:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15573 initiator: MDT0 00000100:00100000:1.0:1715207118.892519:0:15573:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a9056240 going to sleep for 0 seconds 00000100:00100000:1.0:1715207118.892531:0:15573:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678000 pname:cluuid:pid:xid:nid:opc:job mkdir:04532c2b-e5e9-420d-abb0-4480aab1e378:15573:1798523504966720:192.168.201.123@tcp:101:mkdir.0 00010000:00010000:1.0:1715207118.892542:0:15573:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892549:0:15573:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892555:0:15573:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892614:0:15573:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892629:0:15573:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892635:0:15573:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa669d40 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715207118.892640:0:15573:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892646:0:15573:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892653:0:15573:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892660:0:15573:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715207118.892665:0:15573:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 15573 initiator: MDT0 00010000:00010000:1.0:1715207118.892671:0:15573:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669d40/0x7561918bd556ff23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 15573 initiator: MDT0 00000080:00020000:1.0:1715207118.892695:0:15573:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00010000:00010000:2.0:1715207118.954725:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1715207119.370273:0:15708:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12n: @@@@@@ FAIL: cannot create directory '/mnt/lustre/d12n.sanity-hsm' Debug log: 200 lines, 200 kept, 0 dropped, 0 bad.