00010000:00010000:0.0F:1715210094.902809:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1715210095.904800:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:2.0F:1715210096.725386:0:13546:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=159e0f0 00800000:00000080:2.0:1715210096.725412:0:13546:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b6eee280 uarg=0000000000000000 00000002:00000080:2.0:1715210096.725421:0:13546:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b6eee280 uarg=0000000000000000 00000100:00100000:2.0:1715210096.725497:0:13546:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:13546:1798523505215040:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715210096.725605:0:13546:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aa4bcc00 going to sleep for 11 seconds 00000100:00100000:3.0F:1715210096.727319:0:13546:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:13546:1798523505215040:192.168.201.123@tcp:60:lhsmtool_posix.0 00000002:00000080:3.0:1715210096.727347:0:13546:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b6eee280 uarg=0000000000000000 00000100:00100000:3.0:1715210096.727374:0:13546:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a91e5180 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:13546:1798523505215104:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1715210096.727428:0:13546:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a923d480 going to sleep for 11 seconds 00000100:00100000:2.0:1715210096.729063:0:13546:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e5180 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:13546:1798523505215104:192.168.201.123@tcp:60:lhsmtool_posix.0 00000080:00200008:2.0:1715210096.729108:0:13546:0:(file.c:430:ll_file_release()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:2.0:1715210096.729145:0:13546:0:(mdc_request.c:927:mdc_close()) @@@ matched open req@ffff8800a91e5500 x1798523505204352/t4294967520(4294967520) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715209980 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000020:00100000:2.0:1715210096.729170:0:13546:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1715210096.729175:0:13546:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800abe2ed80 x1798523505215232/t0(0) o35->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:23/10 lens 392/840 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00000100:00100000:2.0:1715210096.729192:0:13546:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ed80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:13546:1798523505215232:192.168.201.123@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1715210096.729245:0:13546:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aa4bc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1715210096.731114:0:13546:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ed80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:13546:1798523505215232:192.168.201.123@tcp:35:lhsmtool_posix.0 00000002:00100000:2.0:1715210096.731139:0:13546:0:(mdc_request.c:837:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8800a91e5500 x1798523505204352/t4294967520(4294967520) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715209980 ref 1 fl Complete:RQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200008:2.0:1715210096.731178:0:13546:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200008:3.0:1715210096.734556:0:13546:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00010000:00010000:0.0:1715210096.906792:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1715210097.240724:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715210097.240734:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715210097.240750:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715210097.240755:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715210097.240760:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715210097.240773:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:3.0:1715210097.240819:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2003:1798523505215296:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715210097.240822:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505215360:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715210097.240920:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505215424:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715210097.242784:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505215360:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715210097.242785:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2003:1798523505215296:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715210097.242801:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505215424:192.168.201.123@tcp:400:kworker.0 00010000:00010000:0.0:1715210097.908772:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0F:1715210098.910791:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1715210099.443172:0:20177:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 19:14:58 (1715210098) 00000080:00200008:1.0:1715210099.570763:0:20220:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1715210099.570774:0:20220:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715210099.570796:0:20220:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.570814:0:20220:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715210099.570830:0:20220:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715210099.570842:0:20220:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715210099.570845:0:20220:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd557093a 00000080:00010000:1.0:1715210099.570853:0:20220:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd557093a from it ffff8800a9233cc8 00010000:00010000:1.0:1715210099.570857:0:20220:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715210099.570867:0:20220:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1715210099.571299:0:20220:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1715210099.571305:0:20220:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715210099.571317:0:20220:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.571361:0:20220:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571370:0:20220:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571379:0:20220:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20220 initiator: MDT0 00000100:00100000:1.0:1715210099.571397:0:20220:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c83c0 going to sleep for 0 seconds 00000100:00100000:1.0:1715210099.571414:0:20220:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:04532c2b-e5e9-420d-abb0-4480aab1e378:20220:1798523505215488:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1715210099.571451:0:20220:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571458:0:20220:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571464:0:20220:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571470:0:20220:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571477:0:20220:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571483:0:20220:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa548900 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715210099.571487:0:20220:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571494:0:20220:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571501:0:20220:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571507:0:20220:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715210099.571512:0:20220:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20220 initiator: MDT0 00010000:00010000:1.0:1715210099.571518:0:20220:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548900/0x7561918bd5570ac2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20220 initiator: MDT0 00000080:00020000:1.0:1715210099.571537:0:20220:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:1.0:1715210099.571591:0:20220:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1715210099.571596:0:20220:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715210099.571606:0:20220:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.571615:0:20220:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715210099.571623:0:20220:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715210099.571631:0:20220:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715210099.571634:0:20220:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd557093a 00000080:00010000:1.0:1715210099.571638:0:20220:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd557093a from it ffff8800a9233cc8 00010000:00010000:1.0:1715210099.571641:0:20220:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715210099.571648:0:20220:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715210099.571738:0:20220:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715210099.571747:0:20220:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.571754:0:20220:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715210099.571763:0:20220:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715210099.571770:0:20220:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715210099.571773:0:20220:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd557093a 00000080:00010000:1.0:1715210099.571777:0:20220:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd557093a from it ffff8800a9233b80 00010000:00010000:1.0:1715210099.571784:0:20220:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715210099.571791:0:20220:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715210099.571801:0:20220:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:1.0:1715210099.571808:0:20220:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:3.0:1715210099.572599:0:20221:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715210099.572616:0:20221:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715210099.572627:0:20221:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715210099.572639:0:20221:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715210099.572649:0:20221:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715210099.572652:0:20221:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd557093a 00000080:00010000:3.0:1715210099.572658:0:20221:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd557093a from it ffff88012d4efb80 00010000:00010000:3.0:1715210099.572662:0:20221:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715210099.572681:0:20221:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715210099.572692:0:20221:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:3.0:1715210099.572698:0:20221:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:3.0:1715210099.572726:0:20221:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715210099.572733:0:20221:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715210099.572739:0:20221:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715210099.572747:0:20221:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715210099.572755:0:20221:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715210099.572757:0:20221:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd557093a 00000080:00010000:3.0:1715210099.572761:0:20221:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd557093a from it ffff88012d4efbb0 00010000:00010000:3.0:1715210099.572764:0:20221:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715210099.572770:0:20221:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54b180/0x7561918bd557093a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87892d78 expref: -99 pid: 7849 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715210099.572777:0:20221:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 1 00000080:00200000:3.0:1715210099.572783:0:20221:0:(dcache.c:318:ll_revalidate_dentry()) VFS Op:name=.lustre, flags=17 00000080:00200000:3.0:1715210099.572787:0:20221:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x1:0x0](ffff880136f410c8), inode mode 41ed mask 1 00000080:00200000:3.0:1715210099.572794:0:20221:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), file ffff88012a4f3200, open_flags 8000, mode 0 opened 0 00000080:00200000:3.0:1715210099.572801:0:20221:0:(namei.c:838:ll_lookup_it()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), intent=open 00000002:00010000:3.0:1715210099.572817:0:20221:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: fid,[0x0:0x0:0x0]) in obj [0x200000002:0x1:0x0], intent: open flags 0100001 00010000:00010000:3.0:1715210099.572851:0:20221:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 20221 initiator: MDT0 00010000:00010000:3.0:1715210099.572858:0:20221:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 20221 initiator: MDT0 00000020:00100000:3.0:1715210099.572868:0:20221:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1715210099.572878:0:20221:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a91e6a00 x1798523505215616/t0(0) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/66536 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00010000:00010000:3.0:1715210099.572888:0:20221:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 20221 initiator: MDT0 00000100:00100000:3.0:1715210099.572903:0:20221:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a91e6a00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:20221:1798523505215616:192.168.201.123@tcp:101:lhsmtool_posix.0 00000100:00100000:3.0:1715210099.573036:0:20221:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a923d780 going to sleep for 11 seconds 00000100:00100000:2.0:1715210099.575454:0:20221:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e6a00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:20221:1798523505215616:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1715210099.575469:0:20221:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575480:0:20221:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575487:0:20221:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575493:0:20221:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575500:0:20221:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575505:0:20221:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a96086c0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:2.0:1715210099.575509:0:20221:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575514:0:20221:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575520:0:20221:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 5/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575526:0:20221:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1715210099.575537:0:20221:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 20221 initiator: MDT0 00010000:00010000:2.0:1715210099.575543:0:20221:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800a96086c0/0x7561918bd5570ac9 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 20221 initiator: MDT0 00000002:00100000:2.0:1715210099.575558:0:20221:0:(mdc_locks.c:838:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff8800a91e6a00 x1798523505215616/t4294967522(4294967522) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715210115 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000002:00100000:2.0:1715210099.575598:0:20221:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a91e6a00 x1798523505215616/t4294967522(4294967522) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715210115 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715210099.575622:0:20221:0:(llite_lib.c:2872:ll_update_inode()) inode=[0x200000002:0x2:0x0], updating i_size 4096 00000080:00200000:2.0:1715210099.575625:0:20221:0:(namei.c:157:ll_iget()) got inode: [0x200000002:0x2:0x0](ffff8800b02788c8): rc = 0 00000080:00010000:2.0:1715210099.575630:0:20221:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000002:0x2:0x0](ffff8800b02788c8) for lock 0x0 00000080:00200008:2.0:1715210099.575644:0:20221:0:(file.c:871:ll_file_open()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:2.0:1715210099.575657:0:20221:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a91e6a00 x1798523505215616/t4294967522(4294967522) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715210115 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715210099.575717:0:20221:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x2:0x0](ffff8800b02788c8), inode mode 4040 mask 44 00000080:00200080:2.0:1715210099.575770:0:20221:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=b8b0f0 00800000:00000080:2.0:1715210099.575779:0:20221:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800abd4b6c0 uarg=0000000000000000 00000002:00000080:2.0:1715210099.575789:0:20221:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800abd4b6c0 uarg=0000000000000000 00000100:00100000:2.0:1715210099.575814:0:20221:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f100 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:20221:1798523505215680:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715210099.575866:0:20221:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aa4bc180 going to sleep for 11 seconds 00000100:00100000:3.0:1715210099.577676:0:20221:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f100 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:20221:1798523505215680:192.168.201.123@tcp:59:lhsmtool_posix.0 00000002:00000080:3.0:1715210099.577695:0:20221:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800abd4b6c0 uarg=0000000000000000 00000100:00100000:3.0:1715210099.577721:0:20221:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a91e7800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:20221:1798523505215744:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1715210099.577775:0:20221:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a923d840 going to sleep for 11 seconds 00000100:00100000:3.0:1715210099.579506:0:20221:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e7800 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:20221:1798523505215744:192.168.201.123@tcp:59:lhsmtool_posix.0 00000080:00200000:0.0:1715210099.582522:0:20222:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:0.0:1715210099.582538:0:20222:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715210099.582595:0:20222:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582605:0:20222:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582613:0:20222:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00000100:00100000:0.0:1715210099.582624:0:20222:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012e1e5600 going to sleep for 0 seconds 00000100:00100000:0.0:1715210099.582632:0:20222:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a7800 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:20222:1798523505215808:192.168.201.123@tcp:101:lfs.0 00010000:00010000:0.0:1715210099.582641:0:20222:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582647:0:20222:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582653:0:20222:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582659:0:20222:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582665:0:20222:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582671:0:20222:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa585b00 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:0.0:1715210099.582674:0:20222:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582686:0:20222:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582692:0:20222:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582698:0:20222:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715210099.582702:0:20222:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582708:0:20222:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570ad0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20222 initiator: MDT0 00000080:00020000:0.0:1715210099.582727:0:20222:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:0.0:1715210099.582753:0:20222:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:0.0:1715210099.582761:0:20222:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715210099.582781:0:20222:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582787:0:20222:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582794:0:20222:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00000100:00100000:0.0:1715210099.582802:0:20222:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012e1e5600 going to sleep for 0 seconds 00000100:00100000:0.0:1715210099.582808:0:20222:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a7800 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:20222:1798523505215872:192.168.201.123@tcp:101:lfs.0 00010000:00010000:0.0:1715210099.582815:0:20222:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582821:0:20222:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582837:0:20222:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582843:0:20222:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582848:0:20222:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582853:0:20222:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa584240 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:0.0:1715210099.582856:0:20222:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582862:0:20222:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582867:0:20222:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582872:0:20222:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715210099.582875:0:20222:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582881:0:20222:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa584240/0x7561918bd5570ad7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20222 initiator: MDT0 00000080:00020000:0.0:1715210099.582896:0:20222:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:0.0:1715210099.582912:0:20222:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1715210099.582916:0:20222:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:0.0:1715210099.582923:0:20222:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715210099.582939:0:20222:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582951:0:20222:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582958:0:20222:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00000100:00100000:0.0:1715210099.582967:0:20222:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012e1e5600 going to sleep for 0 seconds 00000100:00100000:0.0:1715210099.582972:0:20222:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a7800 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:20222:1798523505215936:192.168.201.123@tcp:101:lfs.0 00010000:00010000:0.0:1715210099.582978:0:20222:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582984:0:20222:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582990:0:20222:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.582996:0:20222:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.583001:0:20222:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.583006:0:20222:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa585680 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:0.0:1715210099.583009:0:20222:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.583014:0:20222:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.583019:0:20222:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.583032:0:20222:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715210099.583036:0:20222:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20222 initiator: MDT0 00010000:00010000:0.0:1715210099.583041:0:20222:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585680/0x7561918bd5570ade lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20222 initiator: MDT0 00000080:00020000:0.0:1715210099.583055:0:20222:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715210099.595918:0:20226:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715210099.595933:0:20226:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.595961:0:20226:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.595970:0:20226:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.595978:0:20226:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20226 initiator: MDT0 00000100:00100000:1.0:1715210099.595990:0:20226:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c83c0 going to sleep for 0 seconds 00000100:00100000:1.0:1715210099.595998:0:20226:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job mkdir:04532c2b-e5e9-420d-abb0-4480aab1e378:20226:1798523505216000:192.168.201.123@tcp:101:mkdir.0 00010000:00010000:1.0:1715210099.596007:0:20226:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596014:0:20226:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596020:0:20226:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596026:0:20226:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596047:0:20226:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596053:0:20226:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa549200 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715210099.596057:0:20226:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596063:0:20226:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596068:0:20226:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596074:0:20226:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715210099.596078:0:20226:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20226 initiator: MDT0 00010000:00010000:1.0:1715210099.596083:0:20226:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549200/0x7561918bd5570ae5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20226 initiator: MDT0 00000080:00020000:1.0:1715210099.596101:0:20226:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:0.0:1715210099.599742:0:20227:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:0.0:1715210099.599758:0:20227:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715210099.599787:0:20227:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599797:0:20227:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599805:0:20227:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20227 initiator: MDT0 00000100:00100000:0.0:1715210099.599816:0:20227:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012e1e5600 going to sleep for 0 seconds 00000100:00100000:0.0:1715210099.599825:0:20227:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a7800 pname:cluuid:pid:xid:nid:opc:job rm:04532c2b-e5e9-420d-abb0-4480aab1e378:20227:1798523505216064:192.168.201.123@tcp:101:rm.0 00010000:00010000:0.0:1715210099.599833:0:20227:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599840:0:20227:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599845:0:20227:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599852:0:20227:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599864:0:20227:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599869:0:20227:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa585b00 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:0.0:1715210099.599872:0:20227:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599878:0:20227:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599884:0:20227:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599889:0:20227:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715210099.599893:0:20227:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599899:0:20227:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa585b00/0x7561918bd5570aec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20227 initiator: MDT0 00000080:00020000:0.0:1715210099.599916:0:20227:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:0.0:1715210099.599961:0:20227:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:0.0:1715210099.599970:0:20227:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715210099.599989:0:20227:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.599995:0:20227:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600002:0:20227:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20227 initiator: MDT0 00000100:00100000:0.0:1715210099.600010:0:20227:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012e1e5600 going to sleep for 0 seconds 00000100:00100000:0.0:1715210099.600022:0:20227:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a7800 pname:cluuid:pid:xid:nid:opc:job rm:04532c2b-e5e9-420d-abb0-4480aab1e378:20227:1798523505216128:192.168.201.123@tcp:101:rm.0 00010000:00010000:0.0:1715210099.600030:0:20227:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600035:0:20227:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600041:0:20227:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600046:0:20227:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600052:0:20227:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600057:0:20227:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa5861c0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:0.0:1715210099.600060:0:20227:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600065:0:20227:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600071:0:20227:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600076:0:20227:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715210099.600079:0:20227:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20227 initiator: MDT0 00010000:00010000:0.0:1715210099.600084:0:20227:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5861c0/0x7561918bd5570af3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20227 initiator: MDT0 00000080:00020000:0.0:1715210099.600099:0:20227:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715210099.603754:0:20228:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715210099.603770:0:20228:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.603798:0:20228:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603807:0:20228:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603816:0:20228:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20228 initiator: MDT0 00000100:00100000:1.0:1715210099.603827:0:20228:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c83c0 going to sleep for 0 seconds 00000100:00100000:1.0:1715210099.603841:0:20228:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job dd:04532c2b-e5e9-420d-abb0-4480aab1e378:20228:1798523505216192:192.168.201.123@tcp:101:dd.0 00010000:00010000:1.0:1715210099.603852:0:20228:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603858:0:20228:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603864:0:20228:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603870:0:20228:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603876:0:20228:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603881:0:20228:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54af40 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715210099.603885:0:20228:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603890:0:20228:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603896:0:20228:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603901:0:20228:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715210099.603905:0:20228:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20228 initiator: MDT0 00010000:00010000:1.0:1715210099.603911:0:20228:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570afa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20228 initiator: MDT0 00000080:00020000:1.0:1715210099.603928:0:20228:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715210099.610392:0:20229:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715210099.610407:0:20229:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.610460:0:20229:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610470:0:20229:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610478:0:20229:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20229 initiator: MDT0 00000100:00100000:1.0:1715210099.610489:0:20229:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c83c0 going to sleep for 0 seconds 00000100:00100000:1.0:1715210099.610497:0:20229:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:20229:1798523505216256:192.168.201.123@tcp:101:lfs.0 00010000:00010000:1.0:1715210099.610507:0:20229:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610513:0:20229:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610519:0:20229:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610525:0:20229:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610531:0:20229:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610536:0:20229:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa5486c0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715210099.610540:0:20229:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610545:0:20229:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610558:0:20229:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610592:0:20229:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715210099.610596:0:20229:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20229 initiator: MDT0 00010000:00010000:1.0:1715210099.610602:0:20229:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b01 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20229 initiator: MDT0 00000080:00020000:1.0:1715210099.610620:0:20229:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00010000:00010000:0.0:1715210099.912799:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1715210099.943249:0:20239:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715210099.943273:0:20239:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.943321:0:20239:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943333:0:20239:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943343:0:20239:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20239 initiator: MDT0 00000100:00100000:1.0:1715210099.943358:0:20239:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c8240 going to sleep for 0 seconds 00000100:00100000:1.0:1715210099.943370:0:20239:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:20239:1798523505216320:192.168.201.123@tcp:101:lfs.0 00010000:00010000:1.0:1715210099.943382:0:20239:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943389:0:20239:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943395:0:20239:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943408:0:20239:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943440:0:20239:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943446:0:20239:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa5486c0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715210099.943451:0:20239:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943457:0:20239:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943465:0:20239:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943470:0:20239:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715210099.943476:0:20239:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20239 initiator: MDT0 00010000:00010000:1.0:1715210099.943481:0:20239:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa5486c0/0x7561918bd5570b08 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20239 initiator: MDT0 00000080:00020000:1.0:1715210099.943504:0:20239:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715210099.949280:0:20240:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715210099.949295:0:20240:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715210099.949323:0:20240:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949333:0:20240:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949341:0:20240:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20240 initiator: MDT0 00000100:00100000:1.0:1715210099.949358:0:20240:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c8240 going to sleep for 0 seconds 00000100:00100000:1.0:1715210099.949366:0:20240:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job stat:04532c2b-e5e9-420d-abb0-4480aab1e378:20240:1798523505216384:192.168.201.123@tcp:101:stat.0 00010000:00010000:1.0:1715210099.949377:0:20240:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949386:0:20240:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949392:0:20240:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949398:0:20240:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949404:0:20240:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949410:0:20240:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54af40 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715210099.949439:0:20240:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949445:0:20240:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949451:0:20240:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949457:0:20240:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715210099.949461:0:20240:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20240 initiator: MDT0 00010000:00010000:1.0:1715210099.949467:0:20240:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54af40/0x7561918bd5570b0f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20240 initiator: MDT0 00000080:00020000:1.0:1715210099.949491:0:20240:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715210099.958268:0:20243:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715210099.958284:0:20243:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715210099.958313:0:20243:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958323:0:20243:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958342:0:20243:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20243 initiator: MDT0 00000100:00100000:3.0:1715210099.958353:0:20243:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a923d840 going to sleep for 0 seconds 00000100:00100000:3.0:1715210099.958361:0:20243:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e7800 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:20243:1798523505216448:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715210099.958370:0:20243:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958376:0:20243:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958382:0:20243:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958388:0:20243:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958394:0:20243:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958399:0:20243:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a9608000 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715210099.958403:0:20243:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958408:0:20243:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958436:0:20243:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958442:0:20243:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715210099.958446:0:20243:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20243 initiator: MDT0 00010000:00010000:3.0:1715210099.958457:0:20243:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9608000/0x7561918bd5570b16 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 20243 initiator: MDT0 00000080:00020000:3.0:1715210099.958476:0:20243:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000001:02000400:1.0:1715210100.453889:0:20378:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_31c: @@@@@@ FAIL: /home/green/git/lustre-release/lustre/utils/lfs hsm_state /mnt/lustre/d31c.sanity-hsm/f31c.sanity-hsm failed (run as root) 00010000:00010000:0.0:1715210100.914785:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 378 lines, 378 kept, 0 dropped, 0 bad.