00000080:00200080:0.0F:1715206680.328124:0:5737:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8) cmd=401866d5 arg=b930f0 00800000:00000080:0.0:1715206680.328145:0:5737:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800aa1c9800: cmd=401866d5 len=24 karg=ffff8800a9410bc0 uarg=0000000000000000 00000002:00000080:0.0:1715206680.328153:0:5737:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800aa1c9800: cmd=401866d5 len=24 karg=ffff8800a9410bc0 uarg=0000000000000000 00000100:00100000:0.0:1715206680.328194:0:5737:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012aaf5500 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:5737:1798523503856896:192.168.202.138@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1715206680.328244:0:5737:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aaeee540 going to sleep for 145 seconds 00000100:00100000:1.0F:1715206680.329442:0:5737:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012aaf5500 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:5737:1798523503856896:192.168.202.138@tcp:60:lhsmtool_posix.0 00000080:00200008:1.0:1715206680.329474:0:5737:0:(file.c:430:ll_file_release()) START file fid:[0x200000002:0x2:0x0](ffff8800b08900c8), flags 100000 00000002:00100000:1.0:1715206680.329495:0:5737:0:(mdc_request.c:927:mdc_close()) @@@ matched open req@ffff88012a617480 x1798523503856512/t4294967411(4294967411) o101->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206829 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000020:00100000:1.0:1715206680.329509:0:5737:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800aa1c9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1715206680.329512:0:5737:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88012a615c00 x1798523503857024/t0(0) o35->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:23/10 lens 392/840 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00000100:00100000:1.0:1715206680.329520:0:5737:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012a615c00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:5737:1798523503857024:192.168.202.138@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1715206680.329548:0:5737:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b5d15840 going to sleep for 11 seconds 00000100:00100000:2.0F:1715206680.330561:0:5737:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a615c00 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:5737:1798523503857024:192.168.202.138@tcp:35:lhsmtool_posix.0 00000002:00100000:2.0:1715206680.330577:0:5737:0:(mdc_request.c:837:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88012a617480 x1798523503856512/t4294967411(4294967411) o101->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206829 ref 1 fl Complete:RQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200008:2.0:1715206680.330597:0:5737:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff880136f588c8), flags 100000 00000080:00200008:2.0:1715206680.332334:0:5737:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff880136f588c8), flags 100000 00010000:00010000:1.0:1715206680.526187:0:3698:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1715206681.528200:0:3699:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1715206682.530216:0:3697:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0F:1715206682.812152:0:65:0:(pinger.c:253:ptlrpc_pinger_process_import()) 46af332d-2ee7-4d47-b762-82c1eb2b6bbd->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1715206682.812163:0:65:0:(pinger.c:271:ptlrpc_pinger_process_import()) 46af332d-2ee7-4d47-b762-82c1eb2b6bbd->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:3.0:1715206682.812166:0:65:0:(pinger.c:253:ptlrpc_pinger_process_import()) 46af332d-2ee7-4d47-b762-82c1eb2b6bbd->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1715206682.812169:0:65:0:(pinger.c:271:ptlrpc_pinger_process_import()) 46af332d-2ee7-4d47-b762-82c1eb2b6bbd->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:3.0:1715206682.812183:0:65:0:(pinger.c:253:ptlrpc_pinger_process_import()) cbae74da-7571-45ad-b971-0e0378bbd4e4->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1715206682.812186:0:65:0:(pinger.c:271:ptlrpc_pinger_process_import()) cbae74da-7571-45ad-b971-0e0378bbd4e4->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:3.0:1715206682.812214:0:1956:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800aafdb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cbae74da-7571-45ad-b971-0e0378bbd4e4:1956:1798523503857152:192.168.202.138@tcp:400:kworker.0 00000100:00100000:2.0:1715206682.812216:0:1957:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800aafd9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:ff645b43-0d7b-4a37-b9cc-6ac8f8d65c0d:1957:1798523503857088:192.168.202.138@tcp:400:kworker.0 00000100:00100000:2.0:1715206682.812274:0:1957:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800aafd8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cbae74da-7571-45ad-b971-0e0378bbd4e4:1957:1798523503857216:192.168.202.138@tcp:400:kworker.0 00000100:00100000:2.0:1715206682.813188:0:1957:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800aafd9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:ff645b43-0d7b-4a37-b9cc-6ac8f8d65c0d:1957:1798523503857088:192.168.202.138@tcp:400:kworker.0 00000100:00100000:2.0:1715206682.813200:0:1957:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800aafd8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cbae74da-7571-45ad-b971-0e0378bbd4e4:1957:1798523503857216:192.168.202.138@tcp:400:kworker.0 00000100:00100000:3.0:1715206682.813237:0:1956:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800aafdb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cbae74da-7571-45ad-b971-0e0378bbd4e4:1956:1798523503857152:192.168.202.138@tcp:400:kworker.0 00010000:00010000:0.0:1715206683.532150:0:4776:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1715206683.830257:0:6632:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12c: Restore a file with stripe of 2 == 18:18:03 (1715206683) 00000080:00200008:1.0:1715206683.888591:0:6675:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f588c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1715206683.888599:0:6675:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8),name=/ 00000002:00010000:1.0:1715206683.888615:0:6675:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.888629:0:6675:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206683.888642:0:6675:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715206683.888652:0:6675:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) 00000080:00010000:1.0:1715206683.888655:0:6675:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) for lock 0xbbeffb005ec3dd3a 00000080:00010000:1.0:1715206683.888662:0:6675:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0xbbeffb005ec3dd3a from it ffff8800aa2a3cc8 00010000:00010000:1.0:1715206683.888665:0:6675:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206683.888673:0:6675:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1715206683.888983:0:6675:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1715206683.888987:0:6675:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:1.0:1715206683.888997:0:6675:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.889035:0:6675:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889043:0:6675:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889051:0:6675:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6675 initiator: MDT0 00000100:00100000:1.0:1715206683.889094:0:6675:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b5d159c0 going to sleep for 0 seconds 00000100:00100000:1.0:1715206683.889102:0:6675:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a616d80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6675:1798523503857280:192.168.202.138@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1715206683.889109:0:6675:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889113:0:6675:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889117:0:6675:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889120:0:6675:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889124:0:6675:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889127:0:6675:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880136baeac0 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:1.0:1715206683.889130:0:6675:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889137:0:6675:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889142:0:6675:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889145:0:6675:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206683.889149:0:6675:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6675 initiator: MDT0 00010000:00010000:1.0:1715206683.889152:0:6675:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3dfd3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6675 initiator: MDT0 00000080:00020000:1.0:1715206683.889165:0:6675:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:1.0:1715206683.889177:0:6675:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f588c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1715206683.889179:0:6675:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8),name=/ 00000002:00010000:1.0:1715206683.889184:0:6675:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.889188:0:6675:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206683.889193:0:6675:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715206683.889197:0:6675:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) 00000080:00010000:1.0:1715206683.889199:0:6675:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) for lock 0xbbeffb005ec3dd3a 00000080:00010000:1.0:1715206683.889201:0:6675:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0xbbeffb005ec3dd3a from it ffff8800aa2a3cc8 00010000:00010000:1.0:1715206683.889202:0:6675:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206683.889206:0:6675:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715206683.889258:0:6675:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8),name=/ 00000002:00010000:1.0:1715206683.889263:0:6675:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.889267:0:6675:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206683.889272:0:6675:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715206683.889276:0:6675:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) 00000080:00010000:1.0:1715206683.889277:0:6675:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) for lock 0xbbeffb005ec3dd3a 00000080:00010000:1.0:1715206683.889279:0:6675:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0xbbeffb005ec3dd3a from it ffff8800aa2a3b80 00010000:00010000:1.0:1715206683.889281:0:6675:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715206683.889285:0:6675:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715206683.889290:0:6675:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8), inode mode 41ed mask 44 00000080:00200008:1.0:1715206683.889294:0:6675:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff880136f588c8), flags 100000 00000080:00200000:3.0:1715206683.889717:0:6676:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8),name=/ 00000002:00010000:3.0:1715206683.889725:0:6676:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206683.889731:0:6676:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206683.889738:0:6676:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715206683.889743:0:6676:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) 00000080:00010000:3.0:1715206683.889745:0:6676:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) for lock 0xbbeffb005ec3dd3a 00000080:00010000:3.0:1715206683.889748:0:6676:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0xbbeffb005ec3dd3a from it ffff8800a9487b80 00010000:00010000:3.0:1715206683.889750:0:6676:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206683.889754:0:6676:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715206683.889760:0:6676:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8), inode mode 41ed mask 44 00000080:00200008:3.0:1715206683.889766:0:6676:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff880136f588c8), flags 100000 00000080:00200000:3.0:1715206683.889780:0:6676:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8),name=/ 00000002:00010000:3.0:1715206683.889784:0:6676:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206683.889787:0:6676:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206683.889792:0:6676:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715206683.889795:0:6676:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) 00000080:00010000:3.0:1715206683.889797:0:6676:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff880136f588c8) for lock 0xbbeffb005ec3dd3a 00000080:00010000:3.0:1715206683.889798:0:6676:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0xbbeffb005ec3dd3a from it ffff8800a9487bb0 00010000:00010000:3.0:1715206683.889800:0:6676:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715206683.889804:0:6676:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800ac2bdf80/0xbbeffb005ec3dd3a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x8c55e4ad37f660d9 expref: -99 pid: 20489 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715206683.889808:0:6676:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8), inode mode 41ed mask 1 00000080:00200000:3.0:1715206683.889811:0:6676:0:(dcache.c:318:ll_revalidate_dentry()) VFS Op:name=.lustre, flags=17 00000080:00200000:3.0:1715206683.889813:0:6676:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x1:0x0](ffff88012e4b90c8), inode mode 41ed mask 1 00000080:00200000:3.0:1715206683.889818:0:6676:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff88012e4b90c8), file ffff8800ac1ec900, open_flags 8000, mode 0 opened 0 00000080:00200000:3.0:1715206683.889822:0:6676:0:(namei.c:838:ll_lookup_it()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff88012e4b90c8), intent=open 00000002:00010000:3.0:1715206683.889827:0:6676:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: fid,[0x0:0x0:0x0]) in obj [0x200000002:0x1:0x0], intent: open flags 0100001 00010000:00010000:3.0:1715206683.889848:0:6676:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6676 initiator: MDT0 00010000:00010000:3.0:1715206683.889852:0:6676:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6676 initiator: MDT0 00000020:00100000:3.0:1715206683.889863:0:6676:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800aa1c9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1715206683.889869:0:6676:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800aafdb480 x1798523503857408/t0(0) o101->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:12/10 lens 864/66536 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00010000:00010000:3.0:1715206683.889875:0:6676:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6676 initiator: MDT0 00000100:00100000:3.0:1715206683.889884:0:6676:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800aafdb480 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:6676:1798523503857408:192.168.202.138@tcp:101:lhsmtool_posix.0 00000100:00100000:3.0:1715206683.889978:0:6676:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b051e900 going to sleep for 145 seconds 00000100:00100000:2.0:1715206683.891651:0:6676:0:(client.c:2956:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8800aa1c9800: committing for last_committed 4294967412 gen 1 00000100:00100000:2.0:1715206683.891657:0:6676:0:(client.c:3000:ptlrpc_free_committed()) @@@ stopping search req@ffff8800aafdb480 x1798523503857408/t4294967413(4294967413) o101->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206833 ref 3 fl Rpc:RPQU/204/0 rc 0/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000100:00100000:2.0:1715206683.891665:0:6676:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800aafdb480 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:6676:1798523503857408:192.168.202.138@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1715206683.891671:0:6676:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891676:0:6676:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891679:0:6676:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891683:0:6676:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891686:0:6676:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891692:0:6676:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aae76640 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:2.0:1715206683.891694:0:6676:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891697:0:6676:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891700:0:6676:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 5/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891702:0:6676:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1715206683.891705:0:6676:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 6676 initiator: MDT0 00010000:00010000:2.0:1715206683.891707:0:6676:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800aa1c9800 lock: ffff8800aae76640/0xbbeffb005ec3dfda lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 6676 initiator: MDT0 00000002:00100000:2.0:1715206683.891715:0:6676:0:(mdc_locks.c:838:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff8800aafdb480 x1798523503857408/t4294967413(4294967413) o101->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206833 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000002:00100000:2.0:1715206683.891721:0:6676:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800aafdb480 x1798523503857408/t4294967413(4294967413) o101->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206833 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715206683.891735:0:6676:0:(llite_lib.c:2872:ll_update_inode()) inode=[0x200000002:0x2:0x0], updating i_size 10752 00000080:00200000:2.0:1715206683.891737:0:6676:0:(namei.c:157:ll_iget()) got inode: [0x200000002:0x2:0x0](ffff8800b08900c8): rc = 0 00000080:00010000:2.0:1715206683.891740:0:6676:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000002:0x2:0x0](ffff8800b08900c8) for lock 0x0 00000080:00200008:2.0:1715206683.891748:0:6676:0:(file.c:871:ll_file_open()) START file fid:[0x200000002:0x2:0x0](ffff8800b08900c8), flags 100000 00000002:00100000:2.0:1715206683.891755:0:6676:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800aafdb480 x1798523503857408/t4294967413(4294967413) o101->lustre-MDT0000-mdc-ffff8800aa1c9800@192.168.202.138@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206833 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715206683.891794:0:6676:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x2:0x0](ffff8800b08900c8), inode mode 4040 mask 44 00000080:00200080:2.0:1715206683.891849:0:6676:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f588c8) cmd=401866d5 arg=17940f0 00800000:00000080:2.0:1715206683.891864:0:6676:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800aa1c9800: cmd=401866d5 len=24 karg=ffff8800ac3f03a0 uarg=0000000000000000 00000002:00000080:2.0:1715206683.891872:0:6676:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800aa1c9800: cmd=401866d5 len=24 karg=ffff8800ac3f03a0 uarg=0000000000000000 00000100:00100000:2.0:1715206683.891890:0:6676:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800b6324a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:6676:1798523503857472:192.168.202.138@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715206683.891921:0:6676:0:(client.c:2542:ptlrpc_set_wait()) set ffff8801373919c0 going to sleep for 145 seconds 00000100:00100000:2.0:1715206683.892522:0:6676:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800b6324a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:cbae74da-7571-45ad-b971-0e0378bbd4e4:6676:1798523503857472:192.168.202.138@tcp:59:lhsmtool_posix.0 00000080:00200000:0.0:1715206683.894956:0:6677:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:0.0:1715206683.894966:0:6677:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715206683.894984:0:6677:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.894990:0:6677:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.894995:0:6677:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00000100:00100000:0.0:1715206683.895000:0:6677:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aaeee6c0 going to sleep for 0 seconds 00000100:00100000:0.0:1715206683.895004:0:6677:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012aaf6a00 pname:cluuid:pid:xid:nid:opc:job lfs:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6677:1798523503857536:192.168.202.138@tcp:101:lfs.0 00010000:00010000:0.0:1715206683.895009:0:6677:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895012:0:6677:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895016:0:6677:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895019:0:6677:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895028:0:6677:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895031:0:6677:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800ac2bf3c0 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:0.0:1715206683.895033:0:6677:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895036:0:6677:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895039:0:6677:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895056:0:6677:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715206683.895058:0:6677:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895061:0:6677:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf3c0/0xbbeffb005ec3dfe1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6677 initiator: MDT0 00000080:00020000:0.0:1715206683.895071:0:6677:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:0.0:1715206683.895084:0:6677:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:0.0:1715206683.895088:0:6677:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715206683.895098:0:6677:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895101:0:6677:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895105:0:6677:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00000100:00100000:0.0:1715206683.895109:0:6677:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aaeee6c0 going to sleep for 0 seconds 00000100:00100000:0.0:1715206683.895116:0:6677:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012aaf6a00 pname:cluuid:pid:xid:nid:opc:job lfs:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6677:1798523503857600:192.168.202.138@tcp:101:lfs.0 00010000:00010000:0.0:1715206683.895120:0:6677:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895123:0:6677:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895126:0:6677:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895129:0:6677:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895132:0:6677:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895134:0:6677:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800ac2bf180 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:0.0:1715206683.895136:0:6677:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895138:0:6677:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895141:0:6677:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895143:0:6677:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715206683.895145:0:6677:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895148:0:6677:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bf180/0xbbeffb005ec3dfe8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6677 initiator: MDT0 00000080:00020000:0.0:1715206683.895155:0:6677:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:0.0:1715206683.895166:0:6677:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1715206683.895168:0:6677:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:0.0:1715206683.895172:0:6677:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715206683.895180:0:6677:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895184:0:6677:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895187:0:6677:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00000100:00100000:0.0:1715206683.895191:0:6677:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800aaeee6c0 going to sleep for 0 seconds 00000100:00100000:0.0:1715206683.895194:0:6677:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012aaf6a00 pname:cluuid:pid:xid:nid:opc:job lfs:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6677:1798523503857664:192.168.202.138@tcp:101:lfs.0 00010000:00010000:0.0:1715206683.895197:0:6677:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895200:0:6677:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895203:0:6677:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895206:0:6677:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895209:0:6677:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895211:0:6677:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800ac2bef40 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:0.0:1715206683.895213:0:6677:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895220:0:6677:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895223:0:6677:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895226:0:6677:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715206683.895227:0:6677:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6677 initiator: MDT0 00010000:00010000:0.0:1715206683.895230:0:6677:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800ac2bef40/0xbbeffb005ec3dfef lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6677 initiator: MDT0 00000080:00020000:0.0:1715206683.895237:0:6677:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715206683.897682:0:6678:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:3.0:1715206683.897690:0:6678:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206683.897706:0:6678:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897711:0:6678:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897715:0:6678:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6678 initiator: MDT0 00000100:00100000:3.0:1715206683.897721:0:6678:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b051e900 going to sleep for 0 seconds 00000100:00100000:3.0:1715206683.897725:0:6678:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800aafdbb80 pname:cluuid:pid:xid:nid:opc:job lfs:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6678:1798523503857728:192.168.202.138@tcp:101:lfs.0 00010000:00010000:3.0:1715206683.897730:0:6678:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897733:0:6678:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897736:0:6678:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897745:0:6678:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897748:0:6678:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897751:0:6678:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aae76ac0 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:3.0:1715206683.897753:0:6678:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897756:0:6678:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897759:0:6678:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897761:0:6678:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206683.897763:0:6678:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6678 initiator: MDT0 00010000:00010000:3.0:1715206683.897766:0:6678:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76ac0/0xbbeffb005ec3dff6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6678 initiator: MDT0 00000080:00020000:3.0:1715206683.897775:0:6678:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715206683.903847:0:6682:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:3.0:1715206683.903855:0:6682:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206683.903873:0:6682:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903879:0:6682:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903883:0:6682:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6682 initiator: MDT0 00000100:00100000:3.0:1715206683.903893:0:6682:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b051e900 going to sleep for 0 seconds 00000100:00100000:3.0:1715206683.903898:0:6682:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800aafdbb80 pname:cluuid:pid:xid:nid:opc:job mkdir:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6682:1798523503857792:192.168.202.138@tcp:101:mkdir.0 00010000:00010000:3.0:1715206683.903903:0:6682:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903907:0:6682:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903910:0:6682:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903913:0:6682:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903916:0:6682:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903919:0:6682:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aae76880 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:3.0:1715206683.903921:0:6682:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903924:0:6682:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903926:0:6682:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903929:0:6682:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206683.903931:0:6682:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6682 initiator: MDT0 00010000:00010000:3.0:1715206683.903934:0:6682:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3dffd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6682 initiator: MDT0 00000080:00020000:3.0:1715206683.903945:0:6682:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206683.905871:0:6683:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:1.0:1715206683.905879:0:6683:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.905895:0:6683:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905900:0:6683:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905904:0:6683:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6683 initiator: MDT0 00000100:00100000:1.0:1715206683.905915:0:6683:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b5d15a80 going to sleep for 0 seconds 00000100:00100000:1.0:1715206683.905919:0:6683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a616d80 pname:cluuid:pid:xid:nid:opc:job rm:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6683:1798523503857856:192.168.202.138@tcp:101:rm.0 00010000:00010000:1.0:1715206683.905924:0:6683:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905928:0:6683:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905931:0:6683:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905934:0:6683:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905937:0:6683:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905940:0:6683:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880136bad440 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:1.0:1715206683.905941:0:6683:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905944:0:6683:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905947:0:6683:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905950:0:6683:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206683.905952:0:6683:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.905954:0:6683:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad440/0xbbeffb005ec3e004 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6683 initiator: MDT0 00000080:00020000:1.0:1715206683.905967:0:6683:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206683.905991:0:6683:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:1.0:1715206683.905995:0:6683:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.906004:0:6683:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906008:0:6683:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906012:0:6683:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6683 initiator: MDT0 00000100:00100000:1.0:1715206683.906016:0:6683:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b5d15a80 going to sleep for 0 seconds 00000100:00100000:1.0:1715206683.906019:0:6683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a616d80 pname:cluuid:pid:xid:nid:opc:job rm:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6683:1798523503857920:192.168.202.138@tcp:101:rm.0 00010000:00010000:1.0:1715206683.906022:0:6683:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906025:0:6683:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906028:0:6683:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906031:0:6683:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906034:0:6683:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906036:0:6683:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880136bacd80 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:1.0:1715206683.906038:0:6683:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906059:0:6683:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906062:0:6683:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906065:0:6683:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206683.906067:0:6683:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6683 initiator: MDT0 00010000:00010000:1.0:1715206683.906070:0:6683:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bacd80/0xbbeffb005ec3e00b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6683 initiator: MDT0 00000080:00020000:1.0:1715206683.906077:0:6683:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:2.0:1715206683.908143:0:6684:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:2.0:1715206683.908151:0:6684:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1715206683.908167:0:6684:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908172:0:6684:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908177:0:6684:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6684 initiator: MDT0 00000100:00100000:2.0:1715206683.908183:0:6684:0:(client.c:2542:ptlrpc_set_wait()) set ffff8801373919c0 going to sleep for 0 seconds 00000100:00100000:2.0:1715206683.908187:0:6684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800b6324a80 pname:cluuid:pid:xid:nid:opc:job dd:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6684:1798523503857984:192.168.202.138@tcp:101:dd.0 00010000:00010000:2.0:1715206683.908192:0:6684:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908196:0:6684:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908199:0:6684:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908202:0:6684:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908211:0:6684:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908214:0:6684:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa30ad00 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:2.0:1715206683.908216:0:6684:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908219:0:6684:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908222:0:6684:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908225:0:6684:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1715206683.908227:0:6684:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6684 initiator: MDT0 00010000:00010000:2.0:1715206683.908229:0:6684:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aa30ad00/0xbbeffb005ec3e012 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6684 initiator: MDT0 00000080:00020000:2.0:1715206683.908239:0:6684:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206683.911089:0:6685:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:1.0:1715206683.911097:0:6685:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.911113:0:6685:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911118:0:6685:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911123:0:6685:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6685 initiator: MDT0 00000100:00100000:1.0:1715206683.911129:0:6685:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b5d15a80 going to sleep for 0 seconds 00000100:00100000:1.0:1715206683.911133:0:6685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a616d80 pname:cluuid:pid:xid:nid:opc:job md5sum:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6685:1798523503858048:192.168.202.138@tcp:101:md5sum.0 00010000:00010000:1.0:1715206683.911137:0:6685:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911141:0:6685:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911149:0:6685:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911153:0:6685:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911156:0:6685:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911158:0:6685:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880136baeac0 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:1.0:1715206683.911160:0:6685:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911163:0:6685:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911166:0:6685:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911169:0:6685:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206683.911171:0:6685:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6685 initiator: MDT0 00010000:00010000:1.0:1715206683.911174:0:6685:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136baeac0/0xbbeffb005ec3e019 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6685 initiator: MDT0 00000080:00020000:1.0:1715206683.911183:0:6685:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206683.913769:0:6686:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:1.0:1715206683.913776:0:6686:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206683.913791:0:6686:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913802:0:6686:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913806:0:6686:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6686 initiator: MDT0 00000100:00100000:1.0:1715206683.913812:0:6686:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b5d15a80 going to sleep for 0 seconds 00000100:00100000:1.0:1715206683.913816:0:6686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a616d80 pname:cluuid:pid:xid:nid:opc:job lfs:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6686:1798523503858112:192.168.202.138@tcp:101:lfs.0 00010000:00010000:1.0:1715206683.913820:0:6686:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913823:0:6686:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913827:0:6686:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913830:0:6686:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913834:0:6686:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913836:0:6686:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880136bad8c0 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:1.0:1715206683.913838:0:6686:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913841:0:6686:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913844:0:6686:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913851:0:6686:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206683.913853:0:6686:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6686 initiator: MDT0 00010000:00010000:1.0:1715206683.913856:0:6686:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff880136bad8c0/0xbbeffb005ec3e020 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6686 initiator: MDT0 00000080:00020000:1.0:1715206683.913866:0:6686:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715206684.162918:0:6696:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f580c8),name=/ 00000002:00010000:3.0:1715206684.162934:0:6696:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206684.162973:0:6696:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.162979:0:6696:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.162985:0:6696:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6696 initiator: MDT0 00000100:00100000:3.0:1715206684.162996:0:6696:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b051e900 going to sleep for 0 seconds 00000100:00100000:3.0:1715206684.163003:0:6696:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800aafdbb80 pname:cluuid:pid:xid:nid:opc:job lfs:46af332d-2ee7-4d47-b762-82c1eb2b6bbd:6696:1798523503858176:192.168.202.138@tcp:101:lfs.0 00010000:00010000:3.0:1715206684.163010:0:6696:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163014:0:6696:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163018:0:6696:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163022:0:6696:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163025:0:6696:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163029:0:6696:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aae76880 already unused, calling callback (ffffffffa1044e10) 00010000:00010000:3.0:1715206684.163031:0:6696:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163038:0:6696:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163067:0:6696:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163070:0:6696:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206684.163074:0:6696:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6696 initiator: MDT0 00010000:00010000:3.0:1715206684.163077:0:6696:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b0712000 lock: ffff8800aae76880/0xbbeffb005ec3e027 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 6696 initiator: MDT0 00000080:00020000:3.0:1715206684.163091:0:6696:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000001:02000400:1.0:1715206684.475411:0:6828:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12c: @@@@@@ FAIL: release /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm failed 00010000:00010000:0.0:1715206684.534129:0:3698:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 370 lines, 370 kept, 0 dropped, 0 bad.