00010000:00010000:0.0F:1715207762.240696:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:2.0F:1715207762.852653:0:26930:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=17220f0 00800000:00000080:2.0:1715207762.852684:0:26930:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a9739f20 uarg=0000000000000000 00000002:00000080:2.0:1715207762.852694:0:26930:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a9739f20 uarg=0000000000000000 00000100:00100000:2.0:1715207762.852760:0:26930:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e300 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:26930:1798523505022016:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715207762.852869:0:26930:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f480 going to sleep for 11 seconds 00000100:00100000:2.0:1715207762.854784:0:26930:0:(client.c:2956:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8800a9618000: committing for last_committed 4294967416 gen 1 00000100:00100000:2.0:1715207762.854795:0:26930:0:(client.c:3005:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff8800a91e5180 x1798523505021440/t4294967416(4294967416) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207775 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000100:00100000:2.0:1715207762.854809:0:26930:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e300 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:26930:1798523505022016:192.168.201.123@tcp:60:lhsmtool_posix.0 00000002:00000080:2.0:1715207762.854827:0:26930:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800a9739f20 uarg=0000000000000000 00000100:00100000:2.0:1715207762.854846:0:26930:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e300 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:26930:1798523505022080:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715207762.854876:0:26930:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f480 going to sleep for 11 seconds 00000100:00100000:3.0F:1715207762.857550:0:26930:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e300 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:26930:1798523505022080:192.168.201.123@tcp:60:lhsmtool_posix.0 00000080:00200008:3.0:1715207762.857728:0:26930:0:(file.c:430:ll_file_release()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:3.0:1715207762.857782:0:26930:0:(mdc_request.c:927:mdc_close()) @@@ matched open req@ffff8800a91e5180 x1798523505021440/t4294967416(4294967416) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207775 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000020:00100000:3.0:1715207762.857820:0:26930:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1715207762.857831:0:26930:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a91e5880 x1798523505022208/t0(0) o35->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:23/10 lens 392/840 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00000100:00100000:3.0:1715207762.857861:0:26930:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a91e5880 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:26930:1798523505022208:192.168.201.123@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1715207762.857948:0:26930:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c8180 going to sleep for 11 seconds 00000100:00100000:3.0:1715207762.859751:0:26930:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e5880 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:26930:1798523505022208:192.168.201.123@tcp:35:lhsmtool_posix.0 00000002:00100000:3.0:1715207762.859778:0:26930:0:(mdc_request.c:837:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8800a91e5180 x1798523505021440/t4294967416(4294967416) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207775 ref 1 fl Complete:RQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200008:3.0:1715207762.859819:0:26930:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200008:0.0:1715207762.863366:0:26930:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00010000:00010000:0.0:1715207763.242834:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1715207764.040760:0:17330:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715207764.040769:0:17330:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715207764.040780:0:17330:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715207764.040784:0:17330:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715207764.040799:0:17330:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715207764.040803:0:17330:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:2.0:1715207764.040855:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505022336:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715207764.040856:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2003:1798523505022272:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715207764.040970:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505022400:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715207764.040970:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505022464:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715207764.040998:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505022528:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715207764.042879:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2003:1798523505022272:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715207764.042883:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505022336:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715207764.042897:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505022400:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715207764.042909:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505022528:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715207764.042911:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505022464:192.168.201.123@tcp:400:kworker.0 00010000:00010000:3.0:1715207764.244687:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0F:1715207765.246754:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1715207765.610472:0:28156:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 18:36:05 (1715207765) 00000080:00200008:3.0:1715207765.740695:0:28199:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715207765.740709:0:28199:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715207765.740744:0:28199:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207765.740770:0:28199:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207765.740793:0:28199:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715207765.740813:0:28199:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715207765.740818:0:28199:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570034 00000080:00010000:3.0:1715207765.740830:0:28199:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570034 from it ffff8800a98f7cc8 00010000:00010000:3.0:1715207765.740837:0:28199:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207765.740852:0:28199:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00200008:3.0:1715207765.741611:0:28199:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715207765.741621:0:28199:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715207765.741641:0:28199:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207765.741711:0:28199:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741725:0:28199:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741742:0:28199:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28199 initiator: MDT0 00000100:00100000:3.0:1715207765.741787:0:28199:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a90c83c0 going to sleep for 0 seconds 00000100:00100000:3.0:1715207765.741805:0:28199:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e5880 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:04532c2b-e5e9-420d-abb0-4480aab1e378:28199:1798523505022592:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1715207765.741823:0:28199:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741836:0:28199:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741847:0:28199:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741857:0:28199:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741869:0:28199:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741881:0:28199:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa54a400 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715207765.741888:0:28199:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741899:0:28199:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741912:0:28199:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741922:0:28199:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715207765.741931:0:28199:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28199 initiator: MDT0 00010000:00010000:3.0:1715207765.741942:0:28199:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa54a400/0x7561918bd5570122 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28199 initiator: MDT0 00000080:00020000:3.0:1715207765.741982:0:28199:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:3.0:1715207765.742010:0:28199:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715207765.742017:0:28199:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715207765.742029:0:28199:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207765.742040:0:28199:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207765.742054:0:28199:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715207765.742067:0:28199:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715207765.742072:0:28199:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570034 00000080:00010000:3.0:1715207765.742078:0:28199:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570034 from it ffff8800a98f7cc8 00010000:00010000:3.0:1715207765.742083:0:28199:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207765.742096:0:28199:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715207765.742211:0:28199:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:3.0:1715207765.742225:0:28199:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715207765.742237:0:28199:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207765.742251:0:28199:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1715207765.742264:0:28199:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:3.0:1715207765.742275:0:28199:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570034 00000080:00010000:3.0:1715207765.742282:0:28199:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570034 from it ffff8800a98f7b80 00010000:00010000:3.0:1715207765.742288:0:28199:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715207765.742301:0:28199:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1715207765.742317:0:28199:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:3.0:1715207765.742329:0:28199:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715207765.743384:0:28200:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715207765.743402:0:28200:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207765.743413:0:28200:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207765.743427:0:28200:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715207765.743437:0:28200:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715207765.743440:0:28200:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570034 00000080:00010000:1.0:1715207765.743445:0:28200:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570034 from it ffff8800a92dfb80 00010000:00010000:1.0:1715207765.743449:0:28200:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207765.743458:0:28200:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715207765.743469:0:28200:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 44 00000080:00200008:1.0:1715207765.743476:0:28200:0:(file.c:871:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200000:1.0:1715207765.743503:0:28200:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8),name=/ 00000002:00010000:1.0:1715207765.743509:0:28200:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207765.743515:0:28200:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207765.743551:0:28200:0:(ldlm_lock.c:1542:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1715207765.743592:0:28200:0:(dcache.c:301:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) 00000080:00010000:1.0:1715207765.743595:0:28200:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b02580c8) for lock 0x7561918bd5570034 00000080:00010000:1.0:1715207765.743600:0:28200:0:(dcache.c:194:ll_intent_drop_lock()) releasing lock with cookie 0x7561918bd5570034 from it ffff8800a92dfbb0 00010000:00010000:1.0:1715207765.743611:0:28200:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1715207765.743620:0:28200:0:(ldlm_lock.c:929:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa54a880/0x7561918bd5570034 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xfa8b5bed87891e59 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1715207765.743629:0:28200:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715207765.743637:0:28200:0:(dcache.c:318:ll_revalidate_dentry()) VFS Op:name=.lustre, flags=17 00000080:00200000:1.0:1715207765.743642:0:28200:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x1:0x0](ffff880136f410c8), inode mode 41ed mask 1 00000080:00200000:1.0:1715207765.743653:0:28200:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), file ffff8800a9236f00, open_flags 8000, mode 0 opened 0 00000080:00200000:1.0:1715207765.743661:0:28200:0:(namei.c:838:ll_lookup_it()) VFS Op:name=fid, dir=[0x200000002:0x1:0x0](ffff880136f410c8), intent=open 00000002:00010000:1.0:1715207765.743675:0:28200:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: fid,[0x0:0x0:0x0]) in obj [0x200000002:0x1:0x0], intent: open flags 0100001 00010000:00010000:1.0:1715207765.743716:0:28200:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 28200 initiator: MDT0 00010000:00010000:1.0:1715207765.743724:0:28200:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 28200 initiator: MDT0 00000020:00100000:1.0:1715207765.743735:0:28200:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1715207765.743747:0:28200:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a9678a80 x1798523505022720/t0(0) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/66536 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00010000:00010000:1.0:1715207765.743758:0:28200:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 28200 initiator: MDT0 00000100:00100000:1.0:1715207765.743775:0:28200:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9678a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:28200:1798523505022720:192.168.201.123@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1715207765.743980:0:28200:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a96c3540 going to sleep for 11 seconds 00000100:00100000:2.0:1715207765.746440:0:28200:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9678a80 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:28200:1798523505022720:192.168.201.123@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1715207765.746460:0:28200:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746477:0:28200:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746487:0:28200:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 4/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746497:0:28200:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746508:0:28200:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746517:0:28200:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa66a640 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:2.0:1715207765.746522:0:28200:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746599:0:28200:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 4/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746610:0:28200:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 5/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746619:0:28200:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1715207765.746625:0:28200:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 3/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 28200 initiator: MDT0 00010000:00010000:2.0:1715207765.746635:0:28200:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800a9618000 lock: ffff8800aa66a640/0x7561918bd5570129 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 28200 initiator: MDT0 00000002:00100000:2.0:1715207765.746667:0:28200:0:(mdc_locks.c:838:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff8800a9678a80 x1798523505022720/t4294967418(4294967418) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207781 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000002:00100000:2.0:1715207765.746688:0:28200:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a9678a80 x1798523505022720/t4294967418(4294967418) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207781 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715207765.746723:0:28200:0:(llite_lib.c:2872:ll_update_inode()) inode=[0x200000002:0x2:0x0], updating i_size 4096 00000080:00200000:2.0:1715207765.746728:0:28200:0:(namei.c:157:ll_iget()) got inode: [0x200000002:0x2:0x0](ffff8800b02788c8): rc = 0 00000080:00010000:2.0:1715207765.746736:0:28200:0:(llite_internal.h:1965:ll_set_lock_data()) setting l_data to inode [0x200000002:0x2:0x0](ffff8800b02788c8) for lock 0x0 00000080:00200008:2.0:1715207765.746756:0:28200:0:(file.c:871:ll_file_open()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:2.0:1715207765.746777:0:28200:0:(mdc_request.c:816:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a9678a80 x1798523505022720/t4294967418(4294967418) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715207781 ref 2 fl Complete:RPQU/204/0 rc 301/301 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200000:2.0:1715207765.746842:0:28200:0:(file.c:6060:ll_inode_permission()) VFS Op:inode=[0x200000002:0x2:0x0](ffff8800b02788c8), inode mode 4040 mask 44 00000080:00200080:2.0:1715207765.746912:0:28200:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=1b120f0 00800000:00000080:2.0:1715207765.746927:0:28200:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b5edc320 uarg=0000000000000000 00000002:00000080:2.0:1715207765.746944:0:28200:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b5edc320 uarg=0000000000000000 00000100:00100000:2.0:1715207765.746978:0:28200:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:28200:1798523505022784:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715207765.747046:0:28200:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1715207765.748628:0:28200:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:28200:1798523505022784:192.168.201.123@tcp:59:lhsmtool_posix.0 00000002:00000080:2.0:1715207765.748649:0:28200:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800b5edc320 uarg=0000000000000000 00000100:00100000:2.0:1715207765.748675:0:28200:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:28200:1798523505022848:192.168.201.123@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715207765.748714:0:28200:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1715207765.751002:0:28200:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:28200:1798523505022848:192.168.201.123@tcp:59:lhsmtool_posix.0 00000080:00200000:1.0:1715207765.754017:0:28201:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715207765.754033:0:28201:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207765.754064:0:28201:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754079:0:28201:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754089:0:28201:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00000100:00100000:1.0:1715207765.754100:0:28201:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a96c3900 going to sleep for 0 seconds 00000100:00100000:1.0:1715207765.754108:0:28201:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9679180 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:28201:1798523505022912:192.168.201.123@tcp:101:lfs.0 00010000:00010000:1.0:1715207765.754116:0:28201:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754123:0:28201:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754128:0:28201:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754134:0:28201:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754140:0:28201:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754146:0:28201:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa66aac0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715207765.754149:0:28201:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754155:0:28201:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754161:0:28201:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754175:0:28201:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715207765.754180:0:28201:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754185:0:28201:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66aac0/0x7561918bd5570130 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28201 initiator: MDT0 00000080:00020000:1.0:1715207765.754203:0:28201:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715207765.754228:0:28201:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715207765.754236:0:28201:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207765.754256:0:28201:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754262:0:28201:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754268:0:28201:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00000100:00100000:1.0:1715207765.754277:0:28201:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a96c3900 going to sleep for 0 seconds 00000100:00100000:1.0:1715207765.754283:0:28201:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9679180 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:28201:1798523505022976:192.168.201.123@tcp:101:lfs.0 00010000:00010000:1.0:1715207765.754289:0:28201:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754295:0:28201:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754301:0:28201:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754306:0:28201:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754318:0:28201:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754324:0:28201:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa66a400 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715207765.754327:0:28201:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754332:0:28201:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754338:0:28201:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754343:0:28201:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715207765.754346:0:28201:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754351:0:28201:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa66a400/0x7561918bd5570137 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28201 initiator: MDT0 00000080:00020000:1.0:1715207765.754366:0:28201:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:1.0:1715207765.754382:0:28201:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1715207765.754386:0:28201:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715207765.754393:0:28201:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715207765.754410:0:28201:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754417:0:28201:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754424:0:28201:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00000100:00100000:1.0:1715207765.754441:0:28201:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800a96c3900 going to sleep for 0 seconds 00000100:00100000:1.0:1715207765.754447:0:28201:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9679180 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:28201:1798523505023040:192.168.201.123@tcp:101:lfs.0 00010000:00010000:1.0:1715207765.754453:0:28201:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754459:0:28201:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754464:0:28201:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754469:0:28201:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754475:0:28201:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754480:0:28201:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa669680 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715207765.754483:0:28201:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754488:0:28201:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754493:0:28201:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754498:0:28201:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715207765.754501:0:28201:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28201 initiator: MDT0 00010000:00010000:1.0:1715207765.754507:0:28201:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa669680/0x7561918bd557013e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 28201 initiator: MDT0 00000080:00020000:1.0:1715207765.754553:0:28201:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00010000:00010000:0.0:1715207766.248763:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1715207766.282937:0:28334:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12u: @@@@@@ FAIL: failed to mkdir /mnt/lustre/d12u.sanity-hsm Debug log: 222 lines, 222 kept, 0 dropped, 0 bad.