00010000:00010000:0.0F:1715206652.022734:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:2.0F:1715206652.248370:0:31552:0:(dir.c:1969:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b02580c8) cmd=401866d5 arg=1ae10f0 00800000:00000080:2.0:1715206652.248388:0:31552:0:(lmv_obd.c:911:lmv_iocontrol()) lustre-clilmv-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800aa69f100 uarg=0000000000000000 00000002:00000080:2.0:1715206652.248394:0:31552:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0000-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800aa69f100 uarg=0000000000000000 00000100:00100000:2.0:1715206652.248433:0:31552:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f480 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:31552:1798523504923136:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715206652.248495:0:31552:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f600 going to sleep for 143 seconds 00000100:00100000:2.0:1715206652.250184:0:31552:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f480 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:31552:1798523504923136:192.168.201.123@tcp:60:lhsmtool_posix.0 00000002:00000080:2.0:1715206652.250197:0:31552:0:(mdc_request.c:2233:mdc_iocontrol()) lustre-MDT0001-mdc-ffff8800a9618000: cmd=401866d5 len=24 karg=ffff8800aa69f100 uarg=0000000000000000 00000100:00100000:2.0:1715206652.250212:0:31552:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f480 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:31552:1798523504923200:192.168.201.123@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1715206652.250255:0:31552:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012aa8f600 going to sleep for 143 seconds 00000100:00100000:0.0:1715206652.251293:0:31552:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f480 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:31552:1798523504923200:192.168.201.123@tcp:60:lhsmtool_posix.0 00000080:00200008:0.0:1715206652.251320:0:31552:0:(file.c:430:ll_file_release()) START file fid:[0x200000002:0x2:0x0](ffff8800b02788c8), flags 100000 00000002:00100000:0.0:1715206652.251345:0:31552:0:(mdc_request.c:927:mdc_close()) @@@ matched open req@ffff8800a9678000 x1798523504922560/t4294967382(4294967382) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206798 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000020:00100000:0.0:1715206652.251359:0:31552:0:(genops.c:2297:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800a9618000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1715206652.251362:0:31552:0:(client.c:765:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a96a4380 x1798523504923328/t0(0) o35->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:23/10 lens 392/840 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 00000100:00100000:0.0:1715206652.251370:0:31552:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a96a4380 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:31552:1798523504923328:192.168.201.123@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1715206652.251396:0:31552:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012e1e5180 going to sleep for 11 seconds 00000100:00100000:2.0:1715206652.252641:0:31552:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a4380 pname:cluuid:pid:xid:nid:opc:job lhsmtool_posix:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:31552:1798523504923328:192.168.201.123@tcp:35:lhsmtool_posix.0 00000002:00100000:2.0:1715206652.252657:0:31552:0:(mdc_request.c:837:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8800a9678000 x1798523504922560/t4294967382(4294967382) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715206798 ref 1 fl Complete:RQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000080:00200008:2.0:1715206652.252683:0:31552:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000080:00200008:3.0F:1715206652.254860:0:31552:0:(file.c:430:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff8800b02580c8), flags 100000 00000100:00080000:2.0:1715206652.984637:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715206652.984642:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715206652.984648:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715206652.984650:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715206652.984656:0:302:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715206652.984658:0:302:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:3.0:1715206652.984676:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2002:1798523504923392:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715206652.984680:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:04532c2b-e5e9-420d-abb0-4480aab1e378:2003:1798523504923456:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715206652.984732:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523504923584:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715206652.984733:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523504923520:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715206652.984744:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523504923648:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715206652.986188:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2002:1798523504923392:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715206652.986198:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523504923520:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715206652.986203:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523504923648:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715206652.986214:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:04532c2b-e5e9-420d-abb0-4480aab1e378:2003:1798523504923456:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715206652.986221:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523504923584:192.168.201.123@tcp:400:kworker.0 00010000:00010000:1.0F:1715206653.024697:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1715206654.026685:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1715206654.411494:0:309:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 11a: Import a file ==================== 18:17:34 (1715206654) 00000080:00200000:3.0:1715206654.423435:0:316:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206654.423458:0:316:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206654.423528:0:316:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423535:0:316:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423543:0:316:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00000100:00100000:3.0:1715206654.423559:0:316:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012a421300 going to sleep for 0 seconds 00000100:00100000:3.0:1715206654.423591:0:316:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413480 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:316:1798523504923712:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715206654.423600:0:316:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423605:0:316:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423614:0:316:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423619:0:316:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423624:0:316:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423628:0:316:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa549680 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206654.423631:0:316:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423635:0:316:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423639:0:316:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423643:0:316:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206654.423647:0:316:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423651:0:316:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549680/0x7561918bd556fc7c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 316 initiator: MDT0 00000080:00020000:3.0:1715206654.423668:0:316:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:3.0:1715206654.423683:0:316:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206654.423688:0:316:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206654.423699:0:316:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423708:0:316:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423712:0:316:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00000100:00100000:3.0:1715206654.423717:0:316:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012a421300 going to sleep for 0 seconds 00000100:00100000:3.0:1715206654.423720:0:316:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413480 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:316:1798523504923776:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715206654.423724:0:316:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423727:0:316:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423730:0:316:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423733:0:316:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423736:0:316:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423738:0:316:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa548d80 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206654.423740:0:316:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423742:0:316:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423745:0:316:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423753:0:316:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206654.423755:0:316:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423758:0:316:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa548d80/0x7561918bd556fc83 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 316 initiator: MDT0 00000080:00020000:3.0:1715206654.423766:0:316:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200008:3.0:1715206654.423775:0:316:0:(file.c:5701:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff880136f400c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1715206654.423778:0:316:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715206654.423782:0:316:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715206654.423790:0:316:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423794:0:316:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423798:0:316:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00000100:00100000:3.0:1715206654.423802:0:316:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012a421300 going to sleep for 0 seconds 00000100:00100000:3.0:1715206654.423805:0:316:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012a413480 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:316:1798523504923840:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715206654.423808:0:316:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423811:0:316:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423814:0:316:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423817:0:316:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423823:0:316:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423826:0:316:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa549b00 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715206654.423828:0:316:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423831:0:316:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423833:0:316:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423836:0:316:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715206654.423837:0:316:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 316 initiator: MDT0 00010000:00010000:3.0:1715206654.423840:0:316:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549b00/0x7561918bd556fc8a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 316 initiator: MDT0 00000080:00020000:3.0:1715206654.423847:0:316:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0:1715206654.492995:0:366:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715206654.493015:0:366:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715206654.493065:0:366:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493072:0:366:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493078:0:366:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 366 initiator: MDT0 00000100:00100000:1.0:1715206654.493093:0:366:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800ad7dd0c0 going to sleep for 0 seconds 00000100:00100000:1.0:1715206654.493103:0:366:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a9679180 pname:cluuid:pid:xid:nid:opc:job mkdir:04532c2b-e5e9-420d-abb0-4480aab1e378:366:1798523504923904:192.168.201.123@tcp:101:mkdir.0 00010000:00010000:1.0:1715206654.493111:0:366:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493115:0:366:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493119:0:366:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493123:0:366:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493128:0:366:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493131:0:366:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa6698c0 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715206654.493134:0:366:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493138:0:366:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493146:0:366:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493151:0:366:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715206654.493155:0:366:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 366 initiator: MDT0 00010000:00010000:1.0:1715206654.493158:0:366:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa6698c0/0x7561918bd556fc91 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 366 initiator: MDT0 00000080:00020000:1.0:1715206654.493175:0:366:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000001:02000400:3.0:1715206654.801396:0:504:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_11a: @@@@@@ FAIL: cannot create directory '/mnt/lustre/d11a.sanity-hsm' 00010000:00010000:1.0:1715206655.028778:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 123 lines, 123 kept, 0 dropped, 0 bad.