00010000:00010000:3.0F:1715209965.644791:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1715209966.131911:0:14046:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:3.0:1715209966.131944:0:14046:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1715209966.132013:0:14046:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132026:0:14046:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132037:0:14046:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14046 initiator: MDT0 00000100:00100000:3.0:1715209966.132057:0:14046:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012db9f780 going to sleep for 0 seconds 00000100:00100000:3.0:1715209966.132070:0:14046:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a91e5180 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:14046:1798523505205248:192.168.201.123@tcp:101:lfs.0 00010000:00010000:3.0:1715209966.132083:0:14046:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132092:0:14046:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132099:0:14046:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132106:0:14046:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132114:0:14046:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132120:0:14046:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a9609d40 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:3.0:1715209966.132125:0:14046:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132132:0:14046:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132146:0:14046:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132153:0:14046:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1715209966.132160:0:14046:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 14046 initiator: MDT0 00010000:00010000:3.0:1715209966.132166:0:14046:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800a9609d40/0x7561918bd5570a21 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 14046 initiator: MDT0 00000080:00020000:3.0:1715209966.132190:0:14046:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000080:00200000:1.0F:1715209966.136153:0:14047:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:1.0:1715209966.136173:0:14047:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1715209966.136211:0:14047:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136217:0:14047:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136224:0:14047:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14047 initiator: MDT0 00000100:00100000:1.0:1715209966.136235:0:14047:0:(client.c:2542:ptlrpc_set_wait()) set ffff8800b5ff2840 going to sleep for 0 seconds 00000100:00100000:1.0:1715209966.136242:0:14047:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a967bb80 pname:cluuid:pid:xid:nid:opc:job stat:04532c2b-e5e9-420d-abb0-4480aab1e378:14047:1798523505205312:192.168.201.123@tcp:101:stat.0 00010000:00010000:1.0:1715209966.136249:0:14047:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136253:0:14047:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136256:0:14047:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136260:0:14047:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136264:0:14047:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136267:0:14047:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa549d40 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:1.0:1715209966.136270:0:14047:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136273:0:14047:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136282:0:14047:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136286:0:14047:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1715209966.136289:0:14047:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 14047 initiator: MDT0 00010000:00010000:1.0:1715209966.136292:0:14047:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa549d40/0x7561918bd5570a28 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 14047 initiator: MDT0 00000080:00020000:1.0:1715209966.136306:0:14047:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00010000:00010000:0.0F:1715209966.646700:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0F:1715209967.032695:0:9970:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715209967.032708:0:9970:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715209967.032743:0:9970:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715209967.032745:0:9970:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715209967.032752:0:9970:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715209967.032754:0:9970:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:2.0:1715209967.032779:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505205440:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209967.032810:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2003:1798523505205376:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715209967.032842:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505205568:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209967.032884:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505205504:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209967.032906:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505205632:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715209967.034268:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:04532c2b-e5e9-420d-abb0-4480aab1e378:2002:1798523505205440:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715209967.034276:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505205568:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209967.034365:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2003:1798523505205376:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209967.034384:0:2003:0:(client.c:2956:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8800a9618000: committing for last_committed 4294967520 gen 1 00000100:00100000:3.0:1715209967.034392:0:2003:0:(client.c:3005:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff8800a91e5500 x1798523505204352/t4294967520(4294967520) o101->lustre-MDT0000-mdc-ffff8800a9618000@192.168.201.123@tcp:12/10 lens 864/608 e 0 to 0 dl 1715209980 ref 1 fl Complete:RPQU/204/ffffffff rc 301/-1 job:'lhsmtool_posix.0' uid:0 gid:0 00000100:00100000:3.0:1715209967.034404:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505205504:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209967.034417:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505205632:192.168.201.123@tcp:400:kworker.0 00010000:00010000:1.0:1715209967.648725:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1715209968.650730:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1715209969.652746:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1715209970.654686:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1715209971.656723:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1715209972.040754:0:9970:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715209972.040759:0:9970:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715209972.040764:0:9970:0:(pinger.c:253:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715209972.040766:0:9970:0:(pinger.c:271:ptlrpc_pinger_process_import()) 04532c2b-e5e9-420d-abb0-4480aab1e378->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00080000:2.0:1715209972.040774:0:9970:0:(pinger.c:253:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: level EVICTED/10 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1715209972.040775:0:9970:0:(pinger.c:271:ptlrpc_pinger_process_import()) 1bc50a3e-f07c-4b55-83c3-5f3c3d406528->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: EVICTED) 00000100:00100000:2.0:1715209972.040808:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:04532c2b-e5e9-420d-abb0-4480aab1e378:2003:1798523505205760:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209972.040873:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2002:1798523505205696:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715209972.040873:0:2003:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505205888:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209972.040950:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505205824:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209972.041008:0:2002:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505205952:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715209972.042165:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:04532c2b-e5e9-420d-abb0-4480aab1e378:2003:1798523505205760:192.168.201.123@tcp:400:kworker.0 00000100:00100000:2.0:1715209972.042183:0:2003:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2003:1798523505205888:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209972.042267:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd:2002:1798523505205696:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209972.042288:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505205824:192.168.201.123@tcp:400:kworker.0 00000100:00100000:3.0:1715209972.042300:0:2002:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800abe2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:1bc50a3e-f07c-4b55-83c3-5f3c3d406528:2002:1798523505205952:192.168.201.123@tcp:400:kworker.0 00010000:00010000:0.0:1715209972.658711:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1715209973.660773:0:18185:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1715209974.662730:0:2527:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1715209975.664725:0:5362:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1715209976.146285:0:14059:0:(file.c:5598:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff880136f400c8),name=/ 00000002:00010000:0.0:1715209976.146301:0:14059:0:(mdc_locks.c:1426:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1715209976.146339:0:14059:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146345:0:14059:0:(ldlm_request.c:1038:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146351:0:14059:0:(ldlm_request.c:1125:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14059 initiator: MDT0 00000100:00100000:0.0:1715209976.146362:0:14059:0:(client.c:2542:ptlrpc_set_wait()) set ffff88012a421900 going to sleep for 0 seconds 00000100:00100000:0.0:1715209976.146370:0:14059:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a96a6d80 pname:cluuid:pid:xid:nid:opc:job lfs:04532c2b-e5e9-420d-abb0-4480aab1e378:14059:1798523505206016:192.168.201.123@tcp:101:lfs.0 00010000:00010000:0.0:1715209976.146376:0:14059:0:(ldlm_request.c:655:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146381:0:14059:0:(ldlm_request.c:572:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146384:0:14059:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146388:0:14059:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146392:0:14059:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146395:0:14059:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800aa586880 already unused, calling callback (ffffffffa1034e10) 00010000:00010000:0.0:1715209976.146401:0:14059:0:(ldlm_request.c:1599:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146405:0:14059:0:(ldlm_request.c:1609:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146409:0:14059:0:(ldlm_request.c:1300:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146412:0:14059:0:(ldlm_request.c:1316:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1715209976.146415:0:14059:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 14059 initiator: MDT0 00010000:00010000:0.0:1715209976.146418:0:14059:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800b6238800 lock: ffff8800aa586880/0x7561918bd5570a2f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 14059 initiator: MDT0 00000080:00020000:0.0:1715209976.146432:0:14059:0:(file.c:5575:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 00000001:02000400:3.0:1715209976.513783:0:14194:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_31b: @@@@@@ FAIL: /home/green/git/lustre-release/lustre/utils/lfs hsm_state /mnt/lustre/d31b.sanity-hsm/f31b.sanity-hsm failed (run as root) 00010000:00010000:0.0:1715209976.666693:0:2469:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 107 lines, 107 kept, 0 dropped, 0 bad.