00010000:00010000:3.0F:1721219531.660783:0:11573:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0F:1721219531.910702:0:1828:0:(events.c:361:request_in_callback()) peer: 12345-192.168.201.158@tcp (source: 12345-192.168.201.158@tcp) 00000100:00100000:2.0:1721219531.910753:0:1828:0:(events.c:361:request_in_callback()) peer: 12345-192.168.201.158@tcp (source: 12345-192.168.201.158@tcp) 00000100:00100000:3.0:1721219531.910775:0:11164:0:(service.c:173:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1721219531.910794:0:11164:0:(service.c:2140:ptlrpc_server_handle_req_in()) got req x1804827459638912 00000100:00100000:1.0F:1721219531.910818:0:4603:0:(service.c:2140:ptlrpc_server_handle_req_in()) got req x1804827459638976 00000100:00100000:3.0:1721219531.910829:0:11164:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.158@tcp, seq: 181535 00000100:00100000:3.0:1721219531.910834:0:11164:0:(service.c:2309:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a1d9c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+4:21777:x1804827459638912:12345-192.168.201.158@tcp:104: 00000100:00100000:1.0:1721219531.910855:0:4603:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.158@tcp, seq: 9882 00000100:00100000:1.0:1721219531.910860:0:4603:0:(service.c:2309:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a8600980 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+4:21777:x1804827459638976:12345-192.168.201.158@tcp:104: 00010000:00010000:3.0:1721219531.910879:0:11164:0:(ldlm_lockd.c:2154:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1721219531.910882:0:11164:0:(service.c:2358:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a1d9c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+4:21777:x1804827459638912:12345-192.168.201.158@tcp:104: Request processed in 49us (185us total) trans 0 rc 0/0 00000100:00100000:3.0:1721219531.910886:0:11164:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.158@tcp, seq: 181535 00010000:00010000:1.0:1721219531.910891:0:4603:0:(ldlm_lockd.c:2154:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00010000:00010000:0.0F:1721219531.910895:0:11574:0:(ldlm_lockd.c:2784:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:1.0:1721219531.910896:0:4603:0:(service.c:2358:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a8600980 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+3:21777:x1804827459638976:12345-192.168.201.158@tcp:104: Request processed in 36us (142us total) trans 0 rc 0/0 00000100:00100000:1.0:1721219531.910903:0:4603:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.158@tcp, seq: 9882 00010000:00010000:0.0:1721219531.910906:0:11574:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.158@tcp lock: ffff8800a8deef40/0xe0b47d8ca5f72c82 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xd4836d281e0021e9 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219531.910916:0:11574:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a8deef40 already unused, calling callback (ffffffffa030c5c0) 00010000:00010000:3.0:1721219531.910917:0:4613:0:(ldlm_lockd.c:2784:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 10000000:00010000:0.0:1721219531.910918:0:11574:0:(mgc_request.c:1019:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.158@tcp lock: ffff8800a8deef40/0xe0b47d8ca5f72c82 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0xd4836d281e0021e9 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1721219531.910919:0:4613:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.158@tcp lock: ffff8800a9bae1c0/0xe0b47d8ca5f72c7b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xd4836d281e0021e2 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1721219531.910925:0:4613:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a9bae1c0 already unused, calling callback (ffffffffa030c5c0) 00010000:00010000:0.0:1721219531.910925:0:11574:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.158@tcp lock: ffff8800a8deef40/0xe0b47d8ca5f72c82 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xd4836d281e0021e9 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:00010000:3.0:1721219531.910927:0:4613:0:(mgc_request.c:1019:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.158@tcp lock: ffff8800a9bae1c0/0xe0b47d8ca5f72c7b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0xd4836d281e0021e2 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:00010000:0.0:1721219531.910929:0:11574:0:(mgc_request.c:1025:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.158@tcp lock: ffff8800a8deef40/0xe0b47d8ca5f72c82 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xd4836d281e0021e9 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1721219531.910932:0:4613:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.158@tcp lock: ffff8800a9bae1c0/0xe0b47d8ca5f72c7b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0xd4836d281e0021e2 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1721219531.910933:0:11574:0:(mgc_request.c:1029:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:00010000:3.0:1721219531.910937:0:4613:0:(mgc_request.c:1025:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.158@tcp lock: ffff8800a9bae1c0/0xe0b47d8ca5f72c7b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0xd4836d281e0021e2 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:01000000:3.0:1721219531.910942:0:4613:0:(mgc_request.c:1029:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1721219531.910976:0:4613:0:(ldlm_request.c:1301:ldlm_cancel_pack()) ### packing ns: MGC192.168.201.158@tcp lock: ffff8800a9bae1c0/0xe0b47d8ca5f72c7b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e0021e2 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1721219531.910982:0:4613:0:(ldlm_request.c:1305:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1721219531.910984:0:11574:0:(ldlm_request.c:1301:ldlm_cancel_pack()) ### packing ns: MGC192.168.201.158@tcp lock: ffff8800a8deef40/0xe0b47d8ca5f72c82 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e0021e9 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219531.910989:0:11574:0:(ldlm_request.c:1305:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1721219531.911000:0:11574:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.158@tcp lock: ffff8800a8deef40/0xe0b47d8ca5f72c82 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e0021e9 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1721219531.911001:0:4613:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.158@tcp lock: ffff8800a9bae1c0/0xe0b47d8ca5f72c7b lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e0021e2 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219531.911005:0:11574:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.158@tcp lock: ffff8800a8deef40/0xe0b47d8ca5f72c82 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e0021e9 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1721219531.911007:0:4613:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.158@tcp lock: ffff8800a9bae1c0/0xe0b47d8ca5f72c7b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e0021e2 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1721219531.911017:0:1836:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a8983900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1836:1804827511286720:192.168.201.158@tcp:103:ldlm_bl_01.0 00000100:00100000:1.0:1721219531.911018:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9281c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1835:1804827511286784:192.168.201.158@tcp:103:ldlm_bl_07.0 00010000:00010000:0.0:1721219531.911018:0:11574:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1721219531.911019:0:11574:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1721219531.911021:0:11574:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219531.911030:0:4613:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219531.911031:0:4613:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219531.911033:0:4613:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1721219531.911755:0:1836:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a8983900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1836:1804827511286720:192.168.201.158@tcp:103:ldlm_bl_01.0 00000100:00100000:1.0:1721219531.911797:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9281c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1835:1804827511286784:192.168.201.158@tcp:103:ldlm_bl_07.0 00010000:00010000:3.0:1721219532.662872:0:9881:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1721219533.422759:0:4616:0:(mgc_request.c:609:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1721219533.422765:0:4616:0:(mgc_request.c:2064:mgc_process_log()) Process log lustre-client-ffff8800af021000 from 149 10000000:01000000:1.0:1721219533.422768:0:4616:0:(mgc_request.c:1102:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1721219533.422800:0:4616:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219533.422810:0:4616:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219533.422819:0:4616:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219533.422840:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511286848:192.168.201.158@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1721219533.422918:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5000 going to sleep for 6 seconds 00000100:00100000:1.0:1721219533.424205:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511286848:192.168.201.158@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1721219533.424222:0:4616:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219533.424231:0:4616:0:(ldlm_request.c:799:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219533.424283:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511286912:192.168.201.158@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1721219533.424311:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5000 going to sleep for 6 seconds 00000100:00100000:1.0:1721219533.425154:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511286912:192.168.201.158@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1721219533.425199:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511286976:192.168.201.158@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1721219533.425243:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5000 going to sleep for 6 seconds 00000100:00100000:1.0:1721219533.426288:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511286976:192.168.201.158@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1721219533.426505:0:28460:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28460:1804827511287040:192.168.201.158@tcp:502:llog_process_th.0 00000100:00100000:1.0:1721219533.426546:0:28460:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5000 going to sleep for 6 seconds 00000100:00100000:1.0:1721219533.427707:0:28460:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28460:1804827511287040:192.168.201.158@tcp:502:llog_process_th.0 00000020:01000000:1.0:1721219533.427728:0:28460:0:(obd_config.c:1789:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1721219533.427734:0:28460:0:(obd_config.c:1384:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1721219533.427735:0:28460:0:(obd_config.c:1460:class_process_config()) marker 96 (0x1) lustre-clilov rem lustre.test_23e.lustre-OST0000_UUID 00000020:01000000:1.0:1721219533.427739:0:28460:0:(obd_config.c:1904:class_config_llog_handler()) cmd ce022, instance name: lustre-clilov-ffff8800af021000 00000020:00000080:1.0:1721219533.427740:0:28460:0:(obd_config.c:1384:class_process_config()) processing cmd: ce022 00020000:01000000:1.0:1721219533.427748:0:28460:0:(lov_pool.c:476:lov_pool_remove()) lustre-OST0000_UUID removed from test_23e 00000100:00100000:1.0:1721219533.427761:0:28460:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28460:1804827511287104:192.168.201.158@tcp:502:llog_process_th.0 00000100:00100000:1.0:1721219533.427794:0:28460:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5000 going to sleep for 6 seconds 00000100:00100000:1.0:1721219533.428795:0:28460:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28460:1804827511287104:192.168.201.158@tcp:502:llog_process_th.0 00000020:01000000:1.0:1721219533.428812:0:28460:0:(obd_config.c:1789:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1721219533.428815:0:28460:0:(obd_config.c:1384:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1721219533.428816:0:28460:0:(obd_config.c:1460:class_process_config()) marker 96 (0x2) lustre-clilov rem lustre.test_23e.lustre-OST0000_UUID 00000040:00080000:1.0:1721219533.428819:0:28460:0:(llog.c:773:llog_process_thread()) stop processing plain 0x3:10:0 index 152 count 150 00000020:01000000:2.0:1721219533.428866:0:4616:0:(obd_config.c:2041:class_config_parse_llog()) Processed log lustre-client gen 149-151 (rc=0) 10000000:01000000:2.0:1721219533.428882:0:4616:0:(mgc_request.c:2136:mgc_process_log()) MGC192.168.201.158@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1721219533.428885:0:4616:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219533.428891:0:4616:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:01000000:2.0:1721219533.428897:0:4616:0:(mgc_request.c:609:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1721219533.428898:0:4616:0:(mgc_request.c:2064:mgc_process_log()) Process log lustre-sptlrpc-ffff8800ab4a8000 from 1 10000000:01000000:2.0:1721219533.428899:0:4616:0:(mgc_request.c:1102:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1721219533.428910:0:4616:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219533.428913:0:4616:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219533.428917:0:4616:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1721219533.428925:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a8600e40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287168:192.168.201.158@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1721219533.428956:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff8800b6162b40 going to sleep for 6 seconds 00000100:00100000:0.0:1721219533.429757:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a8600e40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287168:192.168.201.158@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1721219533.429766:0:4616:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219533.429771:0:4616:0:(ldlm_request.c:799:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1721219533.429800:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287232:192.168.201.158@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1721219533.429828:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff8800b6d7c540 going to sleep for 6 seconds 00000100:00100000:0.0:1721219533.430792:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287232:192.168.201.158@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1721219533.430809:0:4616:0:(mgc_request.c:2136:mgc_process_log()) MGC192.168.201.158@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1721219533.430811:0:4616:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219533.430817:0:4616:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219533.664781:0:11546:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1721219534.356364:0:1829:0:(events.c:361:request_in_callback()) peer: 12345-192.168.201.158@tcp (source: 12345-192.168.201.158@tcp) 00000100:00100000:3.0:1721219534.356400:0:1829:0:(events.c:361:request_in_callback()) peer: 12345-192.168.201.158@tcp (source: 12345-192.168.201.158@tcp) 00000100:00100000:3.0:1721219534.356429:0:4605:0:(service.c:2140:ptlrpc_server_handle_req_in()) got req x1804827459642560 00000100:00100000:2.0:1721219534.356429:0:11164:0:(service.c:2140:ptlrpc_server_handle_req_in()) got req x1804827459642496 00000100:00100000:3.0:1721219534.356455:0:4605:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.158@tcp, seq: 181536 00000100:00100000:3.0:1721219534.356459:0:4605:0:(service.c:2309:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a89817c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+4:21940:x1804827459642560:12345-192.168.201.158@tcp:104: 00000100:00100000:2.0:1721219534.356459:0:11164:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.158@tcp, seq: 181537 00000100:00100000:2.0:1721219534.356463:0:11164:0:(service.c:2309:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a8983900 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+4:21940:x1804827459642496:12345-192.168.201.158@tcp:104: 00010000:00010000:3.0:1721219534.356509:0:4605:0:(ldlm_lockd.c:2154:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:3.0:1721219534.356513:0:4605:0:(service.c:2358:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a89817c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+4:21940:x1804827459642560:12345-192.168.201.158@tcp:104: Request processed in 54us (113us total) trans 0 rc 0/0 00000100:00100000:3.0:1721219534.356518:0:4605:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.158@tcp, seq: 181536 00010000:00010000:2.0:1721219534.356519:0:11164:0:(ldlm_lockd.c:2154:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1721219534.356523:0:11164:0:(service.c:2358:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a8983900 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38+4:21940:x1804827459642496:12345-192.168.201.158@tcp:104: Request processed in 60us (164us total) trans 0 rc 0/0 00010000:00010000:0.0:1721219534.356534:0:11571:0:(ldlm_lockd.c:2784:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:2.0:1721219534.356535:0:11164:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.158@tcp, seq: 181537 00010000:00010000:0.0:1721219534.356539:0:11571:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219534.356547:0:11571:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a81cdf80 already unused, calling callback (ffffffffa030c5c0) 10000000:00010000:0.0:1721219534.356549:0:11571:0:(mgc_request.c:1019:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219534.356554:0:4614:0:(ldlm_lockd.c:2784:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1721219534.356554:0:11571:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219534.356556:0:4614:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:00010000:0.0:1721219534.356557:0:11571:0:(mgc_request.c:1025:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1721219534.356559:0:11571:0:(mgc_request.c:1029:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1721219534.356563:0:4614:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a9badd40 already unused, calling callback (ffffffffa030c5c0) 10000000:00010000:2.0:1721219534.356565:0:4614:0:(mgc_request.c:1019:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219534.356570:0:4614:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:00010000:2.0:1721219534.356575:0:4614:0:(mgc_request.c:1025:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:01000000:2.0:1721219534.356579:0:4614:0:(mgc_request.c:1029:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1721219534.356592:0:11571:0:(ldlm_request.c:1301:ldlm_cancel_pack()) ### packing ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219534.356593:0:4614:0:(ldlm_request.c:1301:ldlm_cancel_pack()) ### packing ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219534.356595:0:11571:0:(ldlm_request.c:1305:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1721219534.356598:0:4614:0:(ldlm_request.c:1305:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1721219534.356610:0:11571:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219534.356614:0:11571:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72ca5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e002259 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219534.356619:0:4614:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219534.356625:0:4614:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.158@tcp lock: ffff8800a9badd40/0xe0b47d8ca5f72cac lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000000 nid: local remote: 0xd4836d281e002260 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219534.356627:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1721219534.356628:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1721219534.356629:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1721219534.356633:0:1835:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1721219534.356639:0:1837:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a8603900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1837:1804827511287360:192.168.201.158@tcp:103:ldlm_bl_02.0 00010000:00010000:2.0:1721219534.356639:0:4614:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219534.356640:0:4614:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219534.356641:0:4614:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1721219534.356645:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1835:1804827511287296:192.168.201.158@tcp:103:ldlm_bl_05.0 00000100:00100000:2.0:1721219534.357358:0:1837:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a8603900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1837:1804827511287360:192.168.201.158@tcp:103:ldlm_bl_02.0 00000100:00100000:1.0:1721219534.357457:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1835:1804827511287296:192.168.201.158@tcp:103:ldlm_bl_05.0 00010000:00010000:3.0:1721219534.666828:0:11573:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1721219535.002821:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511287488:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219535.002827:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9284740 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1835:1804827511287424:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219535.002885:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9283900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511287552:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219535.002885:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511287616:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219535.004475:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511287488:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219535.004477:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9284740 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1835:1804827511287424:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219535.004483:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511287616:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219535.004488:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9283900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511287552:192.168.201.158@tcp:400:kworker/u8:1.0 00010000:00010000:2.0:1721219535.668809:0:11574:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1721219536.074751:0:4616:0:(mgc_request.c:609:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1721219536.074756:0:4616:0:(mgc_request.c:2064:mgc_process_log()) Process log lustre-client-ffff8800af021000 from 152 10000000:01000000:1.0:1721219536.074761:0:4616:0:(mgc_request.c:1102:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1721219536.074796:0:4616:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72cb3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219536.074805:0:4616:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72cb3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219536.074815:0:4616:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72cb3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219536.074836:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287680:192.168.201.158@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1721219536.074908:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e53c0 going to sleep for 6 seconds 00000100:00100000:1.0:1721219536.076427:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287680:192.168.201.158@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1721219536.076442:0:4616:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72cb3 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002332 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219536.076461:0:4616:0:(ldlm_request.c:799:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72cb3 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xd4836d281e002332 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219536.076506:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287744:192.168.201.158@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1721219536.076534:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e53c0 going to sleep for 6 seconds 00000100:00100000:1.0:1721219536.077334:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287744:192.168.201.158@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1721219536.077353:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287808:192.168.201.158@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1721219536.077385:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e53c0 going to sleep for 6 seconds 00000100:00100000:1.0:1721219536.078321:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511287808:192.168.201.158@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1721219536.078473:0:28499:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28499:1804827511287872:192.168.201.158@tcp:502:llog_process_th.0 00000100:00100000:1.0:1721219536.078505:0:28499:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e53c0 going to sleep for 6 seconds 00000100:00100000:1.0:1721219536.079773:0:28499:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28499:1804827511287872:192.168.201.158@tcp:502:llog_process_th.0 00000020:01000000:1.0:1721219536.079796:0:28499:0:(obd_config.c:1789:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1721219536.079803:0:28499:0:(obd_config.c:1384:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1721219536.079804:0:28499:0:(obd_config.c:1460:class_process_config()) marker 98 (0x1) lustre-clilov del lustre.test_23e 00000020:01000000:1.0:1721219536.079809:0:28499:0:(obd_config.c:1904:class_config_llog_handler()) cmd ce023, instance name: lustre-clilov-ffff8800af021000 00000020:00000080:1.0:1721219536.079811:0:28499:0:(obd_config.c:1384:class_process_config()) processing cmd: ce023 00000100:00100000:1.0:1721219536.079852:0:28499:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28499:1804827511287936:192.168.201.158@tcp:502:llog_process_th.0 00000100:00100000:1.0:1721219536.079878:0:28499:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e56c0 going to sleep for 6 seconds 00000100:00100000:1.0:1721219536.080757:0:28499:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job llog_process_th:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:28499:1804827511287936:192.168.201.158@tcp:502:llog_process_th.0 00000020:01000000:1.0:1721219536.080772:0:28499:0:(obd_config.c:1789:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1721219536.080775:0:28499:0:(obd_config.c:1384:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1721219536.080779:0:28499:0:(obd_config.c:1460:class_process_config()) marker 98 (0x2) lustre-clilov del lustre.test_23e 00000040:00080000:1.0:1721219536.080782:0:28499:0:(llog.c:773:llog_process_thread()) stop processing plain 0x3:10:0 index 155 count 153 00000020:01000000:2.0:1721219536.080820:0:4616:0:(obd_config.c:2041:class_config_parse_llog()) Processed log lustre-client gen 152-154 (rc=0) 10000000:01000000:2.0:1721219536.080834:0:4616:0:(mgc_request.c:2136:mgc_process_log()) MGC192.168.201.158@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1721219536.080837:0:4616:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72cb3 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002332 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219536.080843:0:4616:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.158@tcp lock: ffff8800a81cdf80/0xe0b47d8ca5f72cb3 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002332 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 10000000:01000000:2.0:1721219536.080848:0:4616:0:(mgc_request.c:609:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1721219536.080849:0:4616:0:(mgc_request.c:2064:mgc_process_log()) Process log lustre-sptlrpc-ffff8800ab4a8000 from 1 10000000:01000000:2.0:1721219536.080850:0:4616:0:(mgc_request.c:1102:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1721219536.080860:0:4616:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a9bacfc0/0xe0b47d8ca5f72cba lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219536.080863:0:4616:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.158@tcp lock: ffff8800a9bacfc0/0xe0b47d8ca5f72cba lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1721219536.080867:0:4616:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.158@tcp lock: ffff8800a9bacfc0/0xe0b47d8ca5f72cba lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1721219536.080875:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a8603900 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511288000:192.168.201.158@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1721219536.080900:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff8800b61620c0 going to sleep for 6 seconds 00000100:00100000:0.0:1721219536.081566:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a8603900 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511288000:192.168.201.158@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1721219536.081575:0:4616:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.158@tcp lock: ffff8800a9bacfc0/0xe0b47d8ca5f72cba lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002339 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219536.081581:0:4616:0:(ldlm_request.c:799:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.158@tcp lock: ffff8800a9bacfc0/0xe0b47d8ca5f72cba lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xd4836d281e002339 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1721219536.081607:0:4616:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511288064:192.168.201.158@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1721219536.081634:0:4616:0:(client.c:2492:ptlrpc_set_wait()) set ffff8800b6d7cc00 going to sleep for 6 seconds 00000100:00100000:0.0:1721219536.082368:0:4616:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:4616:1804827511288064:192.168.201.158@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1721219536.082384:0:4616:0:(mgc_request.c:2136:mgc_process_log()) MGC192.168.201.158@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1721219536.082387:0:4616:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.158@tcp lock: ffff8800a9bacfc0/0xe0b47d8ca5f72cba lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002339 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219536.082396:0:4616:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.158@tcp lock: ffff8800a9bacfc0/0xe0b47d8ca5f72cba lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xd4836d281e002339 expref: -99 pid: 4616 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219536.670711:0:4613:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219537.672797:0:9881:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000080:3.0:1721219538.579900:0:28775:0:(module.c:211:libcfs_ioctl()) libcfs ioctl cmd 3221775648 00000001:02000400:3.0:1721219538.579903:0:28775:0:(debug.c:728:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-pfl test 23f: Append with low on space: repeat and remove EXT comp ========================================================== 08:32:18 (1721219538) 00000080:00200000:0.0:1721219538.668181:0:28638:0:(file.c:5211:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8),name=/ 00000002:00010000:0.0:1721219538.668197:0:28638:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:0.0:1721219538.668209:0:28638:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.668220:0:28638:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00010000:0.0:1721219538.668229:0:28638:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) 00000080:00010000:0.0:1721219538.668231:0:28638:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) for lock 0xe0b47d8ca5f7255f 00000080:00010000:0.0:1721219538.668237:0:28638:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f7255f from it ffff8800a9027af8 00010000:00010000:0.0:1721219538.668239:0:28638:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.668245:0:28638:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1721219538.668259:0:28638:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8), inode mode 41ed mask 1 00000080:00200000:0.0:1721219538.668264:0:28638:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=d23f.sanity-pfl, dir=[0x200000007:0x1:0x0](ffff8800b05e80d8), flags=1 00000080:00200000:0.0:1721219538.668267:0:28638:0:(namei.c:833:ll_lookup_it()) VFS Op:name=d23f.sanity-pfl, dir=[0x200000007:0x1:0x0](ffff8800b05e80d8), intent=getattr 00000002:00010000:0.0:1721219538.668272:0:28638:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: d23f.sanity-pfl,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1721219538.668297:0:28638:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.668308:0:28638:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.668314:0:28638:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1721219538.668331:0:28638:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job bash:2df8dc7e-94ba-4991-a575-93aaf526cb45:28638:1804827511288128:192.168.201.158@tcp:101:bash.0 00000100:00100000:0.0:1721219538.668400:0:28638:0:(client.c:2492:ptlrpc_set_wait()) set ffff8800b6d7c3c0 going to sleep for 80 seconds 00000100:00100000:0.0:1721219538.670221:0:28638:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job bash:2df8dc7e-94ba-4991-a575-93aaf526cb45:28638:1804827511288128:192.168.201.158@tcp:101:bash.0 00010000:00010000:0.0:1721219538.670230:0:28638:0:(ldlm_request.c:644:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670238:0:28638:0:(ldlm_request.c:565:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670242:0:28638:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670247:0:28638:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670253:0:28638:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670258:0:28638:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a8deeac0 already unused, calling callback (ffffffffa0fa2120) 00010000:00010000:0.0:1721219538.670260:0:28638:0:(ldlm_request.c:1508:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670274:0:28638:0:(ldlm_request.c:1518:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670279:0:28638:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670285:0:28638:0:(ldlm_request.c:1251:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1721219538.670288:0:28638:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670293:0:28638:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deeac0/0xe0b47d8ca5f72cc1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1721219538.670306:0:28638:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@ffff8800a9286d40 x1804827511288128/t0(0) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 584/640 e 0 to 0 dl 1721219619 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'bash.0' 00010000:00010000:0.0:1721219538.670323:0:28638:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns ffff8800b6ecf000 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:0.0:1721219538.670374:0:28638:0:(file.c:5211:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8),name=/ 00000002:00010000:0.0:1721219538.670380:0:28638:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1721219538.670384:0:28638:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns ffff8800b6ecf000 type 13 mode 30 res 8589934599/1 (0 0) 00010000:00010000:0.0:1721219538.670397:0:28638:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670403:0:28638:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.670408:0:28638:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1721219538.670416:0:28638:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job bash:2df8dc7e-94ba-4991-a575-93aaf526cb45:28638:1804827511288192:192.168.201.158@tcp:101:bash.0 00000100:00100000:0.0:1721219538.670448:0:28638:0:(client.c:2492:ptlrpc_set_wait()) set ffff8800b6d7c3c0 going to sleep for 80 seconds 00000100:00100000:0.0:1721219538.671896:0:28638:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job bash:2df8dc7e-94ba-4991-a575-93aaf526cb45:28638:1804827511288192:192.168.201.158@tcp:101:bash.0 00010000:00010000:0.0:1721219538.671904:0:28638:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671912:0:28638:0:(ldlm_request.c:745:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671919:0:28638:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671926:0:28638:0:(ldlm_request.c:799:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671931:0:28638:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671935:0:28638:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671939:0:28638:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1721219538.671945:0:28638:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff8800a9286d40 x1804827511288192/t0(0) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 576/688 e 0 to 0 dl 1721219619 ref 1 fl Complete:RQU/0/0 rc 0/0 job:'bash.0' 00000002:00010000:0.0:1721219538.671952:0:28638:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671958:0:28638:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns ffff8800b6ecf000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1721219538.671977:0:28638:0:(llite_lib.c:2756:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 14848 00000080:00010000:0.0:1721219538.671981:0:28638:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) for lock 0xe0b47d8ca5f72cc8 00000080:00010000:0.0:1721219538.671984:0:28638:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) 00000080:00010000:0.0:1721219538.671986:0:28638:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f72cc8 from it ffff8800a9027cf8 00010000:00010000:0.0:1721219538.671988:0:28638:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.671992:0:28638:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.673932:0:28799:0:(file.c:5211:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8),name=/ 00000002:00010000:1.0:1721219538.673940:0:28799:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1721219538.673944:0:28799:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.673951:0:28799:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1721219538.673955:0:28799:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) 00000080:00010000:1.0:1721219538.673957:0:28799:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) for lock 0xe0b47d8ca5f7255f 00000080:00010000:1.0:1721219538.673959:0:28799:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f7255f from it ffff8800a53c3b58 00010000:00010000:1.0:1721219538.673960:0:28799:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.673965:0:28799:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.673970:0:28799:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8), inode mode 41ed mask 1 00000080:00200000:1.0:1721219538.673973:0:28799:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=d23f.sanity-pfl, dir=[0x200000007:0x1:0x0](ffff8800b05e80d8), flags=1536 00000080:00200000:1.0:1721219538.673975:0:28799:0:(file.c:5211:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8),name=/ 00000002:00010000:1.0:1721219538.673976:0:28799:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1721219538.673978:0:28799:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.673982:0:28799:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1721219538.673985:0:28799:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) 00000080:00010000:1.0:1721219538.673989:0:28799:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) for lock 0xe0b47d8ca5f7255f 00000080:00010000:1.0:1721219538.673991:0:28799:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f7255f from it ffff8800a53c3bf8 00010000:00010000:1.0:1721219538.673991:0:28799:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.673994:0:28799:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.673998:0:28799:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8), inode mode 41ed mask 3 00000080:00200000:1.0:1721219538.674000:0:28799:0:(file.c:5211:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8),name=/ 00000002:00010000:1.0:1721219538.674002:0:28799:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1721219538.674003:0:28799:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.674006:0:28799:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1721219538.674009:0:28799:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) 00000080:00010000:1.0:1721219538.674010:0:28799:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) for lock 0xe0b47d8ca5f7255f 00000080:00010000:1.0:1721219538.674011:0:28799:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f7255f from it ffff8800a53c3da0 00010000:00010000:1.0:1721219538.674012:0:28799:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.674014:0:28799:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.674017:0:28799:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8), inode mode 41ed mask 3 00000080:00200000:1.0:1721219538.674019:0:28799:0:(namei.c:1899:ll_mkdir()) VFS Op:name=d23f.sanity-pfl, dir=[0x200000007:0x1:0x0](ffff8800b05e80d8) 00010000:00010000:1.0:1721219538.674032:0:28799:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.674041:0:28799:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.674059:0:28799:0:(ldlm_request.c:1301:ldlm_cancel_pack()) ### packing ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.674062:0:28799:0:(ldlm_request.c:1305:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1721219538.674062:0:28799:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8deef40/0xe0b47d8ca5f72cc8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0xd4836d281e00235c expref: -99 pid: 28638 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1721219538.674085:0:28799:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800af021000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1721219538.674089:0:28799:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a862c280 x1804827511288320/t0(0) o36->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 512/448 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1721219538.674098:0:28799:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job mkdir:2df8dc7e-94ba-4991-a575-93aaf526cb45:28799:1804827511288320:192.168.201.158@tcp:36:mkdir.0 00000100:00100000:1.0:1721219538.674125:0:28799:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5a80 going to sleep for 80 seconds 00010000:00010000:1.0:1721219538.674772:0:11546:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1721219538.676812:0:28799:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862c280 pname:cluuid:pid:xid:nid:opc:job mkdir:2df8dc7e-94ba-4991-a575-93aaf526cb45:28799:1804827511288320:192.168.201.158@tcp:36:mkdir.0 00000080:00200000:1.0:1721219538.676828:0:28799:0:(llite_lib.c:2834:ll_read_inode2()) VFS Op:inode=[0x200000401:0x6a25:0x0](ffff8800b065c698) 00000080:00200000:1.0:1721219538.676831:0:28799:0:(llite_lib.c:2756:ll_update_inode()) inode=[0x200000401:0x6a25:0x0], updating i_size 512 00000080:00200000:1.0:1721219538.679346:0:28800:0:(file.c:5211:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8),name=/ 00000002:00010000:1.0:1721219538.679352:0:28800:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1721219538.679356:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.679362:0:28800:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1721219538.679372:0:28800:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) 00000080:00010000:1.0:1721219538.679373:0:28800:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) for lock 0xe0b47d8ca5f7255f 00000080:00010000:1.0:1721219538.679376:0:28800:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f7255f from it ffff8800a85a7af8 00010000:00010000:1.0:1721219538.679377:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.679381:0:28800:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.679386:0:28800:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8), inode mode 41ed mask 1 00000080:00200000:1.0:1721219538.679389:0:28800:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=d23f.sanity-pfl, dir=[0x200000007:0x1:0x0](ffff8800b05e80d8), flags=16 00000080:00200000:1.0:1721219538.679391:0:28800:0:(namei.c:833:ll_lookup_it()) VFS Op:name=d23f.sanity-pfl, dir=[0x200000007:0x1:0x0](ffff8800b05e80d8), intent=0 00000002:00010000:1.0:1721219538.679394:0:28800:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: d23f.sanity-pfl,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1721219538.679406:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.679410:0:28800:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.679413:0:28800:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219538.679421:0:28800:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862b900 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288384:192.168.201.158@tcp:101:lfs.0 00000100:00100000:1.0:1721219538.679448:0:28800:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5a80 going to sleep for 80 seconds 00000100:00100000:1.0:1721219538.681149:0:28800:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862b900 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288384:192.168.201.158@tcp:101:lfs.0 00010000:00010000:1.0:1721219538.681157:0:28800:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681169:0:28800:0:(ldlm_request.c:732:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 4/1,0 mode: --/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681172:0:28800:0:(ldlm_request.c:745:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 4/1,0 mode: --/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681176:0:28800:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681179:0:28800:0:(ldlm_request.c:799:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681183:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 5/2,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681186:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 5/2,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681189:0:28800:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1721219538.681193:0:28800:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=10 disposition=b, status=0 req@ffff8800a862b900 x1804827511288384/t0(0) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 584/640 e 0 to 0 dl 1721219619 ref 1 fl Complete:RQU/0/0 rc 0/0 job:'lfs.0' 00000002:00010000:1.0:1721219538.681198:0:28800:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681202:0:28800:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns ffff8800b6ecf000 type 13 mode 4 res 8589935617/27173 (0 0) 00000080:00200000:1.0:1721219538.681207:0:28800:0:(llite_lib.c:2756:ll_update_inode()) inode=[0x200000401:0x6a25:0x0], updating i_size 512 00000080:00200000:1.0:1721219538.681211:0:28800:0:(namei.c:157:ll_iget()) got inode: [0x200000401:0x6a25:0x0](ffff8800b065c698): rc = 0 00000080:00010000:1.0:1721219538.681214:0:28800:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000401:0x6a25:0x0](ffff8800b065c698) for lock 0xe0b47d8ca5f72ccf 00000080:00010000:1.0:1721219538.681217:0:28800:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000401:0x6a25:0x0](ffff8800b065c698) 00000080:00010000:1.0:1721219538.681219:0:28800:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f72ccf from it ffff8800a85a7a70 00010000:00010000:1.0:1721219538.681220:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681223:0:28800:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce1c0/0xe0b47d8ca5f72ccf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x6a25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e002386 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.681234:0:28800:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000401:0x6a25:0x0](ffff8800b065c698), inode mode 41ed mask 1 00000080:00200000:1.0:1721219538.681238:0:28800:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=f23f.sanity-pfl, dir=[0x200000401:0x6a25:0x0](ffff8800b065c698), flags=0 00000080:00200000:1.0:1721219538.681239:0:28800:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f23f.sanity-pfl, dir=[0x200000401:0x6a25:0x0](ffff8800b065c698), intent=getattr 00000002:00010000:1.0:1721219538.681243:0:28800:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f23f.sanity-pfl,[0x0:0x0:0x0]) in obj [0x200000401:0x6a25:0x0], intent: getattr flags 0 00010000:00010000:1.0:1721219538.681254:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681257:0:28800:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.681261:0:28800:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219538.681267:0:28800:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862b900 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288448:192.168.201.158@tcp:101:lfs.0 00000100:00100000:1.0:1721219538.681294:0:28800:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5a80 going to sleep for 80 seconds 00000100:00100000:1.0:1721219538.682750:0:28800:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862b900 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288448:192.168.201.158@tcp:101:lfs.0 00010000:00010000:1.0:1721219538.682757:0:28800:0:(ldlm_request.c:644:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 4/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682769:0:28800:0:(ldlm_request.c:565:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 4/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682773:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 4/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682776:0:28800:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682779:0:28800:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682782:0:28800:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a81ce400 already unused, calling callback (ffffffffa0fa2120) 00010000:00010000:1.0:1721219538.682784:0:28800:0:(ldlm_request.c:1508:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682786:0:28800:0:(ldlm_request.c:1518:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682789:0:28800:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 5/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682792:0:28800:0:(ldlm_request.c:1251:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1721219538.682793:0:28800:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682796:0:28800:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81ce400/0xe0b47d8ca5f72cd6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1721219538.682807:0:28800:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@ffff8800a862b900 x1804827511288448/t0(0) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 584/640 e 0 to 0 dl 1721219619 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'lfs.0' 00010000:00010000:1.0:1721219538.682817:0:28800:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns ffff8800b6ecf000 type 13 mode 30 res 8589935617/27173 (0 0) 00000080:00200000:1.0:1721219538.682839:0:28800:0:(file.c:5211:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8),name=/ 00000002:00010000:1.0:1721219538.682842:0:28800:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1721219538.682845:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682850:0:28800:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1721219538.682853:0:28800:0:(dcache.c:296:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) 00000080:00010000:1.0:1721219538.682854:0:28800:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8800b05e80d8) for lock 0xe0b47d8ca5f7255f 00000080:00010000:1.0:1721219538.682856:0:28800:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f7255f from it ffff8800a85a7bb0 00010000:00010000:1.0:1721219538.682857:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682860:0:28800:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a8dedb00/0xe0b47d8ca5f7255f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e000047 expref: -99 pid: 10237 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.682865:0:28800:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8800b05e80d8), inode mode 41ed mask 1 00000080:00200000:1.0:1721219538.682867:0:28800:0:(dcache.c:312:ll_revalidate_dentry()) VFS Op:name=d23f.sanity-pfl, flags=17 00000080:00200000:1.0:1721219538.682868:0:28800:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000401:0x6a25:0x0](ffff8800b065c698), inode mode 41ed mask 1 00000080:00200000:1.0:1721219538.682872:0:28800:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000401:0x6a25:0x0](ffff8800b065c698), inode mode 41ed mask 3 00000080:00200000:1.0:1721219538.682873:0:28800:0:(namei.c:1176:ll_atomic_open()) VFS Op:name=f23f.sanity-pfl, dir=[0x200000401:0x6a25:0x0](ffff8800b065c698), file ffff88012a754600, open_flags a141, mode 81b6 opened 0 00000080:00200000:1.0:1721219538.682876:0:28800:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f23f.sanity-pfl, dir=[0x200000401:0x6a25:0x0](ffff8800b065c698), intent=open|creat 00000002:00010000:1.0:1721219538.682886:0:28800:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f23f.sanity-pfl,[0x200000401:0x6a26:0x0]) in obj [0x200000401:0x6a25:0x0], intent: open|creat flags 0120502 00010000:00010000:1.0:1721219538.682900:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6a25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.682904:0:28800:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1721219538.682908:0:28800:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800af021000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1721219538.682910:0:28800:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a862b900 x1804827511288576/t0(0) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 528/66720 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:1.0:1721219538.682914:0:28800:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219538.682921:0:28800:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a862b900 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288576:192.168.201.158@tcp:101:lfs.0 00000100:00100000:1.0:1721219538.682988:0:28800:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5a80 going to sleep for 80 seconds 00000100:00100000:1.0:1721219538.685063:0:28800:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a862b900 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288576:192.168.201.158@tcp:101:lfs.0 00010000:00010000:1.0:1721219538.685084:0:28800:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### server returned different mode CR ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 4/0,1 mode: --/CW res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685092:0:28800:0:(ldlm_request.c:732:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 4/0,1 mode: --/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685095:0:28800:0:(ldlm_request.c:745:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 4/0,1 mode: --/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685099:0:28800:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 4/0,1 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685102:0:28800:0:(ldlm_request.c:799:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 4/0,1 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685123:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 5/1,1 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685127:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 5/1,1 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685130:0:28800:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1721219538.685134:0:28800:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=3 disposition=8000037, status=0 req@ffff8800a862b900 x1804827511288576/t12884902050(12884902050) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 528/600 e 0 to 0 dl 1721219619 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000002:00100000:1.0:1721219538.685140:0:28800:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a862b900 x1804827511288576/t12884902050(12884902050) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 528/600 e 0 to 0 dl 1721219619 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000002:00010000:1.0:1721219538.685145:0:28800:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685149:0:28800:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns ffff8800b6ecf000 type 13 mode 16 res 8589935617/27174 (0 0) 00000080:00200000:1.0:1721219538.685155:0:28800:0:(namei.c:1405:ll_create_it()) VFS Op:name=f23f.sanity-pfl, dir=[0x200000401:0x6a25:0x0](ffff8800b065c698), intent=open|creat 00000080:00200000:1.0:1721219538.685162:0:28800:0:(llite_lib.c:2834:ll_read_inode2()) VFS Op:inode=[0x200000401:0x6a26:0x0](ffff8800b065bed8) 00000080:00200000:1.0:1721219538.685164:0:28800:0:(llite_lib.c:2756:ll_update_inode()) inode=[0x200000401:0x6a26:0x0], updating i_size 0 00000080:00200000:1.0:1721219538.685185:0:28800:0:(file.c:5862:ll_layout_conf()) [0x200000401:0x6a26:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1721219538.685187:0:28800:0:(namei.c:1372:ll_create_node()) setting l_ast_data to inode [0x200000401:0x6a26:0x0](ffff8800b065bed8) 00000080:00010000:1.0:1721219538.685189:0:28800:0:(llite_internal.h:1688:ll_set_lock_data()) setting l_data to inode [0x200000401:0x6a26:0x0](ffff8800b065bed8) for lock 0xe0b47d8ca5f72cdd 00000080:00200000:1.0:1721219538.685194:0:28800:0:(file.c:833:ll_file_open()) VFS Op:inode=[0x200000401:0x6a26:0x0](ffff8800b065bed8), flags 120501 00000002:00100000:1.0:1721219538.685200:0:28800:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8800a862b900 x1804827511288576/t12884902050(12884902050) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 528/600 e 0 to 0 dl 1721219619 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000080:00010000:1.0:1721219538.685234:0:28800:0:(dcache.c:196:ll_intent_drop_lock()) releasing lock with cookie 0xe0b47d8ca5f72cdd from it ffff8800aa948180 00010000:00010000:1.0:1721219538.685235:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685239:0:28800:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1721219538.685245:0:28800:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000401:0x6a26:0x0](ffff8800b065bed8), inode mode 81a4 mask 40 00000080:00200000:1.0:1721219538.685270:0:28800:0:(file.c:5646:ll_inode_permission()) VFS Op:inode=[0x200000401:0x6a26:0x0](ffff8800b065bed8), inode mode 81a4 mask 2 00000080:00200000:1.0:1721219538.685275:0:28800:0:(xattr.c:358:ll_xattr_set()) VFS Op:inode=[0x200000401:0x6a26:0x0](ffff8800b065bed8), xattr lov 00000002:00010000:1.0:1721219538.685280:0:28800:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000401:0x6a26:0x0]) in obj [0x200000401:0x6a25:0x0], intent: open flags 040000002 00010000:00010000:1.0:1721219538.685303:0:28800:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219538.685306:0:28800:0:(ldlm_request.c:1029:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1721219538.685310:0:28800:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800af021000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1721219538.685313:0:28800:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a86297c0 x1804827511288704/t0(0) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 1104/66720 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:1.0:1721219538.685316:0:28800:0:(ldlm_request.c:1117:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1721219538.685323:0:28800:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a86297c0 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288704:192.168.201.158@tcp:101:lfs.0 00000100:00100000:1.0:1721219538.685362:0:28800:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5b40 going to sleep for 80 seconds 00000100:00100000:2.0:1721219538.686361:0:1829:0:(events.c:361:request_in_callback()) peer: 12345-192.168.201.158@tcp (source: 12345-192.168.201.158@tcp) 00000100:00100000:3.0:1721219538.686410:0:4606:0:(service.c:2140:ptlrpc_server_handle_req_in()) got req x1804827459644928 00000100:00100000:3.0:1721219538.686445:0:4606:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.158@tcp, seq: 181538 00000100:00100000:3.0:1721219538.686451:0:4606:0:(service.c:2309:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a8600000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:2df8dc7e-94ba-4991-a575-93aaf526cb45+5:27126:x1804827459644928:12345-192.168.201.158@tcp:104: 00010000:00010000:3.0:1721219538.686487:0:4606:0:(ldlm_lockd.c:2154:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1721219538.686491:0:4606:0:(service.c:2358:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a8600000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:2df8dc7e-94ba-4991-a575-93aaf526cb45+5:27126:x1804827459644928:12345-192.168.201.158@tcp:104: Request processed in 40us (133us total) trans 0 rc 0/0 00000100:00100000:3.0:1721219538.686500:0:4606:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.158@tcp, seq: 181538 00010000:00010000:0.0:1721219538.686514:0:11571:0:(ldlm_lockd.c:2784:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1721219538.686518:0:11571:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.686526:0:11571:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a81cc6c0 already unused, calling callback (ffffffffa0fa2120) 00010000:00010000:0.0:1721219538.686528:0:11571:0:(ldlm_request.c:1508:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.686533:0:11571:0:(ldlm_request.c:1518:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.686538:0:11571:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x428400000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1721219538.686550:0:11571:0:(vvp_object.c:137:vvp_conf_set()) [0x200000401:0x6a26:0x0]: losing layout lock 00010000:00010000:0.0:1721219538.686565:0:11571:0:(ldlm_request.c:1301:ldlm_cancel_pack()) ### packing ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.686569:0:11571:0:(ldlm_request.c:1305:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1721219538.686590:0:11571:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.686599:0:11571:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc6c0/0xe0b47d8ca5f72cdd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6a26:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000000 nid: local remote: 0xd4836d281e0023b7 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1721219538.686606:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1721219538.686607:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1721219538.686608:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1721219538.686613:0:1834:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1721219538.686619:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511288768:192.168.201.158@tcp:103:ldlm_bl_05.0 00000100:00100000:0.0:1721219538.687499:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511288768:192.168.201.158@tcp:103:ldlm_bl_05.0 00010000:00010000:2.0:1721219539.676807:0:4614:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1721219540.010849:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511288896:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219540.010855:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511288832:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219540.010920:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9285580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511288960:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219540.010920:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9283900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289024:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219540.012170:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511288896:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219540.012177:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9283900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289024:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219540.012192:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9286d40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511288832:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219540.012200:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9285580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511288960:192.168.201.158@tcp:400:kworker/u8:1.0 00010000:00010000:3.0:1721219540.678864:0:11573:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219541.680875:0:11574:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219542.682831:0:4613:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219543.684810:0:9881:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1721219544.686784:0:11546:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1721219545.018798:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289152:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219545.018831:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9283900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511289088:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219545.018869:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289280:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219545.018877:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9284740 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511289216:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219545.020187:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9283900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511289088:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219545.020192:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289152:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219545.020195:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9284740 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511289216:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219545.020198:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289280:192.168.201.158@tcp:400:kworker/u8:1.0 00010000:00010000:2.0:1721219545.688839:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219546.690839:0:4614:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219547.692845:0:11573:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219548.694860:0:11574:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219549.696805:0:4613:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1721219550.026839:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289408:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219550.026880:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511289344:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219550.026931:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9281c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511289472:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219550.026932:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9282ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289536:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219550.028250:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289408:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219550.028256:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9282ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289536:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219550.028308:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9280000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511289344:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219550.028315:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9281c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511289472:192.168.201.158@tcp:400:kworker/u8:1.0 00010000:00010000:3.0:1721219550.698840:0:9881:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1721219551.700845:0:11546:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219552.702881:0:11571:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1721219553.704846:0:4614:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219554.706821:0:11573:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1721219555.034874:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289664:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219555.034877:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9282ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511289600:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219555.034941:0:1834:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9280980 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511289728:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219555.034941:0:1835:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a9283440 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289792:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219555.036242:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a92863c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289664:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1721219555.036248:0:1835:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9283440 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:2df8dc7e-94ba-4991-a575-93aaf526cb45:1835:1804827511289792:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219555.036257:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9282ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895aa51d-428d-41b3-8fd7-a0b7d3b7fe38:1834:1804827511289600:192.168.201.158@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1721219555.036265:0:1834:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a9280980 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:2df8dc7e-94ba-4991-a575-93aaf526cb45:1834:1804827511289728:192.168.201.158@tcp:400:kworker/u8:1.0 00010000:00010000:2.0:1721219555.708853:0:11574:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219556.710879:0:4613:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1721219557.712801:0:9881:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1721219558.688185:0:28800:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a86297c0 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511288704:192.168.201.158@tcp:101:lfs.0 00010000:00010000:1.0:1721219558.688215:0:28800:0:(ldlm_request.c:644:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 4/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688227:0:28800:0:(ldlm_request.c:565:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 4/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688233:0:28800:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 4/1,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688238:0:28800:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688244:0:28800:0:(ldlm_lockd.c:1874:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688250:0:28800:0:(ldlm_lockd.c:1888:ldlm_handle_bl_callback()) Lock ffff8800a81cc900 already unused, calling callback (ffffffffa0fa2120) 00010000:00010000:1.0:1721219558.688254:0:28800:0:(ldlm_request.c:1508:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688259:0:28800:0:(ldlm_request.c:1518:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688265:0:28800:0:(ldlm_request.c:1235:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 5/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688271:0:28800:0:(ldlm_request.c:1251:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1721219558.688274:0:28800:0:(ldlm_lockd.c:1898:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1721219558.688277:0:28800:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8800af021000 lock: ffff8800a81cc900/0xe0b47d8ca5f72ce4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x6a25:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 28800 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1721219558.688303:0:28800:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=1 disposition=3, status=-28 req@ffff8800a86297c0 x1804827511288704/t0(0) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 1104/600 e 0 to 0 dl 1721219619 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'lfs.0' 00000080:00200000:1.0:1721219558.688315:0:28800:0:(file.c:694:ll_intent_file_open()) lock enqueue: err: -28 00000080:00200000:1.0:1721219558.688381:0:28800:0:(file.c:406:ll_file_release()) VFS Op:inode=[0x200000401:0x6a26:0x0](ffff8800b065bed8) 00000002:00100000:1.0:1721219558.688400:0:28800:0:(mdc_request.c:944:mdc_close()) @@@ matched open req@ffff8800a862b900 x1804827511288576/t12884902050(12884902050) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 528/600 e 0 to 0 dl 1721219619 ref 1 fl Complete:RPQU/4/ffffffff rc 0/-1 job:'lfs.0' 00000020:00100000:1.0:1721219558.688416:0:28800:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8800af021000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1721219558.688419:0:28800:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8800a86297c0 x1804827511289920/t0(0) o35->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:23/10 lens 392/1080 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1721219558.688431:0:28800:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@ffff8800a86297c0 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511289920:192.168.201.158@tcp:35:lfs.0 00000100:00100000:1.0:1721219558.688475:0:28800:0:(client.c:2492:ptlrpc_set_wait()) set ffff88012d7e5a80 going to sleep for 6 seconds 00000100:00100000:1.0:1721219558.689489:0:28800:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@ffff8800a86297c0 pname:cluuid:pid:xid:nid:opc:job lfs:2df8dc7e-94ba-4991-a575-93aaf526cb45:28800:1804827511289920:192.168.201.158@tcp:35:lfs.0 00000002:00100000:1.0:1721219558.689500:0:28800:0:(mdc_request.c:854:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8800a862b900 x1804827511288576/t12884902050(12884902050) o101->lustre-MDT0000-mdc-ffff8800af021000@192.168.201.158@tcp:12/10 lens 528/600 e 0 to 0 dl 1721219619 ref 1 fl Complete:RQU/4/ffffffff rc 0/-1 job:'lfs.0' 00010000:00010000:0.0:1721219558.714719:0:11546:0:(ldlm_lockd.c:2787:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000080:1.0:1721219558.987561:0:28947:0:(module.c:211:libcfs_ioctl()) libcfs ioctl cmd 3221775648 00000001:02000400:1.0:1721219558.987564:0:28947:0:(debug.c:728:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-pfl test_23f: @@@@@@ FAIL: Create /mnt/lustre/d23f.sanity-pfl/f23f.sanity-pfl failed Debug log: 541 lines, 541 kept, 0 dropped, 0 bad.